Installing Treviz

Introduction

This guide will help you deploy the various modules of Treviz.

Treviz is made of three main components, which all have their own dependencies:

Each component can be installed on a separate server. For small environments, a single machine should however be enough. In this tutorial, we will focus on installing all the components on a single server.

The installation can be made in various ways, we will here describe two of them:

  • Setting up the platform using Docker, docker-compose, and Traefik
  • Setting up the platform the old-fashioned way

Docker Installation

Requirements

You will need:

With those three elements, setting up Treviz is a piece of cake.

Setting up the API

Clone the backend project using the git CLI, and navigate in the project:

cd /path/to/treviz
git clone https://gitlab.com/treviz/treviz-back.git treviz-back
cd treviz-back

Create a .env file, and copy the following configuration, by adapting its values to suit your needs. You mainly need to specify the SMTP parameters, as well as the frontend and backend hosts.

MYSQL_HOST=db
MYSQL_PORT=3306
MYSQL_DATABASE=treviz
MYSQL_ROOT_PASSWORD=<mysql-root-passorwd>
MYSQL_USER=treviz
MYSQL_PASSWORD=<mysql-passorwd>
SMTP_HOST=pro1.mail.ovh.net
SMTP_PORT=587
SMTP_USERNAME=bhuber
SMTP_PASSWORD=test
FRONTEND_HOST=127.0.0.1:4200
BACKEND_HOST=127.0.0.1:8080
RSA_PASSPHRASE=ceciestunttest
RABBITMQ_HOST=rabbitmq
RABBITMQ_PORT=5672
RABBITMQ_DEFAULT_USER=treviz
RABBITMQ_DEFAULT_PASS=treviz

The docker-compose file already specifies all the dependencies that will be needed for the backend, as well as the flags for Traefik. You can check the configuration is correctly completed by prompting:

sudo docker-compose config

If you are satisfied with it, you can now safely launch the Treviz API by running:

sudo docker-compose up -d

Wait a few minutes, and the API should be up and running !

Setting up the event micro-service

The event micro-service is not essential for Treviz to work, but it is required to handle real-time messages in the chat. Much like for the backend, clone it:

cd /path/to/treviz
git clone https://gitlab.com/treviz/treviz-events.git treviz-events
cd treviz-events

Create a .env file, and copy the following lines. You should only need to update the EVENTS_URL parameter to fit the future actual url of the service.

APP_PORT=3000
RABBITMQ_PROTOCOL=amqp
RABBITMQ_HOST=rabbitmq
RABBITMQ_PORT=5672
RABBITMQ_USERNAME=treviz
RABBITMQ_PASSWORD=treviz
EVENTS_URL=events.treviz.org
NODE_ENV=production
PUBLIC_KEY_PATH=./public.pem

In order to authorize impending connections, the service needs to know the public key that was used to generate the authentication tokens. This one should be localized in the treviz backend api container you previously launched.

Copy it here, and launch the service by running:

sudo docker cp api:/var/www/api/var/jwt/public.pem .
sudo docker-compose up -d

The event service should now be running.

Setting up the Frontend

The frontend is even simpler to launch, since it only needs a few parameters, that is to say the url of the services.

cd /path/to/treviz
git clone https://gitlab.com/treviz/treviz-front.git treviz-front
cd treviz-front

Create a .env file with the host names of the three services. For instance:

API_HOST=api.preprod.treviz.org
APP_HOST=app.preprod.treviz.org
NOTIFICATION_HOST=events.preprod.treviz.org

Finally, you can safely run:

sudo docker-compose up -d

Wait a few minutes for the dependencies to be fetched and the application builded, and navigate to the url you specified. You should see the application login page. Congratulation, you have set up your own Treviz instance !

To continue, please refer to the Administration guide.

Bare-bone installation

Requirements

In order to properly work, Treviz must be installed on a server that complies with the following requirements:

Software:

  • NodeJS >= 10.0.0 with NPM
  • PHP 7.2 or higher
  • Composer
  • MySQL 5.7 or higher
  • Angular-Cli to build the frontend
  • Git

Hardware:

  • At least 1GB of RAM
  • At least 1GB of disk space. Treviz does not take much space by default, but the database and additional files might need more space.

You will also need a HTTP Server. We recommand using Nginx, since it will also allow us to proxy our websocket and secure it using TLS.

Setting up the environment

Before you can install the Treviz components, we first need to make sure the environment complies with the previous requirements. For this demo, we will work on a Debian 8 "Jessie".

# add the sources for php 7.1 and RabbitMQ
sudo apt-get install -y apt-transport-https lsb-release ca-certificates erlang-nox
sudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg
echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" >> php.list
echo "deb https://dl.bintray.com/rabbitmq/debian $(lsb_release -sc) erlang" >> php.list
sudo mv php.list /etc/apt/sources.list.d/php.list

sudo apt-get update

# install nginx
sudo apt-get install -y nginx

# install Node.js
curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
sudo apt-get install -y nodejs
sudo apt-get install -y build-essential

# install php7.1, php7.1-fpm and other dependencies
sudo apt-get install -y php7.1 php7.1-opcache libapache2-mod-php7.1 php7.1-mysql php7.1-curl php7.1-json php7.1-gd php7.1-mcrypt php7.1-intl php7.1-mbstring php7.1-xml php7.1-zip php7.1-fpm php7.1-readline

# install MySQL
# You will be asked to prompt a password for "root" account. Note it down, you'll need it later.
sudo apt-get install -y mysql-server mysql-client

# Install RabbitMQ
sudo apt-key adv --keyserver "hkps.pool.sks-keyservers.net" --recv-keys "0x6B73A36E6026DFCA"
wget -O - "https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc" | sudo apt-key add -
sudo apt-get install -y rabbitmq-server
sudo service rabbitmq-server start

# Install angular-cli
sudo npm install -g @angular/cli

# install Composer
php -r "copy('https://getcomposer.org/installer', '/tmp/composer-setup.php');"
php -r "if (hash_file('SHA384', '/tmp/composer-setup.php') === '544e09ee996cdf60ece3804abc52599c22b1f40f4323403c44d44fdfdd586475ca9813a858088ffbc1f233e9b180f061') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"
sudo php /tmp/composer-setup.php --install-dir=/usr/local/bin --filename=composer

# install git
sudo apt-get install -y git

We will now create a database for Treviz to connect to, and create a specific user for MySQL that will only have permissions on this database.

mysql -u root -p

# Prompt your password

mysql> CREATE DATABASE treviz;
mysql> CREATE USER 'treviz'@'localhost' IDENTIFIED BY 'password';
mysql> GRANT ALL PRIVILEGES ON treviz.* TO 'treviz'@'localhost';
mysql> FLUSH PRIVILEGES;
mysql> quit

It goes without saying that you should not try to install Treviz as root; so please take, or create, another user. In this example, we will create a user "treviz", and grant it root permissions.

sudo adduser treviz
sudo adduser treviz sudo
sudo su treviz
cd ~

Setting up treviz-backend

Download the Treviz backend

The first step of installing the backend is to download the code from our Github repository, and change its permissions to allow the nginx user to have access to the files:

cd ~
git clone https://gitlab.com/treviz/treviz-back.git treviz-back
sudo setfacl -R -m u:www-data:rX treviz-back
sudo setfacl -dR -m u:www-data:rwX -m u:$(whoami):rwX treviz-back/var
sudo setfacl -R -m u:www-data:rwX -m u:$(whoami):rwX treviz-back/var
sudo setfacl -R -m u:www-data:rwX -m u:$(whoami):rwX treviz-back/web
cd treviz-back

Preparing your configuration

Before you can install the dependencies, there is a few things you should do:

  • Note down the domains that you will use for your Treviz Instance. This includes:
    • the backend (sub-)domain: we chose "api.treviz.xyz"
    • the **websocket (sub-)domain **: we chose "websocket.treviz.xyz"
    • the websocket port: we chose 8080
    • the frontend (sub-)domain: we chose "app.treviz.xyz"
  • Generate a secret: Symfony uses a string for security issues, like generating csrf tokens. To generate it, you can for instance use pwgen
    sudo apt-get install -y pwgen
    pwgen 48 1 -By
    Note down the output.
  • **Generate SSH Keys **that will be used for Json Web Tokens:
    mkdir -p var/jwt
    openssl genrsa -out var/jwt/private.pem -aes256 4096
    openssl rsa -pubout -in var/jwt/private.pem -out var/jwt/public.pem
  • Have a ready-to-use SMTP Server: note down its host, port, encryption protocol, as well as the user and password used to connect to it
  • Generate the folders that you will use to store uploaded files:
    mkdir -p web/upload/projects/logos \
             web/upload/communities/logos \
             web/upload/communities/backgrounds \
             web/upload/users/avatars \
             web/upload/users/backgrounds \
             web/upload/filessudo
    setfacl -R -m u:www-data:rwX -m u:$(whoami):rwX web/upload
    

Configuring the backend

Now you can install the dependencies with composer.

composer install --optimize-autoloader

You will be asked to prompt a few parameters:

Database parameters:

  • database_host: host of the database Treviz should connect to. In our case, 'localhost'.
  • database_port: the port used to connect to that database. Most likely, 3306
  • database_name: the name of the database created for treviz. In our case, 'treviz'.
  • database_user: the name of the user Treviz should use to connect to the database. Again, "treviz"
  • database_password: the password of this user. In our case, "password" (but we hope you chose a different one)

Security parameters

  • secret: the secret you have generated earlier
  • jwt_private_key_path: the absolute path of the private ssh key you generated. In our case, "%kernel.root_dir%/../var/jwt/private.pem"
  • jwt_private_key_path: the absolute path of the public ssh key you generated. In our case, '%kernel.root_dir%/../var/jwt/public.pem'
  • jwt_key_pass_phrase: The phrase you used to secure the SSh keys.
  • jwt_token_ttl: Time the generated JWT will be valid, in seconds. 3600 is an hour.

SMTP Parameters:

  • mailer_transport: the method used to deliver emails. Most likely SMTP. For more possibilities, check out the swiftmailer documentation on the symfony website.
  • mailer_host: host of your SMTP server
  • mailer_port: port used to connect to the SMTP server
  • mailer_encryption: security protocol used to connect to the SMTP Server.
  • mailer_user: name of the user used to connect
  • mailer_password: password of the user

Parameters specific to your organization:

  • organization_name: Name of the organization you want to set up
  • frontend_url: hostname of the frontend. In our case, "app.treviz.xyz", but it is most likely different for you
  • backend_url: hostname of the backend, in our case "api.treviz.xyz"

RabbitMQ parameters to handle real-time communications

  • rabbitmq_host: Host of the RabbitMQ instance, by default localhost
  • rabbitmq_port: Port on that host, by defaut 5682
  • rabbitmq_usr: The user that can connect to it
  • rabbitmq_pwd: It's password

Folder structure used to store uploaded files: if you have followed our instructions, you should be able to keep it to their default value. Also, keep the default value of jms_serializer.camel_case_naming_strategy.class: it will be used to ensure that the serialization will use the exact same variable names as the backend.

Now that the application is correctly configured, it is time to update the database schema:

php bin/console doctrine:schema:validate
php bin/console doctrine:schema:create

Once this is done, clear the cache and install the assets to allow the administration module and API Documentation to display correctly

php bin/console cache:clear --env=prod --no-debug
php bin/console cache:warmup --env=prod
php bin/console assets:install

Setting up Nginx to serve the backend

We are now going to configure Nginx to serve the backend as expected.

Create a file in /etc/nginx/sites-avavilable called treviz-backand paste it the following lines:

server {
    listen 80;
    listen [::]:80;

    # Specify the root directory of the project; in our case, the web folder of treviz-back
    root /home/treviz/treviz-back/web/;

    # Specify the file to serve
    index app.php;

    # put here the url of the backend
    server_name api.treviz.xyz;

    # Index information
    location / {
            try_files $uri $uri/ /app.php$is_args$args;
    }

    # Deny all access to unnecessary files and folders.
    location ~* /(\.git|cache|bin|logs|backup|tests)/.*$ { return 403; }
    location ~* /(LICENSE\.txt|README\.md|CONTRIBUTING\.md|composer\.lock|composer\.json|package\.json|\.env|\.gitignore|symfony\.lock) { return 403; }

    # PHP Configuration
    location ~ \.php(/|$) {
        fastcgi_pass unix:/var/run/php/php7.1-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
    }

    # Specify the path of the log file
    error_log /var/log/nginx/treviz-back-error.log;
    access_log /var/log/nginx/treviz-back-access.log;

}

Then create a symbolic link in order to enable the virtual host:

sudo ln -s /etc/nginx/sites-available/treviz-back /etc/nginx/sites-enabled/treviz-back

Test and reload the nginx configuration:

sudo service nginx configtest
sudo service nginx restart

Now open a web broser and navigate to the backend url you decided. You should see a message like:

{
    "organization": "Treviz",
    "documentation": "/v1/doc",
    "version": "1.0"
}

This means you have correctly set up the treviz Backend. Congratulations !

Setting up the events micro-service

Treviz uses a specific web service to handle real-time communication with server-sent events.

cd ~
git clone https://gitlab.com/treviz/treviz-events.git treviz-events
cd treviz-events

Open the src/config/index.ts file and update its fields to match the configuration of your RabbitMQ instance, and the location of the public.pem file that was generated for the Treviz API.

const CONFIG = {
    port: 3000,
    amqpOptions: {
        protocol: 'amqp',
        hostname: 'rabbitmq',
        port: 5672,
        username: 'guest',
        password: 'guest'
    },
    pubKeyPath: '/var/www/api/var/jwt/public.pem'
}

export default CONFIG

You can now launch the service using npm:

npm run build && npm run start

Proxy the websocket using Nginx

In the /etc/nginx/sites-available, create a new file called treviz-websocket. Paste in it the following lines:

server {
    listen 80;
    listen [::]:80;

    server_name websocket.treviz.xyz;

    location /ws {
        # Proxy the connection to the port your websocket is listening to
        proxy_pass http://localhost:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_read_timeout 86400; # neccessary to avoid websocket timeout disconnect
        proxy_redirect off;
    }

    error_log /var/log/nginx/treviz-websocket-error.log;
    access_log /var/log/nginx/treviz-websocket-access.log;

}

Then, as we did for the api, create a symbolic link in the /etc/nginx/sites-enabled folder and reload nginx:

sudo ln -s /etc/nginx/sites-available/treviz-websocket /etc/nginx/sites-enabled/treviz-websocket
sudo service nginx configtest
sudo service nginx restart

Setting up treviz-frontend

Download the frontend

Just as for the backend, the first step of installing the frontend is to download the code from our Github repository, and change its permissions to allow the nginx user to have access to the files:

cd ~
git clone https://gitlab.com/treviz/treviz-front.git treviz-front
sudo setfacl -R -m u:www-data:rX treviz-front
cd treviz-front

Install the dependencies

The dependencies can be simply installed using npm:

npm install

Configure the frontend

Specify your parameters in the src/environments/environment.prod.ts file:

export const environment = {
  production: true, 
  api_url: "https://api.treviz.xyz/v1", // Replace by your api url, followed by the version you will use
  notifications_socket_url: "wss://websocket.treviz.xyz/ws"
};

Before you build the project, you might want to personnalize the way your Treviz instance will look:

  • Change the welcome message in src/assets/home.md. It is a markdown text, that will be displayed on the home page, before the user is signed in, and should explain the goal of the platform.
  • Change the information message in src/assets/info.md. It is a message that will be displayed when the user click on the "Info" button on the navbar, and should at least give the contacts of the platform's administrator and maintainer.
  • Change the background image displayed on the landing page, before the user is signed in, by changing the file src/assets/background.jpg

Once you are satisfied with your modifications, build the project using the Angular CLI:

ng build --prod

Expose the frontend via nginx

Just like for the backend, simply create a configuration file in the /etc/nginx/sites-available folder, fill and with the following lines:

server {
    listen 80;
    listen [::]:80;

    root /home/treviz/treviz-front/dist/;

    index index.html index.htm;

    server_name app.treviz.xyz;

    location / {
            # First attempt to serve request as file, then
            # as directory, then fall back to displaying a 404.
            try_files $uri$is_args$args $uri$is_args$args/ /index.html$is_args$args;
            error_page 404 index.html;
    }

    error_log /var/log/nginx/treviz-front-error.log;
    access_log /var/log/nginx/treviz-front-access.log;

}

Then create a symbolic link to enable it, test the configuration and restart nginx.

sudo ln -s /etc/nginx/sites-available/treviz-front /etc/nginx/sites-enabled/treviz-front
sudo service nginx configtest
sudo service nginx restart

Securing with TLS

If securing the access of Treviz with TLS is not technically a requirement, it is still something you should do before letting anyone register on the platform.

Obtaining the certificates

There are plenty of ways of securing a website with TLS; in this guide, we will focus on using Let's Encrypt to obtain the required certificates. First of all, install certbot using the following command line:

sudo apt-get install certbot -t jessie-backports

Then obtain the certificates using the "webroot" plugin:

sudo certbot certonly --webroot -w /home/treviz/treviz-front/dist -d app.treviz.xyz -d treviz.org -w /home/treviz/treviz-back/web -d api.treviz.xyz -d websocket.treviz.xyz

Note down the directory in which your certificates are stored. In our case, it should be something like /etc/letsencrypt/live/app.treviz.xyz. You will need to to write your ssl configuration latter.

You might get an error regarding the websocket domain. If it is the case, modify your treviz-websocket configuration file, and give it a root folder and behavior for other requests than /ws. You may remove it latter, but it is necessarry for let's encrypt to perform its tests.

Finally, set up a cron job to automatically renew your certificates:

sudo certbot renew --dry-run

Generate your own keys

In order to increase the security, we incite you to generate your own Diffie-Helman key, with the following command:

sudo openssl dhparam -out /etc/nginx/dhparam.pem 4096

Generating the key will probably take a while, so go grab a coffee or a part of cake. If you have come this far, you deserve it.

Once the key is generated, we will create a snippet of code that will be used to configure TLS in all our virtual hosts. Create a file called ssl.conf in /etc/nginx/snippets/ and put in it the following code:

# TLS settings
ssl on;
## Replace with the location of your certificates
ssl_certificate /etc/letsencrypt/live/app.treviz.xyz/fullchain.pem; // replace with your path
ssl_certificate_key /etc/letsencrypt/live/app.treviz.xyz/privkey.pem; // replace with your path

ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;

# intermediate configuration. tweak to your needs.
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
ssl_prefer_server_ciphers on;

# Diffie-Hellman custom params
ssl_dhparam /etc/nginx/dhparam.pem;

# HSTS (ngx_http_headers_module is required) (15768000 seconds = 6 months)
add_header Strict-Transport-Security max-age=15768000;

Adapt the nginx configuration

Once you have obtained your certificates and generated your kee, you want two things:

  • enable TLS for all your connections, including the websocket
  • redirect non-secured connections

To achieve this, we will need to change all our nginx configurations. Here are the updated configurations:

treviz-back
# Permanently redirect http to https
server {
    listen 80;
    listen [::]:80;
    server_name api.treviz.xyz;
    return 301 https://$host$request_uri;
}

server{
    listen 443 ssl;

    root /home/treviz/treviz-back/web/;

    # Specify file to serve
    index app.php;

    # Specify the url of your backend
    server_name api.treviz.xyz;

    # Include tls configuration
    include snippets/ssl.conf;

    location / {
            try_files $uri /app.php$is_args$args;
    }

    # Deny all access to unnecessary files and folders.
    location ~* /(\.git|cache|bin|logs|backup|tests)/.*$ { return 403; }
    location ~* /(LICENSE\.txt|README\.md|CONTRIBUTING\.md|composer\.lock|composer\.json|package\.json|\.env|\.gitignore|symfony\.lock) { return 403; }

    # PHP Configuration
    location ~ \.php(/|$) {
        fastcgi_pass unix:/var/run/php/php7.1-fpm.sock;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
    }

    # Specify the path of the log file
    error_log /var/log/nginx/treviz-back-error.log;
    access_log /var/log/nginx/treviz-back-access.log;
}
treviz-websocket
server {
    listen 80;
    listen [::]:80;
    server_name websocket.treviz.wyz;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl;

    server_name websocket.treviz.xyz;

    include snippets/ssl.conf;
    location /ws {
        proxy_pass http://localhost:8080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;

        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_read_timeout 86400; # neccessary to avoid websocket timeout disconnect
        proxy_redirect off;
    }        

    access_log /var/log/treviz-websocket-access.log;
    error_log /var/log/treviz-websocket-error.log;
}
treviz-front
# Permanently redirect http to https
server {
    listen 80;
    listen [::]:80;
    server_name app.treviz.xyz;
    return 301 https://$host$request_uri;
}

server{
    listen 443 ssl;

    root /home/treviz/treviz-front/dist/;

    # Specify file to serve
    index index.html index.htm;

    # Specify the url of your frontend
    server_name app.treviz.xyz;

    # Include tls configuration
    include snippets/ssl.conf;

    location / {
        try_files $uri$is_args$args $uri$is_args$args/ /index.html$is_args$args;
        error_page 404 index.html;
    }

    # Specify the path of the log file
    error_log /var/log/nginx/treviz-front-error.log;
    access_log /var/log/nginx/treviz-front-access.log;
}

Test the nginx configuration, reloas the service, and navigate to your treviz instance. You should see the connection is now secured.

sudo service nginx configtest
sudo service nginx restart