Commit 30b2aa7e authored by David Trattnig's avatar David Trattnig
Browse files

Full documentation for dev and prod environments.

parent cd05814a
......@@ -13,12 +13,6 @@ specifically build for the requirements of community radios.
- [Features](#features)
- [Architecture](#architecture)
- [Installation](#installation)
- [Configuration](#configuration)
- [Running Engine](#running-engine)
- [Development](#development)
- [Production](#production)
- [API Server](#api-server)
- [Logging](#logging)
- [About](#about)
- [Resources](#resources)
......@@ -51,127 +45,10 @@ To learn more, checkout the [Engine Developer Guide](docs/developer-guide.md) or
## Installation
Aura Engine runs on any modern Debian-based OS. It requires at least `Node 13`, `Python 3.7`.
- [Installation for Development](docs/installation-development.md)
- [Installation for Production](docs/installation-production.md)
- [Installation using Docker](docs/installation-docker.md)
You also need to install `Liquidsoap 1.4.1` using [OPAM](https://www.liquidsoap.info/doc-1.4.1/install.html).
Then do
```bash
git clone https://gitlab.servus.at/autoradio/engine
```
By default Aura Engine uses MariaDB for persistence. When starting the installation, please
ensure you have root access to your database instance. The installation script automatically
creates a database plus an associated user with password. If you want to use your own database
system, select "Other / Manually" during the database installation step.
Development Environment
```bash
sudo ./install.sh
```
Production Environment
```bash
sudo ./install.sh prod
```
## Configuration
In your development environment edit the file
```shell
./configuration/engine.ini
```
to configure the engine.
In production, or if the file exists, Engine uses the config location
```shell
/etc/aura/engine.ini
```
Read more about detailed settings in the [Configuration Guide](docs/configuration-guide.md).
## Running Engine
### Development
While developing there is a simple convencience script `run.sh`
to get you started. Call the engine's components in following order:
```shell
./run.sh # Starts the engine-core component
./run.sh lqs # Starts the engine-liquidsoap component
./run.sh api # Starts the engine-api component
```
In development mode Engine uses the [Flask](https://palletsprojects.com/p/flask/) development server.
This server should not be used in your production environment.
### Production
In production the process is slightly different to ensure the
engine's components are always running i.e. restart themselves after some system
restart or crash. Therefore they are executed using a system service:
```bash
systemctl start aura-engine
systemctl start aura-lqs
```
and on system boot run following:
```bash
systemctl enable aura-engine
systemctl enable aura-lqs
```
#### API Server
For production Engine API uses the WSGI HTTP Server [`Gunicorn`](https://gunicorn.org/).
In production Gunicorn is used in combination with some proxy server, such as Nginx.
> Although there are many HTTP proxies available, we strongly advise that you use Nginx. If you choose another proxy server you need to make sure that it buffers slow clients when you use default Gunicorn workers. Without this buffering Gunicorn will be easily susceptible to denial-of-service attacks. You can use Hey to check if your proxy is behaving properly. — [**Gunicorn Docs**](http://docs.gunicorn.org/en/latest/deploy.html).
[`Supervisor`](http://supervisord.org/) is a preferable solution to run the gunicorn server
in the background and also start it automatically on reboot.
**Start the API service with Supervisor**
```shell
sudo supervisorctl reread
sudo supervisorctl update
sudo supervisorctl avail
sudo supervisorctl restart engine-api
```
In case you want to reload whole supervisor service
```shell
sudo service supervisor restart
```
## Logging
All Engine logs for production can be found in `/var/log/aura/engine`
You can access the service logs using one of:
```
journalctl -u aura-lqs
journalctl -u aura-engine
```
## About
......
# Install for Development
## Prerequisites
Aura Engine runs on any modern Debian-based OS. It requires at least
- `Node 13`
- `Python 3.7`
- `OPAM`
- `git`
Additionally you'll need these system packages:
```bash
apt-get install \
redis-server \
libsndfile1 \
ffmpeg \
quelcom \
```
**Install Liquidsoap**
Now it's time to install the Liquidsoap and its dependencies.
Install `Liquidsoap 1.4.1` or newer, using [OPAM](https://www.liquidsoap.info/doc-1.4.1/install.html).
Let's initialize OPAM for the user you are going to run the engine with:
```bash
opam init
```
## Setup Database
The following installation script sets up the database.
```bash
bash script/setup-db.sh
```
By default Aura Engine uses MariaDB for persistence. When starting this script, please
ensure you have root access to your database instance. The installation script automatically
creates a database plus an associated user with password. If you want to use your own database
system, select "Other / Manually" during the database installation step.
In case of MariaDB the script also installs these system dependencies:
apt-get install mariadb-server libmariadbclient-dev
When this is completed, carefully check if any error occured. In case your database has been setup
automatically, note the relevant credentials for later use in your `engine.ini` configuration.
**Cloning the project**
Then clone the project to some development folder:
```bash
git clone https://gitlab.servus.at/autoradio/engine
```
## Installation
```shell
./install.sh
```
This script does the following:
- Install Liquidsoap components using OPAM (`script/install-opam-packages`)
- NodeJS Modules (`script/install-web.sh`)
- Python Packages (`requirements.txt`)
- Creates a default configuration file in `configuration/engine.ini`
When this is completed, carefully check if any error occured.
## Configuration
In your development environment edit following file to configure the engine:
```shell
./configuration/engine.ini
```
Please note if some configuration exists under `/etc/aura/engine.ini` the configuration is
read from there by default.
Read more about detailed settings in the [Configuration Guide](docs/configuration-guide.md).
## Running Engine
Use the convencience script `run.sh` to get engine started in different ways:
**Run the Engine**
This includes the Liquidsoap audio engine, but does not start the API server.
```shell
./run.sh
```
**Run the Engine Core only**
This requires to start the Liquidsoap component within some other terminal. This is helpful for
debugging purposes.
```shell
./run.sh core
```
**Run Liquidsoap only**
This requires to start the core component in another terminal. This is helpful for
debugging purposes.
```shell
./run.sh lqs
./run.sh api
```
**Run the Engine API**
This requires to start the core component in another terminal.
```shell
./run.sh lqs
./run.sh api
```
In development mode Engine uses the default [Flask](https://palletsprojects.com/p/flask/) web server.
Please be careful not to use this type of server in your production environment.
## Logging
All Engine logs for development can be found under `./logs`.
# Install for Production
## Prerequisites
Aura Engine runs on any modern Debian-based OS. It requires at least
- `Node 13`
- `Python 3.7`
- `OPAM`
- `git`
Additionally you'll need these system packages:
```bash
apt-get install \
supervisor \
gunicorn \
redis-server \
libsndfile1 \
ffmpeg \
quelcom \
```
**Create an user**
While previous packages need superuser rights to be installed, the following ones are installed for the user which is
executing the engine. In your development environment you can skip this step. In production you first need to create
a user called `engineuser`.
```bash
bash script/create-engineuser.sh
```
And switch to that user
```bash
su engineuser
```
**Install Liquidsoap**
Now it's time to install the Liquidsoap and its dependencies.
Install `Liquidsoap 1.4.1` or newer, using [OPAM](https://www.liquidsoap.info/doc-1.4.1/install.html).
Let's initialize OPAM for the user you are going to run the engine with:
```bash
engineuser:$ opam init
```
**Setup Database**
The following installation script sets up the database.
```bash
bash script/setup-db.sh
```
By default Aura Engine uses MariaDB for persistence. When starting this script, please
ensure you have root access to your database instance. The installation script automatically
creates a database plus an associated user with password. If you want to use your own database
system, select "Other / Manually" during the database installation step.
In case of MariaDB the script also installs these system dependencies:
apt-get install mariadb-server libmariadbclient-dev
If you have chosen to setup your database automatically, note the relevant credentials.
**Cloning the project**
Create the folder `/opt/aura` and clone the engine project:
```bash
git clone https://gitlab.servus.at/autoradio/engine
```
Now you should have `/opt/aura/engine/`.
**Initialize folders and permissions**
Call this script to create the required log folders and update all permissions.
```bash
bash script/initialize.sh
```
## Installation
The following installation script also sets up the database.
By default Aura Engine uses MariaDB for persistence. When starting the installation, please
ensure you have root access to your database instance. The installation script automatically
creates a database plus an associated user with password. If you want to use your own database
system, select "Other / Manually" during the database installation step.
```shell
./install.sh prod
```
This script does the following:
- Install Liquidsoap components using OPAM (`script/install-opam-packages`)
- NodeJS Modules (`script/install-web.sh`)
- Python Packages (`requirements.txt`)
- Creates a log folder in `/var/log/aura/`
- Creates a default configuration file in `/etc/aura/engine.ini`
When this is completed, carefully check if any error occured. In case your database has been setup
automatically, note the relevant credentials for later use in your `engine.ini` configuration.
## Configuration
In your production environment edit following file to configure the engine:
```shell
nano /etc/aura/engine.ini
```
Read more about detailed settings in the [Configuration Guide](docs/configuration-guide.md).
## Running Engine
In production the process of starting the engine is slightly different to ensure the engine's components
are always running i.e. restart themselves after some system restart or crash. Therefore they are executed
using [Supervisor](http://supervisord.org/). While running might also work using a `systemd` service, the
recommened option to use in combination with Gunicorn ([API server](Running the API Server), see below),
is Supervisor. Beside others, Supervisor has the advantage that you are able to run services without
being superuser.
Now, given you are in the engine's home directory `/opt/aura/engine/`, simply type following to start
the services:
```shell
supervisord
```
This picks up the supervisor configuration provided in the local `supervisord.conf` and the service configurations
located in `configuration/supervisor/*.conf`.
Note that the supervisor daemon starts all (both) services at once. If want more fine-grained control for
starting services individually, please check-out the next section.
**Listing available Services**
```shell
supervisorctl avail
```
You should get these two services with their actual state listed:
```c++
aura-engine in use auto 666:666
aura-engine-api in use auto 999:999
```
## The API Server
For production Engine API uses the WSGI HTTP Server [`Gunicorn`](https://gunicorn.org/).
In production Gunicorn is used in combination with some proxy server, such as Nginx.
> Although there are many HTTP proxies available, we strongly advise that you use Nginx. If you choose another proxy
server you need to make sure that it buffers slow clients when you use default Gunicorn workers. Without this buffering
Gunicorn will be easily susceptible to denial-of-service attacks. You can use Hey to check if your proxy is behaving properly.
[**Gunicorn Docs**](http://docs.gunicorn.org/en/latest/deploy.html).
## Maintanence using Supervisor
Please remember to call all `supervisorctl` commands from within your engine home directory (`/opt/aura/engine/`),
to pickup the correct `supervisord.conf`.
**Starting Services**
```shell
supervisorctl start <service-name>
```
**Stopping Services**
```shell
supervisorctl stop <service-name>
```
**Restarting Services**
```shell
supervisorctl restart <service-name>
```
**Refresh after changing configurations**
```shell
supervisorctl restart <service-name>
```
**Start the API service with Supervisor**
```shell
sudo supervisorctl update
sudo supervisorctl restart engine-api
```
In case you want to reload whole supervisor service
```shell
sudo service supervisor restart
```
## Logging
All Engine logs for production can be found under:
```shell
`/var/log/aura`
```
This includes individual logs from Engine Core, Liquidsoap and the API.
But also `stout` outputs from supervisor services are written there.
Additionally you'll finde Supervisor specific logs under:
```
`/var/log/supervisor`
```
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment