diff --git a/README.md b/README.md index 699f9af059a04c135eb4222a264746d99c3a2311..17640f66a7a64dff173f698fcb65a17b770be3f8 100644 --- a/README.md +++ b/README.md @@ -5,63 +5,114 @@ <img src="https://gitlab.servus.at/autoradio/meta/-/raw/master/images/aura-engine.png" width="250" align="right" /> Aura Engine is a scheduling and play-out engine as part of [Aura Radio Software Suite](#About), specifically build for -the requirements of community radios. +the requirements of community radio stations. <!-- TOC --> - [Aura Engine](#aura-engine) - - [Features](#features) - - [Architecture](#architecture) - - [Installation](#installation) + - [Functionality](#functionality) + - [Scheduler](#scheduler) + - [Versatile playlists](#versatile-playlists) + - [Default playlists](#default-playlists) + - [Heartbeat Monitoring](#heartbeat-monitoring) + - [Logging](#logging) + - [Getting started](#getting-started) + - [Using Docker](#using-docker) - [Read more](#read-more) - [About](#about) - - [Resources](#resources) <!-- /TOC --> -## Features +## Functionality -- **Scheduler** to automatically broadcast your radio programme (see [AURA Dashboard](https://gitlab.servus.at/aura/dashboard) for an schedule user interface) -- **Play audio from multiple sources** including files, streams and analog live audio -- **Dynamic switching of sources** organized as playlists -- **Stream output** to an Icecast Server -- **Multichannel Line-out** -- **Silence Detector** to avoid [Dead Air](https://en.wikipedia.org/wiki/Dead_air) -- **Auto DJ** which plays random music when silence is detected -- **ReplayGain** normalization done using passed [ReplayGain](https://en.wikipedia.org/wiki/ReplayGain) meta data -- **API** to query Track-Service, monthly reports and information for displaying the Studio Clock (see [Engine API](https://gitlab.servus.at/aura/engine-api)) +In conjuction with other AURA components Engine provides several features: + +- **Scheduler** to automatically broadcast your radio programme (see [AURA Dashboard](https://gitlab.servus.at/aura/dashboard) for an user interface to do scheduling) +- **Analog input and outputs** provided by [Engine Core](https://gitlab.servus.at/aura/engine-core) +- **Streaming to an [Icecast](https://icecast.org/) Server including [Icy Metadata](https://cast.readme.io/docs/icy)** provided by [Engine Core](https://gitlab.servus.at/aura/engine-core) +- **Auto DJ with Silcence Detector** provided by [Engine Core](https://gitlab.servus.at/aura/engine-core) - **Web Application for a Track Service** (see [AURA Player](https://gitlab.servus.at/aura/player)) - **Web Application providing a Studio Clock** (see [Engine Clock](https://gitlab.servus.at/aura/engine-clock)) -- **Bulk and Timeslot Recorder** (This will be provided after v1.1 by a planned `engine-recorder` component) +- **Bulk and Timeslot Recorder** (*Not yet available*, this will be provided after v1.1 by a planned `engine-recorder` component) +- **API** to query Track-Service, monthly reports and information for displaying the Studio Clock (see [Engine API](https://gitlab.servus.at/aura/engine-api)) + +### Scheduler + +Engine provide a scheduling functionality by polling external API endpoints frequently. Those API endpoints are provided by [Steering](https://gitlab.servus.at/aura/steering) to retrieve schedule information and [Tank](https://gitlab.servus.at/aura/tank) to retrieve playlist information. To define your schedule you'll also need [AURA Dashboard](https://gitlab.servus.at/aura/dashboard) which is an elegent web user interface to manage your shows, playlists and schedules. + +Ideally any audio is scheduled some time before the actual, planned playout to avoid timing issues with buffering and preloading. Nonetheless, playlists can also be scheduled after a given calendar timeslot has started already. In such case the playout starts as soon it's preloaded. + +If for some reason the playout is corrupted, stopped or too silent to make any sense, then this <u>triggers a fallback using the silence detector</u> (see chapter below). + +> Note: If you delete any existing timeslot in Dashboard/Steering this is only reflected in Engine until the start of the scheduling window. The scheduling window is defined by the start of the timeslot minus a configured offset in seconds (compare your Engine configuration). + +#### Versatile playlists + +It's possible to schedules playlists with music or pre-recorded shows stored on the **file system**, via external **streams** or live from an **analog input** in the studio. All types of sources can be mixed in a single playlist. + +The switching between types of audio source is handled automatically, with configured fadings applied. + +> Note: Any live sources or streams not specifing a length property, are automatically expanded to the left duration of the timeslot. + +#### Default playlists + +While a timeslot can have a specific playlist assigned, it's also possible to define default playlists +for schedules and shows: -Read more on the [Engine Features](docs/engine-features.md) page. +- **Default Schedule Playlist**: This playlist is defined on the level of some recurrence rules (*Schedule*). + In case the timeslot doesn't have any specific playlist assigned, this playlist is broadcasted. -## Architecture +- **Default Show Playlist**: This playlist can be assigned to some show. If neither the specific timeslot + playlist nor the default schedule playlist is specificed the *default show playlist* is broadcasted. -AURA Engine as part of the AURA Radio Suite uses an modulear architecture -based on a REST API. All external information is retrieved using JSON data-structures. -To learn more, checkout the [Engine Developer Guide](docs/developer-guide.md) or visit - the [Aura Meta](https://gitlab.servus.at/autoradio/meta) repository. + -## Installation +If none of these playlists have been specified the *Auto DJ* feature of [Engine Core](https://gitlab.servus.at/aura/engine-core) takes over (optional). -- [Installation for Development](docs/installation-development.md) -- [Installation for Production](docs/installation-production.md) -- [Running with Docker](docs/running-docker.md) -- [Setup the Audio Store](docs/setup-audio-store.md) +### Heartbeat Monitoring + +Instead of checking all status properties, the Heartbeat only validates the vital ones required to run the engine. If all of those are valid, as network socket request is sent to a defined server. This heartbeat is sent continiously based on the configured `heartbeat_frequency`. The service receiving this heartbeat ticks can decide what to do with that information. One scenario could be switching to another Engine instance or any other failover scenario. Under `contrib/heartbeat-monitor` you'll find some sample application digesting these heartbeat signals. + +### Logging + +All Engine logs can be found in the local `./logs` directory. Adapt the log-level within your configuration to get more or less verbose log output. Whenever the Engine's status turns into some unhealthy state, additionally this is logged to [Engine API](https://gitlab.servus.at/aura/engine-api). Also, when it returns to some valid state this is logged to the Engine API. + +## Getting started + +Ensure that you have also dependencies such as `steering`, `tank`, `dashboard`, `engine-core`, and `engine-api` up and running. There's a how-to in the [Meta](https://gitlab.servus.at/aura/meta) repository to get quickly started using [Docker Compose](https://docs.docker.com/compose/). + +For production we recommend running Engine using Docker Compose. If you want to install for AURA development or simply prefer to run natively, check out the [Bare Metal Installation](docs/bare-metal-installation.md). + +## Using Docker + +If you only want to run the single Engine Docker container, you can do this in a few, simple steps. Before getting started copy the default configuration file to `config/engine.docker.ini`: + +```shell + cp config/sample-docker.engine.ini config/engine.docker.ini +``` + +You'll need to do a few configurations which are required: + - The password `db_pass` for the local database holding scheduling information + - The app secret `api_tank_secret` for connecting to [AURA Tank](https://gitlab.servus.at/aura/tank) + - Also check the `ENV` variables defined in the `run.sh` script. + +Now start the engine with: + +```shell + ./run.sh docker:engine +``` ## Read more +- [Bare Metal Installation](docs/bare-metal-installation.md) - [Developer Guide](docs/developer-guide.md) -- [Engine Features](docs/engine-features.md) -- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md) +- [Setting up the Audio Store [meta]](https://gitlab.servus.at/aura/meta/-/blob/master/docs/setup-audio-store.md) ## About -Aura Engine is a scheduling and play-out engine as part of [Aura Radio Software Suite](#About), specifically build for -the requirements of community radios. +Aura Engine is a scheduling and play-out engine as part of [Aura Radio Software Suite](#About), specifically build for the requirements of community radios. [<img src="https://gitlab.servus.at/autoradio/meta/-/raw/master/images/aura-logo.png" width="150" />](https://gitlab.servus.at/aura/meta) @@ -69,15 +120,5 @@ AURA stands for Automated Radio and is a swiss army knife for community radio st | [<img src="https://gitlab.servus.at/aura/meta/-/raw/master/images/aura-steering.png" width="150" align="left" />](https://gitlab.servus.at/aura/steering) | [<img src="https://gitlab.servus.at/aura/meta/-/raw/master/images/aura-dashboard.png" width="150" align="left" />](https://gitlab.servus.at/aura/dashboard) | [<img src="https://gitlab.servus.at/aura/meta/-/raw/master/images/aura-tank.png" width="150" align="left" />](https://gitlab.servus.at/aura/tank) | [<img src="https://gitlab.servus.at/aura/meta/-/raw/master/images/aura-engine.png" width="150" align="left" />](https://gitlab.servus.at/aura/engine) | |---|---|---|---| -| [Steering](https://gitlab.servus.at/aura/steering) | [Dashboard](https://gitlab.servus.at/aura/dashboard) | [Tank](https://gitlab.servus.at/aura/tank) | [Engine](https://gitlab.servus.at/aura/engine)<br/>[Engine API](https://gitlab.servus.at/aura/engine-api)<br/>[Engine Clock](https://gitlab.servus.at/aura/engine-clock) | - -## Resources - -* **Python**: https://docs.python.org/ -* **OPAM (OCaml Package Manager)**: https://opam.ocaml.org/ -* **Liquidsoap**: https://www.liquidsoap.info/doc-1.4.0/ -* **Jack Audio**: https://jackaudio.org/ -* **Flask**: https://palletsprojects.com/p/flask/ -* **Systemd**: https://systemd.io/ -* **Supervisor**: http://supervisord.org/ -* **Gunicorn**: https://gunicorn.org/ \ No newline at end of file +| [Steering](https://gitlab.servus.at/aura/steering) | [Dashboard](https://gitlab.servus.at/aura/dashboard) | [Tank](https://gitlab.servus.at/aura/tank) | [Engine](https://gitlab.servus.at/aura/engine)<br/>[Engine Core](https://gitlab.servus.at/aura/engine-core)<br/>[Engine API](https://gitlab.servus.at/aura/engine-api)<br/>[Engine Clock](https://gitlab.servus.at/aura/engine-clock) | + diff --git a/docs/bare-metal-installation.md b/docs/bare-metal-installation.md new file mode 100644 index 0000000000000000000000000000000000000000..33a89b8eefac42c3728dbd74c433f7627c0db780 --- /dev/null +++ b/docs/bare-metal-installation.md @@ -0,0 +1,176 @@ +# Install for Development + +<!-- TOC --> + +- [Install for Development](#install-for-development) + - [Prerequisites](#prerequisites) +- [Setting up the database](#setting-up-the-database) + - [Preparation](#preparation) + - [Configuration](#configuration) + - [Running Engine](#running-engine) + - [Daemonized Engine](#daemonized-engine) + - [Running with Systemd](#running-with-systemd) + - [Running with Supervisor](#running-with-supervisor) + - [Logging](#logging) + - [Read more](#read-more) + +<!-- /TOC --> + +## Prerequisites + +Aura Engine runs on any modern Debian-based OS. It requires at least + +- [`Python 3.8+`](https://www.python.org/downloads/release/python-380/) +- [`pip`](https://pip.pypa.io/en/stable/) +- [`git`](https://git-scm.com/) +- ['virtualenv'](https://pypi.org/project/virtualenv/) (development only) +- DBMS server/client libraries for MariaDB or PostgreSQL (see below) + +# Setting up the database + +Depending on the DBMS you are planning to use you'll need to have the relevant server/client libaries to be present. + +**MariaDB** + +```shell +sudo apt install \ + python3.8-dev \ + default-libmysqlclient-dev \ + mariadb-server \ + libmariadbclient-dev +``` + +The following installation script sets up the initial databases and users. + +```bash + bash scripts/setup-db.sh +``` + +As soon as this is completed, carefully check if any error occured. In case your database has been setup automatically, note the relevant credentials for later use in your `engine.ini` configuration. + +**Setting up the project structure** + +Then clone all `engine` related projects to some development folder. In production we recommend `/opt/aura/` as the system base. + +```bash +mkdir ~/code/aura +cd ~/code/aura +git clone https://gitlab.servus.at/autoradio/engine +git clone https://gitlab.servus.at/autoradio/engine-core +git clone https://gitlab.servus.at/autoradio/engine-api + ... +``` + +If you are developing engine you also might want the projects `steering`, `dashboard` and `tank` in there, or you'll start them with Docker Compose (see [how to run Aura Web with Docker](https://gitlab.servus.at/autoradio/meta) in the [meta](https://gitlab.servus.at/autoradio/meta) repository) + +## Preparation + +In case of an development environment, create a virtual env for your Python dependencies: + +```shell +virtualenv -p python3.8 python +``` + +No worries, the resulting `python` directory is excluded from the git respostory. + +To activate that environment, run + +```shell +source python/bin/activate +``` + +Then, install the required dependencies + +```shell +pip install -r requirements.txt +``` + +Create your base configuration from the sample configuration + +```shell +# Development +cp config/sample.development.engine.ini config/engine.ini +# Production +cp config/sample.production.engine.ini config/engine.ini +``` + +## Configuration + +In your development environment edit following file to configure the engine: + +```shell +./config/engine.ini +``` + +> Please note, if some configuration exists under `/etc/aura/engine.ini` the configuration by default is drawn from there. + +While the configuration has plenty of configuration options, you only need to set a few mandatory ones, given you are running the other components (such as 'engine-core', "engine-api" etc.) at the default settings too. + +Required modifications are: + - The password `db_pass` for the local database holding scheduling information + - The app secret `api_tank_secret` for connecting to [AURA Tank](https://gitlab.servus.at/aura/tank) + +## Running Engine + +If you have defined a virtual env during the installation step you'll need to activate it first. You'll have to do so whenever you gonna start your development environment: + +```shell +source python/bin/activate +``` + +There's a convencience script `run.sh` to get engine started + +```shell + engine$ ./run.sh +``` + +Keep in mind you'll also need to start Engine Core separately + +```shell + engine-core$ ./run.sh +``` + +In order to have a full engine experience also the other AURA Components are required to be running. For convencience in starting the full environment checkout how to run Aura Web using Docker Compose within the [Meta Repository](https://gitlab.servus.at/aura/tank) + +## Daemonized Engine + +For this you can utilize either [Systemd](https://systemd.io/) or [Supervisor](http://supervisord.org/). Please check the their manuals on how to use these services. + +The daemon configs are expecting you run engine under the user `engineuser` and being located under `/opt/aura/engine`, `/opt/aura/engine-api` and `/opt/aura/engine-core` respectively. Do prepare the project root and permissions you can use the script `scripts/initialize-systemd.sh`. To create the matching user and audio group membership run `scripts/create-engineuser.sh`. + +### Running with Systemd + +Copy the unit files in `/opt/aura/engine/config/systemd/aura-engine.service` to your systemd unit directory, and reload the systemd daemon: + +```shell +cp /opt/aura/engine/config/systemd/* /etc/systemd/system/ +systemctl daemon-reload +``` + +### Running with Supervisor + +Now, given you are in the engine's home directory `/opt/aura/engine/`, simply type following to start the services: + +```shell +supervisord +``` + +This picks up the supervisor configuration provided in the local `supervisord.conf` and the service configurations located in `config/supervisor/*.conf`. + +Then you'll need to reload the supervisor configuration using `sudo`: + +```shell +sudo supervisorctl reload +``` + + + +## Logging + +All Engine logs can be found under `./logs`. + +## Read more + +- [Bare Metal Installation](docs/bare-metal-installation.md) +- [Developer Guide](docs/developer-guide.md) +- [Setting up the Audio Store [meta]](https://gitlab.servus.at/aura/meta/-/blob/master/docs/setup-audio-store.md) diff --git a/docs/developer-guide.md b/docs/developer-guide.md index 358415ba1f5404fc8b1e8fd8069f0566a060886d..ed0c023e0cf6ac95074a2c06c8be9a8e8c1895a1 100644 --- a/docs/developer-guide.md +++ b/docs/developer-guide.md @@ -8,19 +8,15 @@ This page gives insights on extending Aura Engine internals or through the API. - [AURA Componentes](#aura-componentes) - [Engine Components](#engine-components) - [API](#api) - - [Engine Startup Phases](#engine-startup-phases) - - [Scheduling](#scheduling) - - [More infos for debugging](#more-infos-for-debugging) - - [Debugging Liquidsoap](#debugging-liquidsoap) - - [Tips on configuring the audo interface](#tips-on-configuring-the-audo-interface) + - [Scheduler](#scheduler) + - [Docker](#docker) - [Read more](#read-more) <!-- /TOC --> ## AURA Componentes -AURA Engine as part of the AURA Radio Suite uses an modulear architecture -based on a REST API. All external information is retrieved using JSON data-structures. +AURA Engine as part of the AURA Radio Suite uses an modulear architecture based on a REST API. All external information is retrieved using JSON data-structures. To get the basic architectural overview, visit the [Aura Meta](https://gitlab.servus.at/autoradio/meta) repository. @@ -35,8 +31,7 @@ For example: If you need to test and develop against the Engine's API you'll also need to get the `engine-api` project running. -For a start it's recommended to create a general `aura` project folder. In there you start cloning all the sub-projects. -After having all the sub-projects configured, and verified that they are working, take a look at the AURA `meta` project. +For a start it's recommended to create a general `aura` project folder. In there you start cloning all the sub-projects. After having all the sub-projects configured, and verified that they are working, take a look at the AURA `meta` project. There's a convenience script to start all of the three main dependencies (Steering, Dashboard, Tank) all at once: @@ -49,24 +44,13 @@ There's a convenience script to start all of the three main dependencies (Steeri *...TBD...* - ## API You can find the AURA API definition here: https://gitlab.servus.at/autoradio/meta/blob/master/api-definition.md OpenAPI definition for Engine API: https://app.swaggerhub.com/apis/AURA-Engine/engine-api/ -## Engine Startup Phases - -When you start Engine the following is happening: - -1. Python `run.py`: Initializes `src/core/engine.py` (The virtual mixer; class for remote-controlling Liquidsoap), Scheduler -2. Python `run.py`: Start Liquidsoap. -3. Liquidsoap: When Liquidsoap finished its startup, it creates a socket file as configured in `socket_dir` of `engine.ini`. -4. Python `src/core/liquidsoap/client.py`: Connects to that socket file. -5. Python `src/schedulung/scheduler.py`: Continously loads timeslots from the API endpoints, stores them in the local database and starts the playout as per the timeslots. - -## Scheduling +## Scheduler Scheduling is split into multiple phases. Below you see a timeline with one timeslot planned at a certain point in time and the involved phase before: @@ -130,79 +114,22 @@ point in time and the involved phase before: this <u>triggers a fallback using the silence detector</u>. -## More infos for debugging - -### Debugging Liquidsoap - -Connect to Liquidsoap via Telnet - - telnet 127.0.0.1 1234 - -List available commands - - help - -List all available channels - - list - -List all input channels connected to the mixer - - mixer.input - -Set the volume of mixer `input 0` to `100%` - - mixer.volume 0 100 +## Docker -Push some audio file to the filesystem `queue 0` +Build your own, local Docker image - in_filesystem_0.push /path/to/your/file.mp3 - - -### Tips on configuring the audo interface - -Configure your audio device in the `[soundcard]` section of `engine.ini`. - -You can configure up to **five** line IN and OUT stereo channels. Your hardware should -support that. When you use JACK, you will see the additional elements popping up when -viewing your connections (with e.g. Patchage). - -**Pulse Audio:** When using Ubuntu, Pulse Audio is selected by default. This is convenient, -as you won't have the need to adapt any Engine setting to get audio playing initially. - -**ALSA:** When you use ALSA, you will have to play around with ALSA settings. In the folder -`./src/liquidsoap` is a scipt called alsa_settings_tester.liq. You can start it -with 'liquidsoap -v --debug alsa_settings_tester.liq'. Changing and playing with -settings may help you to find correct ALSA settings. - -**Jack Audio**: Install the JACK daemon and GUI: - -```bash - sudo apt-get install jackd qjackctl +```shell +./run.sh docker:build ``` -Please ensure to enable "*realtime process priority*" when installing JACK to keep latency low. -Now, you are able to configure your hardware settings using following command: +Releasing a new version to DockerHub -```bash - qjackctl +```shell +./run.sh docker:push ``` -Next you need to install the JACK plugin for Liquidsoap: - -```bash -sudo apt install \ - liquidsoap-plugin-jack -``` - - ## Read more -- [Overview](/README.md) -- [Installation for Development](installation-development.md) -- [Installation for Production](installation-production.md) -- [Running with Docker](running-docker.md) -- [Setup the Audio Store](docs/setup-audio-store.md) -- [Developer Guide](developer-guide.md) -- [Engine Features](engine-features.md) -- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md) \ No newline at end of file +- [Bare Metal Installation](docs/bare-metal-installation.md) +- [Developer Guide](docs/developer-guide.md) +- [Setting up the Audio Store [meta]](https://gitlab.servus.at/aura/meta/-/blob/master/docs/setup-audio-store.md) diff --git a/docs/engine-features.md b/docs/engine-features.md deleted file mode 100644 index cee669e1271014cb6129f97b06157c6aaa075d8f..0000000000000000000000000000000000000000 --- a/docs/engine-features.md +++ /dev/null @@ -1,237 +0,0 @@ -# Aura Engine Features - -This page gives a more detailed overview of the Aura Engine features and how to configure them. - -<!-- TOC --> - -- [Aura Engine Features](#aura-engine-features) - - [Multi-channel Input (Filesystem, Stream, Analog)](#multi-channel-input-filesystem-stream-analog) - - [Multi-channel output](#multi-channel-output) - - [Analog line-out](#analog-line-out) - - [Stream to Icecast](#stream-to-icecast) - - [Scheduling](#scheduling) - - [Default Playlists](#default-playlists) - - [Auto DJ](#auto-dj) - - [Silence Detector](#silence-detector) - - [ReplayGain and LoudNorm Normalization](#replaygain-and-loudnorm-normalization) - - [Monitoring](#monitoring) - - [Send mails on errors and warnings](#send-mails-on-errors-and-warnings) - - [Engine Health Information via Engine API](#engine-health-information-via-engine-api) - - [Engine Heartbeat](#engine-heartbeat) - - [Logging](#logging) - - [Read more](#read-more) - -<!-- /TOC --> - -## Multi-channel Input (Filesystem, Stream, Analog) - -It's possible to schedules playlists with music or pre-recorded show stored on the **file system**, -via external **streams** or live from an **analog input** in the studio. All types of sources can -be mixed in a single playlist. - -> Note: Any live sources or streams not specifing a length property, are automatically expanded to -the left duration of the timeslot. - -The switching between types of audio source is handled automatically. To learn more check out the -[Scheduling](### Secure Scheduling) section. - -## Multi-channel output - -### Analog line-out - -In most scenarios it might be sufficient to broadcast via only one analog stereo output. -If needed you can configure up to five stereo pairs. - -### Stream to Icecast - -Engine allows to stream to multiple Icecast Servers simultaniousely. It is also sending meta information -to the streaming server, using the *Icy* protocol. - -To configure your Icecast connectivity check-out the `[stream]` section in your configuration. - -## Scheduling - -Engine provide a scheduling functionality by polling external API endpoints frequently. Those API endpoints -are provided by [Steering](https://gitlab.servus.at/aura/steering) to retrieve schedule information and -[Tank](https://gitlab.servus.at/aura/tank) to retrieve playlist information. To define your schedule you'll -also need [AURA Dashboard](https://gitlab.servus.at/aura/dashboard) which is an elegent web user interface to -manage your shows, playlists and schedules. - -Ideally any audio is scheduled some time before the actual, planned playout to avoid timing issues with buffering and -preloading. Nonetheless, playlists can also be scheduled after a given calendar timeslot has started already. -In such case the playout starts as soon it's preloaded. - -If for some reason the playout is corrupted, stopped or too silent to make any sense, then -this <u>triggers a fallback using the silence detector</u> (see chapter below). - -> Note: If you delete any existing timeslot in Dashboard/Steering this is only reflected in Engine until the start -of the scheduling window. The scheduling window is defined by the start of the timeslot minus a configured offset -in seconds (compare your Engine configuration). - -## Default Playlists - -While a timeslot can have a specific playlist assigned, it's also possible to define default playlists -for schedules and shows: - -- **Default Schedule Playlist**: This playlist is defined on the level of some recurrence rules (*Schedule*). - In case the timeslot doesn't have any specific playlist assigned, this playlist is broadcasted. - -- **Default Show Playlist**: This playlist can be assigned to some show. If neither the specific timeslot - playlist nor the default schedule playlist is specificed the *default show playlist* is broadcasted. - - - - -If none of these playlists have been specified the *Fallback Handling* mechanism takes over (see next chapter). -## Auto DJ - -There might be scenarios where some user has forgotting to assign a playlist to a timeslot or even -missed to define a timeslot. To avoid such broadcast ending up in [Dead Air](https://en.wikipedia.org/wiki/Dead_air), -Engine provides a *Fallback Handling* or *Auto DJ* feature. - -There a two ways to feed music into the Auto DJ: - -- **Audio Folder on your harddrive** -- **[M3U Playlist](https://en.wikipedia.org/wiki/M3U)** - -Both are watched and automatically updated upon content change. Media is played in an randomized order, meaning -they are shuffled and played until nothing is left for the given playlist/folder. Then it starts all over again. - -To configure the behavior of fallbacks, check out the `[fallback]` section in your `engine.ini` configuration. - -Please note that engine also extracts meta-information from music played by the Auto DJ and updates the *Studio -Clock* and *Track Service* feature accordingly. - -It's also possible to report incidents where Auto DJ starts playing to e.g. programme coordinators. To do so -you simple have to define a mail server (see section [Monitoring](#Monitoring)) and a valid email address: - -```ini -# Set to "true" if you want to notify programme-coordinators about about fallback situations, otherwise "false" -mail_coordinator_enabled="true" -# If you want to address multiple programme-coordinators separate their emails by space -coordinator_mail="programme-coordinator@your-radio.org" -``` - -### Silence Detector - -In order to avoid [Dead Air](https://en.wikipedia.org/wiki/Dead_air), the aforementioned Auto DJ feature is -triggered using a Silence Detector reacting to situations where no or unwanted sound is on air. The Silence -Detector allows detection of absoulte silence, weak signals or even noise. - -To configure the sensitivity of the Silence Detector adapt following properties in -`engine.ini`: - -```ini -# max_blank => maximum time of blank from source (defaults to 20., seconds, float) -# min_noise => minimum duration of noise on source to switch back over (defaults to 0, seconds, float) -# threshold => power in dB under which the stream is considered silent (defaults to -40., float) -fallback_max_blank="10." -fallback_min_noise="0." -fallback_threshold="-50." -``` - -Let's assume you want to broadcast a live show. At the given example the Silence Detector will -react upon 10 seconds of silence, it will evaluate if there is a schedule or show fallback playlist -defined. If yes, it will immediately schedule and play such playlist. If there is no such playlist -the station fallback will kick in by playing any randomized music. - -As soon some signal from the live source is sensed again, the audio routing switches back to the live -channel. - -> Note, all these fallback source are fully integrated with the Engine's playlog and track-service -feature including indications from which fallback level some audio is broadcasted. - -## ReplayGain and LoudNorm Normalization - -If the played media files contain [ReplayGain](https://en.wikipedia.org/wiki/ReplayGain) meta information, -then such audio is automatically normalized accordingly. Note, if you are using Engine in conjunction with -[AURA Tank](https://gitlab.servus.at/aura/tank) then also a [LoudNorm](http://k.ylo.ph/2016/04/04/loudnorm.html) -normalization step is applied. In that case files are re-encoded by Tank itself. - -## Monitoring - -You have following options to monitor the Engine: - -- Send mails on errors and warnings -- Engine Status Information -- Engine Heartbeat -- Logging - -### Send mails on errors and warnings - -To activate you'll need to set some mail account within the `[monitoring]` section of your configuration. - -```ini -[monitoring] -mail_server="mail.o94.at" -mail_server_port="587" -mail_user="aura@o94.at" -mail_pass="---SECRET--PASSWORD---" - -# If you want to send multiple adminmails, make them space separated -admin_mail="admin-email@your.domain" - -# Which from mailadress should be used -from_mail="monitoring@aura.engine" - -# The beginning of the subject. With that you can easily apply filter rules using a mail client -mailsubject_prefix="[Aura Engine]" -``` - -### Engine Health Information via Engine API - -Whenever the Engine's status turns into some unhealthy state this is logged to [Engine API](https://gitlab.servus.at/aura/engine-api). -Also, when it returns to some valid state this is logged to the Engine API. - -### Engine Heartbeat - -Instead of checking all status properties, the Heartbeat only validates the vital ones -required to run the engine. If all of those are valid, as network socket request is sent -to a defined server. - -This heartbeat is sent continiously based on the configured `heartbeat_frequency`. - -```ini -# Server where heartbeat info is sent to -heartbeat_server = "127.0.0.1" -# Some UDP port -heartbeat_port = 43334 -# Seconds how often the vitality of the Engine should be checked (0 = disabled) -heartbeat_frequency = 1 -``` - -The service receiving this heartbeat ticks can decide what to do with that information. -One scenario could be switching to another Engine instance or any other failover scenario. -Under `contrib/heartbeat-monitor` you'll find some sample application digesting these -heartbeat signals. - -### Logging - -In development all Engine logs can be found under `./log`, and for production they can -are located in `/var/log/aura`. Adapt the log-level within your configuration to get -more or less verbose log output: - -```ini -logdir="/var/log/aura" - -# Possible values: debug, info, warning, error, critical -loglevel="info" -``` - -The log directory holds individual logs from Engine Core, Liquidsoap and the API. -But also `stout` outputs from supervisor services are written there. - -Additionally you'll finde Supervisor specific logs under`/var/log/supervisor`. - - - -## Read more - -- [Overview](/README.md) -- [Installation for Development](installation-development.md) -- [Installation for Production](installation-production.md) -- [Running with Docker](running-docker.md) -- [Setup the Audio Store](docs/setup-audio-store.md) -- [Developer Guide](developer-guide.md) -- [Engine Features](engine-features.md) -- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md) \ No newline at end of file diff --git a/docs/frequently-asked-questions.md b/docs/frequently-asked-questions.md deleted file mode 100644 index b17bb0dcb9828dd61667e786be29b1539b1c38a6..0000000000000000000000000000000000000000 --- a/docs/frequently-asked-questions.md +++ /dev/null @@ -1,89 +0,0 @@ - -# Frequently Asked Questions - -<!-- TOC --> - -- [Frequently Asked Questions](#frequently-asked-questions) - - [Which Audio Interface / Soundcard is compatible with Aura?](#which-audio-interface--soundcard-is-compatible-with-aura) - - [ALSA Settings](#alsa-settings) - - [In the Liquidsoap Logs I get 'Error when starting output output_lineout_0: Failure("Error while setting open_pcm: Device or resource busy")!'. What does it mean?](#in-the-liquidsoap-logs-i-get-error-when-starting-output-output_lineout_0-failureerror-while-setting-open_pcm-device-or-resource-busy-what-does-it-mean) - - [How can I find the audio device IDs, required for settings in engine.ini?](#how-can-i-find-the-audio-device-ids-required-for-settings-in-engineini) - - [I have issues with starting the Engine](#i-have-issues-with-starting-the-engine) - - [I have issues during some Engine play-out](#i-have-issues-during-some-engine-play-out) - - [Read More](#read-more) - -<!-- /TOC --> - -## Which Audio Interface / Soundcard is compatible with Aura? - -Basically any audio device which is supported by Linux Debian/Ubuntu and has ALSA drivers. -Engine has been tested with following audio interfaces: - -- ASUS Xonar DGX, -- Roland Duo-Capture Ex -- Onboard Soundcard (HDA Intel ALC262) -- Native Instruments Komplete Audio 6 - -## ALSA Settings - -### In the Liquidsoap Logs I get 'Error when starting output output_lineout_0: Failure("Error while setting open_pcm: Device or resource busy")!'. What does it mean? - -You probably have set a wrong or occupied device ID. - - -### How can I find the audio device IDs, required for settings in engine.ini? - -* **ALSA**: You can get the device numbers or IDs by executing: - - cat /proc/asound/cards - -* **Pulse Audio**: You might not need this for Pulse Audio, but still, to see all available devices use: - - pactl list - -**If you cannot find correct ALSA settings** -Well, this is - at least for me - a hard one. I could not manage to find correct ALSA settings for the above mentioned soundcards. The best experience i had with the ASUS Xonar DGX, but still very problematic (especially the first couple of minutes after starting liquidsoap). Since i enabled JACK support i only use that. It is also a bit of trial and error, but works pretty much out of the box. - -**If you experience 'hangs' or other artefacts on the output signal** - - * Reduce the quality (especially, when hangs are on the stream) or - * Check the logs (especially the Liquidsoap logs) for any configuration issues - * Check ther performance of your computer or audio hardware - * Install the realtime kernel with - - ```bash - apt install linux-image-rt-amd64 - reboot - ``` - -## I have issues with starting the Engine - -**Cannot connect to socketpath /opt/aura/engine/src/liquidsoap/engine.sock. Reason: [Errno 111] Connection refused** - -- This indicates some issue with the proper startup of Liquidsoap or any related audio device. Check the Liquidsoap logs for details. - -**[clock.wallclock_alsa:2] Error when starting output lineout: Failure("Error while setting open_pcm: No such file or directory")!** - -- This might indicate some issue with your ALSA device. Check if `aplay -l` and `aplay -L` returns some valid device(s). You can also try `alsamixer`. -- It might be also helpful to set your default audio device in `/etc/asound.conf`. -- Also check if your user (รจ.g. `engineuser`) belongs to the group `audio`. -- Check the audio interface configuration section in `engine.ini`. Verify if the default settings `input_device_0="hw:0"` and `output_device_0="hw:0"` are valid device IDs. - -## I have issues during some Engine play-out - -**Some stuttering sound is hearable. The logs say "strange error flushing buffer ..."** - -- Check your Icecast connection. Is it up and running? -- Maybe there is some authentication issue or an Icecast limitation of max clients. See https://github.com/savonet/liquidsoap/issues/524. - - -## Read More - -- [Overview](/README.md) -- [Installation for Development](installation-development.md) -- [Installation for Production](installation-production.md) -- [Running with Docker](running-docker.md) -- [Setup the Audio Store](docs/setup-audio-store.md) -- [Developer Guide](developer-guide.md) -- [Engine Features](engine-features.md) -- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md) \ No newline at end of file diff --git a/docs/installation-development.md b/docs/installation-development.md deleted file mode 100644 index d24a6822e1a90e0bc2ebb7146b5a38040b44f418..0000000000000000000000000000000000000000 --- a/docs/installation-development.md +++ /dev/null @@ -1,239 +0,0 @@ -# Install for Development - -<!-- TOC --> - -- [Install for Development](#install-for-development) - - [Prerequisites](#prerequisites) - - [Setup Database](#setup-database) - - [Installation](#installation) - - [Configuration](#configuration) - - [Running Engine](#running-engine) - - [Logging](#logging) - - [Read more](#read-more) - -<!-- /TOC --> - -## Prerequisites - -Aura Engine runs on any modern Debian-based OS. It requires at least - -- `Python 3.8+` -- `git` - -Additionally you'll need these system packages: - -```bash -sudo apt-get install \ - opam \ - libsndfile1 \ - ffmpeg \ - quelcom \ - python3-pip \ - virtualenv \ - libssl-dev -``` - -Depending on the database management system you gonna use, you'll also need to install those packages. - -In case of MariaDB this is: - -```shell -sudo apt-get install \ - python3.8-dev \ - default-libmysqlclient-dev \ - mariadb-server \ - libmariadbclient-dev -``` - -**Liquidsoap Repository** - -Engine requires at least `Liquidsoap 1.4.3` or newer, installed using [OPAM (OCaml Package Manager)](https://opam.ocaml.org/). - -Add the current Liquidsoap repository from [this installation guide](https://www.liquidsoap.info/doc-1.4.3/install.html). - -The other steps required for the Liquidsoap installation are handled by the `install.sh` script. If you experience any -errors, carefully review them and consult the official documentation for installing Liquidsoap. - -## Setup Database - -The following installation script sets up the database. - -```bash - bash scripts/setup-db.sh -``` - -By default Aura Engine uses MariaDB for persistence. When starting this script, please -ensure you have root access to your database instance. The installation script automatically -creates a database plus an associated user with password. If you want to use your own database -system, select "Other / Manually" during the database installation step. - -In case of MariaDB the script also installs these system dependencies: - - apt-get install mariadb-server libmariadbclient-dev - -When this is completed, carefully check if any error occured. In case your database has been setup -automatically, note the relevant credentials for later use in your `engine.ini` configuration. - -**Cloning the project** - -Then clone the project to some development folder: - -```bash -git clone https://gitlab.servus.at/autoradio/engine -``` - -## Installation - -Create a virtual environment for your Python dependencies: - -```shell -virtualenv -p python3.8 python -``` - -To activate that environment, run - -```shell -source python/bin/activate -``` - -Install the required dependencies - -```shell -./install.sh -``` - -This script does the following: - -- Install Liquidsoap components using OPAM (`scripts/install-opam-packages`) -- NodeJS Modules (`scripts/install-web.sh`) -- Python Packages (`requirements.txt`) -- Creates a default configuration file in `config/engine.ini` - -When this is completed, carefully check if any error occured. - - -## Configuration - -In your development environment edit following file to configure the engine: - -```shell -./config/engine.ini -``` - -Please note if some configuration exists under `/etc/aura/engine.ini` the configuration is -read from there by default. - -Now, specify at least following settings to get started: - -```ini - [database] - db_user="aura" - db_name="aura_engine" - db_pass="---SECRET--PASSWORD---" -``` - -Set the URLs to the *Steering* and *Tank* API: - -```ini -[api] - -# The URL to get the health status -api_steering_status = "http://aura.local:8000/api/v1/" -# The URL to get the Calendar via Steering -api_steering_calendar="http://aura.local:8000/api/v1/playout" -# The URL to get show details via Steering -api_steering_show="http://aura.local:8000/api/v1/shows/${ID}/" - -## TANK ## - -# The URL to get the health status -api_tank_status = "http://aura.local:8040/healthz/" -# The URL to get playlist details via Tank -api_tank_playlist="http://aura.local:8040/api/v1/playlists/${ID}" - -## ENGINE-API ## - -# Engine ID (1 or 2) -api_engine_number = 1 -# Engine API endpoint to store playlogs -api_engine_store_playlog = "http://localhost:8008/api/v1/playlog" -# Engine API endpoint to store clock information -api_engine_store_clock = "http://localhost:8008/api/v1/clock" -# Engine API endpoint to store health information -api_engine_store_health = "http://localhost:8008/api/v1/source/health/${ENGINE_NUMBER}" -``` - -Ensure that the Liquidsoap installation path is valid: - -```ini -[lqs] -liquidsoap_path="/home/david/.opam/4.08.0/bin/liquidsoap" -``` - -Finally Engine needs to be able to access the audio and playlist folders, where all the tracks of the playlists are stored via *Tank*: - -```ini -[audiosource] -audio_source_folder="/var/audio/source" -audio_source_extension=".flac" -audio_playlist_folder="/var/audio/playlist" -``` - -If the audio device desired for playback is set as `default`, the Engine now should be ready to play -sound. You can check the default audio hardware by executing `aplay -L` on the command line. If that's -not the case you can set the default device in `/etc/asound.conf`. More advanced audio device settings -can be looked up in the [Configuration Guide](docs/configuration-guide.md). - -Read about all other available settings in the [Configuration Guide](docs/configuration-guide.md). - -## Running Engine - -Use the convencience script `run.sh` to get engine started in different ways: - -**Run the Engine** - -If you have defined a virtual env during the installation step you'll ned to activate it whenever -you gonna start the engine: - -```shell -source python/bin/activate -``` - -This includes the Liquidsoap audio engine, but does not start the API server. - -```shell - ./run.sh -``` - -**Run the Engine Core and Liquidsoap separately** - -When developing and debugging engine it is helpful to start the core and the Liquidsoap -component separately. In such case it is important to start both in the correct order. - -First start the core of the engine: - -```shell - ./run.sh core -``` - -When engine-core completed its boot phase, indicated by a log that it is waiting for -Liquidsoap, you can run following: - -```shell - ./run.sh lqs -``` - -## Logging - -All Engine logs for development can be found under `./logs`. - -## Read more - -- [Overview](/README.md) -- [Installation for Development](installation-development.md) -- [Installation for Production](installation-production.md) -- [Running with Docker](running-docker.md) -- [Setup the Audio Store](docs/setup-audio-store.md) -- [Developer Guide](developer-guide.md) -- [Engine Features](engine-features.md) -- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md) \ No newline at end of file diff --git a/docs/installation-production.md b/docs/installation-production.md deleted file mode 100644 index d4398e8b6a39b411bfcd02e72b05d7be45eb278f..0000000000000000000000000000000000000000 --- a/docs/installation-production.md +++ /dev/null @@ -1,361 +0,0 @@ -# Install for Production - -<!-- TOC --> - -- [Install for Production](#install-for-production) - - [Prerequisites](#prerequisites) - - [Installation](#installation) - - [Configuration](#configuration) - - [Running Engine](#running-engine) - - [Running with Systemd](#running-with-systemd) - - [Run the Engine with a single command](#run-the-engine-with-a-single-command) - - [Individual Units for Engine Core and Liquidsoap](#individual-units-for-engine-core-and-liquidsoap) - - [Running with Supervisor](#running-with-supervisor) - - [Logging](#logging) - - [Read more](#read-more) - -<!-- /TOC --> - -## Prerequisites - -Aura Engine runs on any modern Debian-based OS. It requires at least - -- `Python 3.8+` -- `git` - -Additionally you'll need these system packages below. - -```shell -sudo apt-get update -sudo apt-get install \ - supervisor \ - opam \ - libsndfile1 \ - ffmpeg \ - quelcom \ - python3-pip \ - virtualenv \ - libssl-dev -``` - - - -Depending on the database management system you gonna use, you'll also need to install those packages. - -In case of MariaDB this is: - -```shell -sudo apt-get install \ - python-dev \ - default-libmysqlclient-dev \ - mariadb-server \ - libmariadbclient-dev -``` - -**Create an user** - -While previous packages need superuser rights to be installed, the following ones are installed for the user which is -executing the engine. In your development environment you can skip this step. In production you first need to create -a user called `engineuser`. - -```shell - sudo adduser engineuser - sudo adduser engineuser audio sudo -``` - -And switch to that user - -```shell - su engineuser -``` - -**Liquidsoap Repository** - -Engine requires at least `Liquidsoap 1.4.3` or newer, installed using [OPAM (OCaml Package Manager)](https://opam.ocaml.org/). - -Add the current Liquidsoap repository from [this installation guide](https://www.liquidsoap.info/doc-1.4.3/install.html). - -The other steps required for the Liquidsoap installation are handled by the `install.sh` script. If you experience any -errors, carefully review them and consult the official documentation for installing Liquidsoap. - -**Cloning the project** - -Create the folder `/opt/aura` and clone the engine project from there: - -```shell - engineuser:/opt/aura/$ git clone https://gitlab.servus.at/autoradio/engine -``` - -Now you should have `/opt/aura/engine/`. - -Let's move inside the home of engine: - -```shell - engineuser:/opt/aura/$ cd engine -``` - -**Setup the database** - -The following installation script sets up the database. You either need to be logged in as root -or have sudo rights. - -```shell - root:/opt/aura/engine/$ bash scripts/setup-db.sh -``` - -By default Aura Engine uses MariaDB for persistence. When starting this script, please -ensure you have root access to your database instance. The installation script automatically -creates a database plus an associated user with password. If you want to use your own database -system, select "Other / Manually" during the database installation step. - -If you have chosen to setup your database automatically, note the relevant credentials. - - -**Initialize folders and permissions** - -Call this script to create the required log folders and update all permissions. - -```bash - root:/opt/aura/engine$ bash scripts/initialize.sh -``` - -## Installation - -The following installation script also sets up the database. - -By default Aura Engine uses MariaDB for persistence. When starting the installation, please -ensure you have root access to your database instance. The installation script automatically -creates a database plus an associated user with password. If you want to use your own database -system, select "Other / Manually" during the database installation step. - -```shell -engineuser:/opt/aura/engine$ ./install.sh prod -``` - -This script does the following: - -- Install Liquidsoap components using OPAM (`scripts/install-opam-packages`) -- Python Packages (`requirements.txt`) -- Creates a default Engine configuration file in `/etc/aura/engine.ini` - -When this is completed, carefully check if any error occured. In case your database has been setup -automatically, note the relevant credentials for later use in your `engine.ini` configuration. - -## Configuration - -In your production environment edit following file to configure the engine: - -```shell -engineuser:/opt/aura/engine$ nano /etc/aura/engine.ini -``` - -Now, specify at least the following settings to get started: - -```ini - [database] - db_user="aura" - db_name="aura_engine" - db_pass="---SECRET--PASSWORD---" -``` - -Define the API URLs to the *Steering*, *Tank* and *Engine API*: - -```ini -[api] - -# The URL to get the health status -api_steering_status = "http://aura.local:8000/api/v1/" -# The URL to get the Calendar via Steering -api_steering_calendar="http://aura.local:8000/api/v1/playout" -# The URL to get show details via Steering -api_steering_show="http://aura.local:8000/api/v1/shows/${ID}/" - -## TANK ## - -# The URL to get the health status -api_tank_status = "http://aura.local:8040/healthz/" -# The URL to get playlist details via Tank -api_tank_playlist="http://aura.local:8040/api/v1/playlists/${ID}" - -## ENGINE-API ## - -# Engine ID (1 or 2) -api_engine_number = 1 -# Engine API endpoint to store playlogs -api_engine_store_playlog = "http://localhost:8008/api/v1/playlog" -# Engine API endpoint to store clock information -api_engine_store_clock = "http://localhost:8008/api/v1/clock" -# Engine API endpoint to store health information -api_engine_store_health = "http://localhost:8008/api/v1/source/health/${ENGINE_NUMBER}" -``` - -Ensure that the Liquidsoap installation path is valid: - -```ini -[lqs] -liquidsoap_path="/home/engineuser/.opam/4.08.0/bin/liquidsoap" -``` - -**Configuring the Audio Store** - -Finally Engine needs to be able to access the audio and playlist folders, where all the tracks of the playlists are stored via *Tank*: - -```ini -[audiosource] -audio_source_folder="/var/audio/source" -audio_source_extension=".flac" -audio_playlist_folder="/var/audio/playlist" -``` - -There is some document on how to [Setup the Audio Store](docs/setup-audio-store.md). - -If the audio device desired for playback is set as `default`, the Engine now should be ready to play -sound. You can check the default audio hardware by executing `aplay -L` on the command line. If that's -not the case you can set the default device in `/etc/asound.conf`. More advanced audio device settings -can be looked up in the [Configuration Guide](docs/configuration-guide.md). - -Read about all other available settings in the [Configuration Guide](docs/configuration-guide.md). - -## Running Engine - -In production the process of starting the engine is slightly different compared to some development environment. -This is due to the need of ensuring the engine's components are always running i.e. letting them to restart -automatically after some system restart or crash has happened. - -For this you can utilize either [Systemd](https://systemd.io/) or [Supervisor](http://supervisord.org/). - -### Running with Systemd - -Copy the unit files in `/opt/aura/engine/config/systemd/aura-engine.service` to your systemd unit directory, -and reload the systemd daemon: - -```shell -cp /opt/aura/engine/config/systemd/* /etc/systemd/system/ -systemctl daemon-reload -``` - -#### Run the Engine with a single command - -This unit file starts engine-core and engine-liquidsoap within one command. Here Liquidsoap is started as -as subprocess. For more fine-grained control the units in the next section are recommended instead. - -The most common way to start the engine is using the combined command which starts Liquidsoap (`aura-engine-lqs`) -and the Engine Core: - -```shell -systemctl start aura-engine -``` - -Now you are able to stop and restart the scheduler without affecting the playout of Liquidsoap. Note, Liquidsoap -will play as long it has already scheduled data for the current timeslot. If it doesn't it will still the defined -station fallback scenario. - -```shell -systemctl restart aura-engine -systemctl stop aura-engine -systemctl status aura-engine -``` - -#### Individual Units for Engine Core and Liquidsoap - -If you prefer to start both Engine components individually you can use the units `aura-engine-core` and -`aura-engine-lqs`. But keep in mind there is no dependency between those units. If you start Engine Core -the Liquidsoap part will not be started by itself and vice versa. - -**Start Liquidsoap and check its status** - -```shell -sudo systemctl start aura-engine-lqs -sudo systemctl status aura-engine-lqs -``` - -**Start Engine Core and check its status** - -```shell -sudo systemctl start aura-engine-core -sudo systemctl status aura-engine-core -``` - -**Use journalctl to observe the logs** - -```shell -sudo journalctl -f -``` - -### Running with Supervisor - -Now, given you are in the engine's home directory `/opt/aura/engine/`, simply type following to start -the services: - -```shell -supervisord -``` - -This picks up the supervisor configuration provided in the local `supervisord.conf` and the service configurations -located in `config/supervisor/*.conf`. - -Then you'll need to reload the supervisor configuration using `sudo`: - -```shell -sudo supervisorctl reload -``` - -Note that the supervisor daemon starts all (both) services at once. If you want more fine-grained control for -starting services individually, please check-out the next section. - -**Listing available Services** - -```shell -engineuser:/opt/aura/engine$ supervisorctl avail -``` - -You should get these all services with their actual state listed: - -```c++ -aura-engine in use auto 666:666 -aura-engine-lqs in use auto 999:999 -``` - -**Maintanence using Supervisor** - -Please remember to call all `supervisorctl` commands from within your engine home directory (`/opt/aura/engine/`), -to pickup the correct `supervisord.conf`. - -**Starting, stopping and restarting a service** - -```shell - supervisorctl start <service-name> - supervisorctl stop <service-name> - supervisorctl restart <service-name> -``` - -**Reloading the whole supervisor service** - -```shell - sudo service supervisor restart -``` - -## Logging - -All Engine logs for production can be found under: - -```shell - `/var/log/aura` -``` - -and - -```shell - `/opt/aura/engine/logs` -``` - -## Read more - -- [Overview](/README.md) -- [Installation for Development](installation-development.md) -- [Installation for Production](installation-production.md) -- [Running with Docker](running-docker.md) -- [Setup the Audio Store](docs/setup-audio-store.md) -- [Developer Guide](developer-guide.md) -- [Engine Features](engine-features.md) -- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md) \ No newline at end of file diff --git a/docs/running-docker.md b/docs/running-docker.md deleted file mode 100644 index 56a432db5a91f17ce51f39b06d2798ef567a46da..0000000000000000000000000000000000000000 --- a/docs/running-docker.md +++ /dev/null @@ -1,78 +0,0 @@ - -# Running Engine with Docker - -Docker provides a simple way to get your engine with all dependencies running. - -Here you can find the official AURA Engine Docker images: - -https://hub.docker.com/repository/docker/autoradio/engine - -> Note: The Engine Docker image is in *POC* state and waiting to be fully implemented. -It's not yet ready to be offically used. If you want to try AURA Engine meanwhile -try the [Standard Installation](docs/installation-development). - -<!-- TOC --> - -- [Running Engine with Docker](#running-engine-with-docker) - - [Basic configuration](#basic-configuration) - - [Start an image](#start-an-image) - - [Configure an image](#configure-an-image) - - [Making Docker Releases](#making-docker-releases) - - [Build your own, local Docker image](#build-your-own-local-docker-image) - - [Releasing a new version to DockerHub](#releasing-a-new-version-to-dockerhub) - - [Read more](#read-more) - -<!-- /TOC --> - -## Basic configuration - -Create a default configuration and edit according to your settings - -```shell -cp config/sample-docker.engine.ini config/docker/engine.ini -``` - -Create a symlink in `./audio` to point to the audio source of tank - -```shell -ln -s /path/to/tank/audio-store ./audio/source -``` - -## Start an image - -```shell -./run.sh docker:engine -``` - -*To be extended ...* - -## Configure an image - -*To be extended ...* - -## Making Docker Releases - -This section is only relevant if you are an Engine Developer. - -### Build your own, local Docker image - -```shell -./run.sh docker:build -``` - -### Releasing a new version to DockerHub - -```shell -./run.sh docker:push -``` - -## Read more - -- [Overview](/README.md) -- [Installation for Development](installation-development.md) -- [Installation for Production](installation-production.md) -- [Running with Docker](running-docker.md) -- [Setup the Audio Store](docs/setup-audio-store.md) -- [Developer Guide](developer-guide.md) -- [Engine Features](engine-features.md) -- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md) \ No newline at end of file diff --git a/docs/setup-audio-store.md b/docs/setup-audio-store.md deleted file mode 100644 index 30843e118c8d270684754fa4e71d357c22042931..0000000000000000000000000000000000000000 --- a/docs/setup-audio-store.md +++ /dev/null @@ -1,162 +0,0 @@ - -# Setting up the Audio Store - -The *Audio Store* is a folder which is utilized by AURA Tank and Engine to exchange audio files. - -Assuming AURA Engine and Tank are hosted on different machines, the `audio_source_folder` must by shared -using some network share. - -In case you are hosting Engine and Tank on the same machine (e.g. in development), you can skip -this documentation. Just think about pointing them to the same directory. - -<!-- TOC --> - -- [Setting up the Audio Store](#setting-up-the-audio-store) - - [Share Location](#share-location) - - [Share Type](#share-type) - - [Setting up SSHFS](#setting-up-sshfs) - - [Configuring Engine](#configuring-engine) - - [Configuring Tank](#configuring-tank) - - [Read more](#read-more) - -<!-- /TOC --> - -By default Engine expects audio files shared by Tank in `/var/audio/source`. - -This can be configurated in `engine.ini`: - -```ini -[audiosource] -audio_source_folder="/var/audio/source" -``` - -Now, this folder must be somehow writable by Tank. - -## Share Location - -You have following options where your share can be located: - -1. **Engine and all other AURA components (Tank, Dashboard, Steering) are running on the same instance.** This is the most simple solution, - as Engine and Tank can share the same directory locally. But this scenario requires some more sophisticated tuning of the system resources - to avoid e.g. some overload of multiple Uploads in Tank may affect the performance of engine. You can eliminate this risk by setting CPU and - memory limits for Steering, Dashboard and Tank using Docker or `systemd-cgroups`. A disadvantage here is the case of maintainence of system - reboot. This would mean that all components are offline at once. - -2. **Physical directory where the Engine lives, mounted to Tank**. This may cause an issue with the mount, when no network connection to Engine - is unavailable or the instance is rebooting. - -3. **Physical directory where the Tank lives, mounted to Engine.** This may cause an issue with the mount, when no network connection to Tank - is unavailable or the instance is rebooting. - -4. **Central Data Store or *Storage Box*** which is mountet to Engine and Tank. In this case a downtime of the store make both, Engine and Tank - dysfunctional. - -5. **Replicated storage solution using [Gluster](https://www.gluster.org/), both Engine and Tank have their virtual audio directory mounted.** - That's the ideal approach, because if any of the instances is down, the other has all the data available. - -In any case, you should think about some backup solution involving this directory. - -## Share Type - -Then, there's the question how the share is managed. Beside other you have the options -to use [NFS](https://en.wikipedia.org/wiki/Network_File_System), [SSHFS](https://en.wikipedia.org/wiki/SSHFS) or even something like [Gluster](https://www.gluster.org/). - -For our initial setup we have chosen to use *SSHFS*. - -Please share your experience with other share types, and we will include it in further releases of -this documentation. - -## Setting up SSHFS - -SSHFS allows you to access the filesystem on a remote computer via SSH. Interaction with files and folders behaves similar to any local data. - -This example is setting up the `audio_source_folder` on the Engine instance. - -### Configuring Engine - -First, you'll need to create an user which enables Tank to access the `audio_source_folder` on Engine: - -```shell -adduser tankuser -chown tankuser:engineuser /var/audio/source -``` - -Ensure that `engineuser` has no permissions to write the directory: - -```shell -chmod u=+rwx,go=+rx-w /var/audio/source -``` - -### Configuring Tank - -On the Tank side you need to install `sshfs`: - -```shell -sudo apt-get install sshfs -``` - -Then create an `audio-store` folder inside the AURA home: - -```shell -:/opt/aura/$ mkdir audio-store -``` - -Try if you can connect to the engine over SSH using your `tankuser`: - -```shell -ssh tankuser@192.168.0.111 -p22 -``` - -Replace `-p22` with the actual port number your SSH service is running with. For security reasons it's recommended -to run SSH not over the default port 22. - -Uncomment following setting in `/etc/fuse.conf` to allow the tank-user access the share with write permissions: - -```conf -# Allow non-root users to specify the allow_other or allow_root mount options. -user_allow_other -``` - -Now create the mount: - -```shell -sudo sshfs -o allow_other -o IdentityFile=~/.ssh/id_rsa tankuser@192.168.0.111:/var/audio/source /opt/aura/audio-store -p22 -``` - -Replace `192.168.0.111` with the actual IP for your Engine and `-p22` with the actual port number your Engine's SSH service -is running with. - -To make this mount persistent i.e. keep it alive even after a system reboot, you'll need to add a configuration -in the `/etc/fstab` file by adding this at the end: - -```yaml -# Audio Store @ AURA Engine -sshfs#tankuser@192.168.0.111:/var/audio/source /opt/aura/audio-store fuse auto,port=22,identityfile=~/.ssh/id_rsa,allow_other,_netdev 0 0 - -``` - -Again, check for the correct port number in the line above. - -To take this into effect you'll need to remount the filesystem with `sudo mount -a` or reboot the machine. When mounting -you'll need to authenticate with the `tankuser` password once. - -Then review if your Tank's Docker configuration mounts the exact same volume (`/opt/aura/audio-store`). -If not edit the tank configuration `/etc/aura/tank.env` and set following property: - -```shell -TANK_STORE_PATH=/opt/aura/audio-store -``` - -Finally, do some testing if the directory is writable from Tank's system (`touch some-file`) and if the Engine's side can read -this file. Then restart your Tank Docker container and you should be good to go. - -## Read more - -- [Overview](/README.md) -- [Installation for Development](installation-development.md) -- [Installation for Production](installation-production.md) -- [Running with Docker](running-docker.md) -- [Setup the Audio Store](docs/setup-audio-store.md) -- [Developer Guide](developer-guide.md) -- [Engine Features](engine-features.md) -- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md)