docs/running-docker.md

+
+# Running Engine with Docker
+
+Docker provides a simple way to get your engine with all dependencies running.
+
+Here you can find the official AURA Engine Docker images:
+
+https://hub.docker.com/repository/docker/autoradio/engine
+
+> Note: The Engine Docker image is in *POC* state and waiting to be fully implemented.
+It's not yet ready to be offically used. If you want to try AURA Engine meanwhile
+try the [Standard Installation](docs/installation-development).
+
+<!-- TOC -->
+
+- [Running Engine with Docker](#running-engine-with-docker)
+    - [Basic configuration](#basic-configuration)
+    - [Start an image](#start-an-image)
+    - [Configure an image](#configure-an-image)
+    - [Making Docker Releases](#making-docker-releases)
+        - [Build your own, local Docker image](#build-your-own-local-docker-image)
+        - [Releasing a new version to DockerHub](#releasing-a-new-version-to-dockerhub)
+    - [Read more](#read-more)
+
+<!-- /TOC -->
+
+## Basic configuration
+
+Create a default configuration and edit according to your settings
+
+```shell
+cp configuration/sample-docker.engine.ini configuration/docker/engine.ini
+```
+
+Create a symlink in `./audio` to point to the audio source of tank
+
+```shell
+ln -s /path/to/tank/audio-store ./audio/source
+```
+
+## Start an image
+
+```shell
+./run.sh docker:engine
+```
+
+*To be extended ...*
+
+## Configure an image
+
+*To be extended ...*
+
+## Making Docker Releases
+
+This section is only relevant if you are an Engine Developer.
+
+### Build your own, local Docker image
+
+```shell
+./run.sh docker:build
+```
+
+### Releasing a new version to DockerHub
+
+```shell
+./run.sh docker:push
+```
+
+## Read more
+
+- [Overview](/README.md)
+- [Installation for Development](installation-development.md)
+- [Installation for Production](installation-production.md)
+- [Running with Docker](running-docker.md)
+- [Setup the Audio Store](docs/setup-audio-store.md)
+- [Developer Guide](developer-guide.md)
+- [Engine Features](engine-features.md)
+- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md)
\ No newline at end of file

docs/setup-audio-store.md

+
+# Setting up the Audio Store
+
+The *Audio Store* is a folder which is utilized by AURA Tank and Engine to exchange audio files.
+
+Assuming AURA Engine and Tank are hosted on different machines, the `audio_source_folder` must by shared 
+using some network share. 
+
+In case you are hosting Engine and Tank on the same machine (e.g. in development), you can skip 
+this documentation. Just think about pointing them to the same directory.
+
+<!-- TOC -->
+
+- [Setting up the Audio Store](#setting-up-the-audio-store)
+    - [Share Location](#share-location)
+    - [Share Type](#share-type)
+    - [Setting up SSHFS](#setting-up-sshfs)
+        - [Configuring Engine](#configuring-engine)
+        - [Configuring Tank](#configuring-tank)
+    - [Read more](#read-more)
+
+<!-- /TOC -->
+
+By default Engine expects audio files shared by Tank in `/var/audio/source`.
+
+This can be configurated in `engine.ini`:
+
+```ini
+[audiosource]
+audio_source_folder="/var/audio/source"
+```
+
+Now, this folder must be somehow writable by Tank.
+
+## Share Location
+
+You have following options where your share can be located:
+
+1. **Engine and all other AURA components (Tank, Dashboard, Steering) are running on the same instance.** This is the most simple solution,
+  as Engine and Tank can share the same directory locally. But this scenario requires some more sophisticated tuning of the system resources
+  to avoid e.g. some overload of multiple Uploads in Tank may affect the performance of engine. You can eliminate this risk by setting CPU and
+  memory limits for Steering, Dashboard and Tank using Docker or `systemd-cgroups`. A disadvantage here is the case of maintainence of system
+  reboot. This would mean that all components are offline at once.
+
+2. **Physical directory where the Engine lives, mounted to Tank**. This may cause an issue with the mount, when no network connection to Engine
+   is unavailable or the instance is rebooting.
+
+3. **Physical directory where the Tank lives, mounted to Engine.** This may cause an issue with the mount, when no network connection to Tank 
+   is unavailable or the instance is rebooting.
+
+4. **Central Data Store or *Storage Box*** which is mountet to Engine and Tank. In this case a downtime of the store make both, Engine and Tank
+   dysfunctional.
+
+5. **Replicated storage solution using [Gluster](https://www.gluster.org/), both Engine and Tank have their virtual audio directory mounted.**
+   That's the ideal approach, because if any of the instances is down, the other has all the data available.
+
+In any case, you should think about some backup solution involving this directory.
+
+## Share Type
+
+Then, there's the question how the share is managed. Beside other you have the options
+to use [NFS](https://en.wikipedia.org/wiki/Network_File_System), [SSHFS](https://en.wikipedia.org/wiki/SSHFS) or even something like [Gluster](https://www.gluster.org/).
+
+For our initial setup we have chosen to use *SSHFS*.
+
+Please share your experience with other share types, and we will include it in further releases of
+this documentation.
+
+## Setting up SSHFS
+
+SSHFS allows you to access the filesystem on a remote computer via SSH. Interaction with files and folders behaves similar to any local data.
+
+This example is setting up the `audio_source_folder` on the Engine instance.
+
+### Configuring Engine
+
+First, you'll need to create an user which enables Tank to access the `audio_source_folder` on Engine:
+
+```shell
+adduser tankuser
+chown tankuser:engineuser /var/audio/source
+```
+
+Ensure that `engineuser` has no permissions to write the directory:
+
+```shell
+chmod u=+rwx,go=+rx-w /var/audio/source
+```
+
+### Configuring Tank
+
+On the Tank side you need to install `sshfs`:
+
+```shell
+sudo apt-get install sshfs
+```
+
+Then create an `audio-store` folder inside the AURA home:
+
+```shell
+:/opt/aura/$ mkdir audio-store
+```
+
+Try if you can connect to the engine over SSH using your `tankuser`:
+
+```shell
+ssh tankuser@192.168.0.111 -p22
+```
+
+Replace `-p22` with the actual port number your SSH service is running with. For security reasons it's recommended
+to run SSH not over the default port 22.
+
+Uncomment following setting in `/etc/fuse.conf` to allow the tank-user access the share with write permissions:
+
+```conf
+# Allow non-root users to specify the allow_other or allow_root mount options.
+user_allow_other
+```
+
+Now create the mount:
+
+```shell
+sudo sshfs -o allow_other -o IdentityFile=~/.ssh/id_rsa tankuser@192.168.0.111:/var/audio /opt/aura/audio-store -p22
+```
+
+Replace `192.168.0.111` with the actual IP for your Engine and `-p22` with the actual port number your Engine's SSH service
+is running with.
+
+To make this mount persistent i.e. keep it alive even after a system reboot, you'll need to add a configuration
+in the `/etc/fstab` file by adding this at the end:
+
+```yaml
+# Audio Store @ AURA Engine
+sshfs#tankuser@192.168.0.111:/var/audio /opt/aura/audio-store fuse auto,port=22,identityfile=~/.ssh/id_rsa,allow_other,_netdev 0 0
+
+```
+
+Again, check for the correct port number in the line above.
+
+To take this into effect you'll need to remount the filesystem with `sudo mount -a` or reboot the machine. When mounting
+you'll need to authenticate with the `tankuser` password once.
+
+Then review if your Tank's Docker configuration mounts the exact same volume (`/opt/aura/audio-store`).
+If not edit the tank configuration `/etc/aura/tank.env` and set following property:
+
+```shell
+TANK_STORE_PATH=/opt/aura/audio-store
+```
+
+Finally, do some testing if the directory is writable from Tank's system (`touch some-file`) and if the Engine's side can read
+this file. Then restart your Tank Docker container and you should be good to go.
+
+## Read more
+
+- [Overview](/README.md)
+- [Installation for Development](installation-development.md)
+- [Installation for Production](installation-production.md)
+- [Running with Docker](running-docker.md)
+- [Setup the Audio Store](docs/setup-audio-store.md)
+- [Developer Guide](developer-guide.md)
+- [Engine Features](engine-features.md)
+- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md)

engine-core.py

+#!/bin/sh
+''''which python3.8 >/dev/null 2>&1 && exec python3.8 "$0" "$@"     # '''
+''''which python3.7  >/dev/null 2>&1 && exec python3.7  "$0" "$@"   # '''
+''''exec echo "Error: Snaaakey Python, where are you?"                # '''
+
+#
+# Aura Engine (https://gitlab.servus.at/aura/engine)
+#
+# Copyright (C) 2017-2020 - The Aura Engine Team.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+import os
+import sys
+import signal
+import logging
+import subprocess
+
+from flask               import Flask
+from flask_sqlalchemy    import SQLAlchemy
+
+from modules.base.logger import AuraLogger
+from modules.base.config import AuraConfig
+from modules.base.utils  import SimpleUtil
+
+
+config = AuraConfig()
+def configure_flask():
+    app.config["SQLALCHEMY_DATABASE_URI"] = config.get_database_uri()
+    app.config['BABEL_DEFAULT_LOCALE'] = 'de'
+    app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
+
+# FIXME Instatiate SQLAlchemy without the need for Flask
+app = Flask(__name__)
+configure_flask()
+DB = SQLAlchemy(app)
+
+
+class AuraEngine:
+    """
+    AuraEngine does the following:
+
+    1. Initialize the engine and scheduler
+    2. Initialize Redis
+    3. Start Liquidsoap in a separate thread which connects to the engine
+
+    """
+    logger = None
+    config = None
+    server = None
+    messenger = None
+    controller = None
+    engine = None
+    scheduler = None
+    lqs = None
+    lqs_startup = None
+
+
+
+    def __init__(self):
+        """
+        Initializes Engine Core.
+        """
+        self.config = config
+        AuraLogger(self.config)
+        self.logger = logging.getLogger("AuraEngine")
+        
+
+
+
+    def startup(self, lqs_startup):
+        """
+        Starts Engine Core.
+        """                
+        from modules.scheduling.scheduler import AuraScheduler
+        from modules.core.engine import Engine
+        from modules.cli.redis.adapter import ServerRedisAdapter
+
+        # If Liquidsoap should be started automatically
+        self.lqs_startup = lqs_startup
+
+        # Check if the database has to be re-created
+        if self.config.get("recreate_db") is not None:
+            AuraScheduler(self.config, None, None)
+
+        # Create scheduler and Liquidsoap communicator
+        self.engine = Engine(self.config)
+        self.scheduler = AuraScheduler(self.config, self.engine, self.on_initialized)
+
+        # Create the Redis adapter
+        self.messenger = ServerRedisAdapter(self.config)
+        self.messenger.scheduler = self.scheduler
+        self.messenger.engine = self.engine
+
+        # And finally wait for redis message / start listener thread
+        self.messenger.start()
+
+
+
+    def on_initialized(self):
+        """
+        Called when the engine is initialized, before the Liquidsoap connection is established."
+        """
+        self.logger.info(SimpleUtil.green("Engine Core initialized - Waiting for Liquidsoap connection ..."))
+        if self.lqs_startup:
+            self.start_lqs(False, False)
+        else:
+            self.logger.info(SimpleUtil.yellow("Please note, Liquidsoap needs to be started manually."))
+
+
+    def start_lqs(self, debug_output, verbose_output):
+        """
+        Starts Liquidsoap.
+        """
+        lqs_path = self.config.get("liquidsoap_path")
+        lqs_cwd = os.getcwd() + "/" + self.config.get("liquidsoap_working_dir")
+        lqs_output = ""
+        lqs_output = self.get_debug_flags(debug_output, verbose_output)
+
+        self.lqs = subprocess.Popen([lqs_path, lqs_output, "engine.liq"], \
+            cwd=lqs_cwd, \
+            stdout=subprocess.PIPE, \
+            shell=False)
+
+
+
+    def get_lqs_cmd(self, debug_output, verbose_output):
+        """
+        Returns a shell command string to start Liquidsoap
+        """
+        lqs_path = self.config.get("liquidsoap_path")
+        lqs_cwd = os.getcwd() + "/" + self.config.get("liquidsoap_working_dir")
+        lqs_output = self.get_debug_flags(debug_output, verbose_output)
+        return "(cd %s && %s %s ./engine.liq)" % (lqs_cwd, lqs_path, lqs_output)
+
+
+
+    def get_debug_flags(self, debug_output, verbose_output):
+        """
+        Build Liquidsoap debug parameters.
+        """
+        output = ""
+        if debug_output:
+            output += "--debug " 
+        if verbose_output:
+            output += "--verbose "
+        return output
+
+
+
+    def exit_gracefully(self, signum, frame):
+        """
+        Shutdown of the engine. Also terminates the Liquidsoap thread.
+        """
+        if self.lqs:
+            self.lqs.terminate()
+            self.logger.info("Terminated Liquidsoap")
+
+        if self.engine:
+            self.engine.terminate()
+                
+        if self.messenger:
+            self.messenger.terminate()
+        
+        self.logger.info("Gracefully terminated Aura Engine! (signum:%s, frame:%s)" % (signum, frame))
+        sys.exit(0)        
+
+
+
+#
+# START THE ENGINE
+#
+
+
+if __name__ == "__main__":        
+    engine = AuraEngine()
+    start_lqs = True
+    lqs_cmd = False
+
+    signal.signal(signal.SIGINT, engine.exit_gracefully)
+    signal.signal(signal.SIGTERM, engine.exit_gracefully)
+
+    if len(sys.argv) >= 2:
+        if "--without-lqs" in sys.argv:
+            start_lqs = False
+        if "--get-lqs-command" in sys.argv:
+            lqs_cmd = True
+        if "--use-test-data" in sys.argv:
+            engine.config.set("use_test_data", True)
+        if "--recreate-database" in sys.argv:
+            engine.config.set("recreate_db", True)
+
+    if lqs_cmd:
+        print(engine.get_lqs_cmd(True, True))
+    else:
+        engine.startup(start_lqs)

meta.py

+# Meta
+__author__ = "David Trattnig and Gottfried Gaisbauer"
+__copyright__ = "Copyright 2017-2020, Aura Engine Team"
+__credits__ = ["David Trattnig", "Gottfried Gaisbauer", "Michael Liebler"]
+__license__ = "GNU Affero General Public License (AGPL) Version 3"
+__version__ = "0.8.2"
+__version_info__ = (0, 8, 2)
+__maintainer__ = "David Trattnig"
+__email__ = "david.trattnig@subsquare.at"
+__status__ = "Development"
\ No newline at end of file

modules/cli/redis/channels.py

+#
+# Aura Engine (https://gitlab.servus.at/aura/engine)
+#
+# Copyright (C) 2017-2020 - The Aura Engine Team.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+
+from enum import Enum
+
+
+
+class RedisChannel(Enum):
+    STANDARD = "aura"
+
+    DPE_REPLY = "delete_playlist_entry_reply"
+    FNP_REPLY = "fetch_new_programme_reply"
+    GAP_REPLY = "get_act_programme_reply"
+    GS_REPLY  = "get_status_reply"
+    GCS_REPLY = "get_connection_status_reply"
+    GNF_REPLY = "get_next_file_reply"
+    IPE_REPLY = "insert_playlist_entry_reply"
+    IP_REPLY  = "init_player_reply"
+    TS_REPLY  = "track_service_reply"
+    MPE_REPLY = "move_playlist_entry_reply"
+    PMQ_REPLY = "print_message_queue_reply"
+    RDB_REPLY = "recreate_database_reply"
+    SNF_REPLY = "get_next_file_reply"