Documentation for scheduling and fallbacks.

docs/engine-features.md

@@ -5,15 +5,16 @@ This page gives a more detailed overview of the Aura Engine features and how to
 <!-- TOC -->
 
 - [Aura Engine Features](#aura-engine-features)
-    - [Input](#input)
-        - [Play audio from multiple sources](#play-audio-from-multiple-sources)
-        - [Dynamic switching of sources](#dynamic-switching-of-sources)
-    - [Output](#output)
+    - [Multi-channel Input (Filesystem, Stream, Analog)](#multi-channel-input-filesystem-stream-analog)
+    - [Multi-channel output](#multi-channel-output)
+        - [Analog line-out](#analog-line-out)
+        - [Stream to Icecast](#stream-to-icecast)
         - [Record output to filesystem](#record-output-to-filesystem)
-        - [Stream output to an Icecast Server](#stream-output-to-an-icecast-server)
-        - [Multichannel Line-out](#multichannel-line-out)
-    - [Blank Detenction / Silence Detecter](#blank-detenction--silence-detecter)
-    - [Auto Pilot a.k.a. Fallback Handling](#auto-pilot-aka-fallback-handling)
+    - [Secure Scheduling](#secure-scheduling)
+        - [Fallback Stages](#fallback-stages)
+        - [Scheduling Stages](#scheduling-stages)
+        - [Pro-active Fallback Handling (1st Level Fallback)](#pro-active-fallback-handling-1st-level-fallback)
+        - [Fallback Handling using the Silence Detector (2nd Level Fallback)](#fallback-handling-using-the-silence-detector-2nd-level-fallback)
     - [API Endpoints](#api-endpoints)
     - [Web Applications](#web-applications)
     - [Monitoring](#monitoring)
@@ -25,53 +26,157 @@ This page gives a more detailed overview of the Aura Engine features and how to
 
 <!-- /TOC -->
 
-## Input
+## Multi-channel Input (Filesystem, Stream, Analog)
 
-### Play audio from multiple sources
+It's possible to schedules playlists with music or pre-recorded show stored on the **filessystem**,
+via external **streams** or live from an **analog input** in the studio. All types of sources can
+be mixed in a single playlist.
 
-It's possible to air playlists with music or recordings stored on the **filessystem**,
-via external **streams** or live from a **studio**.
+> Note: Since live sources and streams do not specific a length property, any playlists populated
+with such source are expecting this item to run until the end of the schedule. Any other entries
+following such item are therfore skipped. This might change in a future version of AURA Tank,
+allowing to set a length property.
 
-### Dynamic switching of sources
+The switching between types of audio source is handled automatically. To learn more check-out the
+[Scheduling](### Secure Scheduling) section.
 
-    TODO extend: * switch the soundserver at the correct time to a given source for a specific show
+## Multi-channel output
 
-## Output
+### Analog line-out
+
+In most scenarios it might be sufficient to broadcast via only one analog stereo output channel.
+If need you can configure up to five stereo pairs.
+
+### Stream to Icecast
+
+Engine allows to stream to multiple Icecast Servers simultaniousely. It is also sending meta information
+to the streaming server, using the *Icy* protocol.
+
+To configure your Icecast connectivity check-out the `[stream]` section in your configuration.
 
 ### Record output to filesystem
 
-    TODO extend:  * record what is broadcasted
+Engine allows recording broadcasts to the filesystem using up to multiple recorders. Each recorder can
+be configured to save in a different resolution in audio quality.
+
+## Secure Scheduling
+
+Engine performs scheduling in a secured way, to meet the requirements of community
+radios. It tries to pro-actively reacted to scenarios, where the upload of some
+pre-recorded show has been forgotten, or some live schedule is not taking place.
+
+Usually in such cases the broadcast might end up with some timeslot filled with silence.
+To avoid this, Engine implements multiple levels of fallback handling.
 
-### Stream output to an Icecast Server
+### Fallback Stages
 
-    TODO extend:  * stream to an icecast server
+The available fallbacks are evaluated in following order:
 
-### Multichannel Line-out
+1. **Show Fallback**: If the schedule for some show has no playlist assigned, the
+    playlist assigned as a *show fallback* is used instead. In the dashboard this can
+    be done as seen in the screenshot below.
 
-    TODO extend:  * play to line-out
+    ![Setting for the Show Fallback in AURA Dashboard](images/dashboard-fallback-setting.png "Show Fallback in Dashboard")
+
+2. **Timeslot Fallback**: If the show fallback is not assigned, a configured fallback
+    playlist for the related timeslot is used.
+
+3. **Station Fallback**: If everything goes wrong, meaning all the previous fallback
+    playlists are missing or are invalid, the *station fallback* will be triggered. This
+    fallback type is specified by a playlist ID set in the Engine's configuration
+    (see `scheduling_station_fallback_id` in `engine.ini`)
+
+### Scheduling Stages
+
+To understand how fallbacks are handled by Engine, it's good to know how the Engine's
+scheduling is structured. Below you see a timline with one schedule planned at a certain
+point in time and the involved phase before:
+
+```
+=====[   Scheduling Window   ]-[   Pre-Rolling   ]-[   Schedule Play-out   ]=====
+```
 
-## Blank Detenction / Silence Detecter
+- **Scheduling Window**: Within the scheduling window any commands for controlling
+    the mixer of the soundsystem are prepared and queued.
 
-The engine offers a simple way to detect scenarios where no music is on air.
-It possible to configure the sensitivity of the Silence Detector and automatically
-transition play-out to a Fallback Playlist (see Auto Pilot).
+    Until the start of the window, schedules can be added or removed
+    via external API Endpoints (e.g. using Steering or Dashboard). Until here any
+    changes on the schedule itself will be reflected in the actual play-out. Be aware,
+    that this only affects the schedule ("timeslot") itself, it does not involve related
+    playlists and their entries. The latter can still be modified within the scheduling
+    window.
 
-## Auto Pilot a.k.a. Fallback Handling
+    The start and the end of the window is defined by the start of the schedule minus
+    a configured amount of seconds (see `scheduling_window_start` and `scheduling_window_end`
+    in `engine.ini`).
 
-In case there is no schedule delivered by Steering, engine provides multiple
-fallback handling scenarios. The available fallbacks are evaluated in following order:
+    During the scheduling window, the external API Endpoints are pulled continiously, to
+    check for updated schedules and related playlists. Also, any changes to playlists and
+    its entries are respected within that window (see `fetching_frequency` in `engine.ini`).
 
-1. **Timeslot Fallback**:
+    <u>Pro-active fallbach handling</u> is already happening at this phase.
 
-2. **Show Fallback**:
+    > Important: It's vital that the the scheduling window is wider than the fetching frequency.
+    Otherwise one fetch might never hit a scheduling window, hence not being able to schedule stuff.
 
-3. **Station Fallback**:
+- **Pre-Rolling**: Now, the playlist entries are going to be "pre-rolled". This means that filesystem
+    entries are pre-loaded and entries which are based on audio streams are buffered. This
+    is required to allow a seamless play-out, when its time to do so (in the next stage).
+    Due to their nature, playlist entries which hold live audio sources are not affected by
+    this stage at all.
 
-3.1 Station Fallback via remote playlist
+    Within this stage users can neither change playlists nor its entries anymore. <u>Pro-active
+    fallback handling</u> will happen though (see next chapter below for an explanation).
 
-3.2 Station Fallback via local music folder
+    Set the maximum time reserved for pre-rolling in your configuration (compare `preroll_offset`
+    in `engine.ini`).
+
+    > Important: This stage can require up to 4x pre-rolling steps, one for default play-out
+    and for each of the 3 fallback types. So the expected time to load some source multiplied
+    by 4 should be set as the actual value. Also, the offset should not exceed the time between
+    the end of the scheduling-window and the start of the actual schedule playout.
+
+- **Schedule-Playout**: Finally the actual play-out is happening. The faders of the virtual
+    mixers are pushed all the way up and transitions between playlist entries with different
+    types of sources (file, stream and analog inputs) are performed automatically. At the end
+    of each schedule the channel is faded-out, no matter if the total length of the playlist
+    entries would require a longer timeslot.
+
+    If for some reason the playout is corrupted, stopped or too silent to make any sense, then
+    this <u>triggers a 2nd level fallback using the silence detector</u> (see chapter below).
+
+### Pro-active Fallback Handling (1st Level Fallback)
+
+This type of fallback handling is performed before the actual playout, within the scheduling
+window and pre-rolling stage. Here Engine pro-actively verifies the validity of the playlists.
+For example it checks if some playlist ist existing at all, if it holds entries or if the
+entries are existing.
+
+Furthermore, it checks for any unexpected error when entries are pre-rolled. For example Engine
+tries to buffer some HTTP Audio Stream, but it is not reachable. In this case this will
+trigger the scheduling of some fallback playlist.
+
+Pro-active fallback handling has the advantage that any (even short) periods of silence
+during playlist can be avoided. Still, there are cases, where any failure cannot be detected
+beforehand. Therefore a 2nd level fallback solution is required (see below).
+
+### Fallback Handling using the Silence Detector (2nd Level Fallback)
+
+Engine offers a simple way to detect situations where no sound is on air, the so-called
+Silence Detector. It allows detection of absoulte silence, weak signals or even noise.
+
+To configure the sensitivity of the Silence Detector check-out following properties in
+`engine.ini`:
+
+```
+# max_blank => maximum time of blank from source (defaults to 20., seconds, float)
+# min_noise => minimum duration of noise on source to switch back over (defaults to 0, seconds, float)
+# threshold => power in dB under which the stream is considered silent (defaults to -40., float)
+fallback_max_blank="20."
+fallback_min_noise="0."
+fallback_threshold="-50."
+```
 
-Note: All filenames in the music folder should be ASCII encoded, otherwise this may cause engine failures.
 
 ## API Endpoints