Skip to content
Snippets Groups Projects
Commit 0751cf0e authored by David Trattnig's avatar David Trattnig
Browse files

Reworked docs. #43 #44

parent 6fa9a7f4
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
docs/engine-features.md
+ 60
− 95
View file @ 0751cf0e
......@@ -9,11 +9,9 @@ This page gives a more detailed overview of the Aura Engine features and how to
- [Multi-channel output](#multi-channel-output)
- [Analog line-out](#analog-line-out)
- [Stream to Icecast](#stream-to-icecast)
- [Record output to filesystem](#record-output-to-filesystem)
- [Scheduling](#scheduling)
- [Fallback Handling](#fallback-handling)
- [Pro-active Fallback Handling (1st Level Fallback)](#pro-active-fallback-handling-1st-level-fallback)
- [Fallback Handling using the Silence Detector (2nd Level Fallback)](#fallback-handling-using-the-silence-detector-2nd-level-fallback)
- [Silence Detector](#silence-detector)
- [Monitoring](#monitoring)
- [Send mails on errors and warnings](#send-mails-on-errors-and-warnings)
- [Engine Health Information via Engine API](#engine-health-information-via-engine-api)
......@@ -29,10 +27,8 @@ It's possible to schedules playlists with music or pre-recorded show stored on t
via external **streams** or live from an **analog input** in the studio. All types of sources can
be mixed in a single playlist.
> Note: Since live sources and streams do not specify a length property, any playlists populated
with such source are expecting this item to run until the end of the schedule. Any other entries
following such item are therfore skipped. This might change in a future version of AURA Tank,
allowing to set a length property.
> Note: Any live sources or streams not specifing a length property, are automatically expanded to
the left duration of the timeslot.
The switching between types of audio source is handled automatically. To learn more check out the
[Scheduling](### Secure Scheduling) section.
......@@ -51,11 +47,6 @@ to the streaming server, using the *Icy* protocol.
To configure your Icecast connectivity check-out the `[stream]` section in your configuration.
### Record output to filesystem
Engine allows recording broadcasts to the filesystem using up to multiple recorders. Each recorder can
be configured to save in a different resolution in audio quality.
## Scheduling
Engine provide a scheduling functionality by polling external API endpoints frequently.
......@@ -63,24 +54,23 @@ Engine provide a scheduling functionality by polling external API endpoints freq
Scheduling is split into multiple phase. Below you see a timline with one schedule planned at a certain
point in time and the involved phase before:
```
================= [ Scheduling Window ] ============= [ Schedule Play-out ] ====
```ascii
================= [ Scheduling Window ] ============ [ Timeslot Play-out ] ====
== (FILESYSTEM A) ========================== [ Pre-Roll ] [ Play 4 ] ====================================
== (STREAM A) ========================================== [ Pre-Roll ] [ Play 1 ] ==========================
== (LIVE 1) ====================================================== [ Pre-Roll ] [ Play 1 ] ================
== (FILESYSTEM B) ========================================================== [ Pre-Roll ] [ Play 4 ] ====
== (FILESYSTEM A) ========================== [ Preload ] [ Play 4 ] ====================================
== (STREAM A) ========================================== [ Preload ] [ Play 1 ] ==========================
== (LIVE 1) ====================================================== [ Preload ] [ Play 1 ] ================
== (FILESYSTEM B) ========================================================== [ Preload ] [ Play 4 ] ====
```
- **Scheduling Window**: Within the scheduling window any commands for controlling
the mixer of the soundsystem are prepared and queued.
Until the start of the window, schedules can be added or removed
via external API Endpoints (e.g. using Steering or Dashboard). Until here any
changes on the schedule itself will be reflected in the actual play-out. Be aware,
that this only affects the schedule ("timeslot") itself, it does not involve related
playlists and their entries. The latter can still be modified within the scheduling
window.
Until the start of the window, timeslot can be added or removed via external API Endpoints
(e.g. using Steering or Dashboard). Until here any changes on the timeslot itself will be reflected
in the actual play-out. This only affects the start and end time of the "timeslot" itself.
It does not involve related playlists and their entries. Those can still be modified after the
scheduling window has started.
The start and the end of the window is defined by the start of the schedule minus
a configured amount of seconds (see `scheduling_window_start` and `scheduling_window_end`
......@@ -90,120 +80,95 @@ point in time and the involved phase before:
check for updated schedules and related playlists. Also, any changes to playlists and
its entries are respected within that window (see `fetching_frequency` in `engine.ini`).
<u>Pro-active fallback handling</u> by verifying the existence of playlists is already
happening at this stage.
> Important: It's vital that the the scheduling window is wider than the fetching frequency.
Otherwise one fetch might never hit a scheduling window, hence not being able to schedule stuff.
- **Queuing and Pre-Rolling**: Before any playlist entries of the schedule can be turned into
sound, they need to be grouped, queued an pre-rolled.
Within this stage users can neither change playlists nor its entries anymore. <u>Pro-active
fallback handling</u> will happen though (see next chapter below for an explanation).
1. First, all entries are aggregated when they hold filesystem entries.
Given you have a playlist with 10 entries, the first 4 are consisting of files, the next two
of a a stream and a live source. The last 4 are files again. These entries are now
aggregated into 4 groups: one for the files, one for the stream, one for the live entry
and another one for files. For each group a timer for executing the next step is created.
- **Queuing and Pre-Loading**: Before any playlist entries of the schedule can be turned into
sound, they need to be queued and pre-loaded. Ideally the pre-loading happens somewhat before
the scheduled play-out time to avoid any delays in timing. Set the maximum time reserved for
pre-loading in your configuration (compare `preload_offset`in `engine.ini`).
2. Now, the playlist entries are going to be "pre-rolled". This means that filesystem
entries are queued and pre-loaded and entries which are based on audio streams are buffered.
This is required to allow a seamless play-out, when its time to do so (in the next stage).
Due to their nature, playlist entries which hold live audio sources are not affected by
this stage at all.
Set the maximum time reserved for pre-rolling in your configuration (compare `preroll_offset`
in `engine.ini`).
> Important: This stage can require up to 4x pre-rolling steps, one for default play-out
and for each of the 3 fallback types. So the expected time to load some source multiplied
by 4 should be set as the actual value. Also, the offset should not exceed the time between
the end of the scheduling-window and the start of the actual schedule playout.
> Important: The offset should not exceed the time between the end of the scheduling-window and the
start of the actual schedule playout.
- **Play-out**: Finally the actual play-out is happening. The faders of the virtual mixers are pushed
all the way up, as soon it's "time to play" for one of the pre-rolled entries.
all the way up, as soon it's "time to play" for one of the pre-loaded entries.
Transitions between playlist entries with different types of sources (file, stream and analog
inputs) are performed automatically. At the end of each schedule the channel is faded-out,
no matter if the total length of the playlist entries would require a longer timeslot.
Since each scheduled playlist can consist of multiple entry types such as *file*, *live*,
and *stream*, the play-out of the schedule is actually a bit more complex. The paragraphs
below go into more details.
If for some reason the playout is corrupted, stopped or too silent to make any sense, then
this <u>triggers a 2nd level fallback using the silence detector</u> (see chapter below).
this <u>triggers a fallback using the silence detector</u> (see chapter below).
## Fallback Handling
Engine performs scheduling with some additional handling for extended security, to meet the requirements
of community radios. It tries to pro-actively reacted to scenarios, where the upload of some
pre-recorded show has been forgotten, or some live schedule is not taking place.
Engine is able to react to common community radio scenarios, like the upload of some
pre-recorded show has been forgotten, or some live show is actually not taking place.
Usually in such cases the broadcast might end up with some timeslot filled with silence.
To avoid this, Engine implements multiple levels of fallback handling.
To understand how fallbacks are handled by Engine in detail, it's good to know how the Engine's
scheduling is executed. Read more about the scheduling approach in the chapter above.
To avoid this, Engine provides multiple levels of fallback handling.
The available fallbacks are evaluated in following order:
1. **Schedule Fallback**: If the show fallback is not assigned, a configured fallback
playlist for the related timeslot is used.
playlist for the related timeslot is used. This playlist is aired in normal order.
It's important to note, in case that playlists containing anything different than
file entries, are ignored (i.e. live or stream content).
2. **Show Fallback**: If the schedule for some show has no playlist assigned, the
playlist assigned as a *show fallback* is used instead. In the dashboard this can
be done as seen in the screenshot below.
be done as seen in the screenshot below.
![Setting for the Show Fallback in AURA Dashboard](images/dashboard-fallback-setting.png "Show Fallback in Dashboard")
3. **Station Fallback**: If everything goes wrong, meaning all the previous fallback
playlists are missing or are invalid, the *station fallback* will be triggered. This
fallback type is specified by a playlist ID set in the Engine's configuration
(see `scheduling_station_fallback_id` in `engine.ini`)
### Pro-active Fallback Handling (1st Level Fallback)
This type of fallback handling is performed before the actual playout, within the scheduling
window and queuing/pre-rolling stage. Here Engine pro-actively verifies the validity of the playlists.
For example it checks if some playlist ist existing at all, if it holds entries or if the
entries are existing.
This playlist is aired in normal order. It's important to note, in case that playlists
containing anything different than file entries, are ignored (i.e. live or stream content).
Furthermore, it checks for any unexpected error when entries are pre-rolled. For example Engine
tries to buffer some HTTP Audio Stream, but it is not reachable. In this case this will
trigger the scheduling of some fallback playlist.
3. **Station Fallback**: If everything goes wrong, meaning all the previous fallback
playlists are not assigned or are invalid, the *station fallback* will be triggered. This
fallback type is specified by either some **M3U Playlist** or an **Audio Folder**. Both are watched
and automatically updated upon content change. Media is played in an randomized way, meaning
they are shuffled and played until nothing is left for the given playlist/folder. Then it
starts all over again.
Pro-active fallback handling has the advantage that any (even short) periods of silence
during playlist can be avoided. Still, there are cases, where any failure cannot be detected
beforehand. Therefore a 2nd level fallback solution is required (see below).
To configure the behavior of fallbacks, check out the `[fallback]` section in your `engine.ini` configuration.
### Fallback Handling using the Silence Detector (2nd Level Fallback)
### Silence Detector
Engine offers a simple way to detect situations where no sound is on air, the so-called
Silence Detector. It allows detection of absoulte silence, weak signals or even noise.
The aforementioned fallback levels are triggered using a Silence Detector reacting to situations
where no or unwanted sound is on air. The Silence Detector allows detection of absoulte silence,
weak signals or even noise.
To configure the sensitivity of the Silence Detector check-out following properties in
To configure the sensitivity of the Silence Detector adapt following properties in
`engine.ini`:
```
```ini
# max_blank => maximum time of blank from source (defaults to 20., seconds, float)
# min_noise => minimum duration of noise on source to switch back over (defaults to 0, seconds, float)
# threshold => power in dB under which the stream is considered silent (defaults to -40., float)
fallback_max_blank="20."
fallback_max_blank="10."
fallback_min_noise="0."
fallback_threshold="-50."
```
Let's assume you want to broadcast a live show. At the given example the Silence Detector will
react upon 10 seconds of silence, it will evaluate if there is a schedule or show fallback playlist
defined. If yes, it will immediately schedule and play such playlist. If there is no such playlist
the station fallback will kick in by playing any randomized music.
As soon some signal from the live source is sensed again, the audio routing switches back to the live
channel.
> Note, all these fallback source are fully integrated with the Engine's playlog and track-service
feature including indications from which fallback level some audio is broadcasted.
## Monitoring
You have following options to monitor the Engine:
* Send mails on errors and warnings
* Engine Status Information
* Engine Heartbeat
* Logging
- Send mails on errors and warnings
- Engine Status Information
- Engine Heartbeat
- Logging
### Send mails on errors and warnings
......
modules/scheduling/scheduler.py
+ 17
− 1
View file @ 0751cf0e
......@@ -608,7 +608,23 @@ class AuraScheduler(threading.Thread):
def queue_playlist_entries(self, schedule, entries, fade_in, fade_out):
"""
Creates sound-system player commands for all playlist items to be executed at the scheduled time.
Since each scheduled playlist can consist of multiple entry types such as *file*, *live*,
and *stream*, the play-out of the schedule is actually a bit more complex. Before any playlist
entries of the schedule can be turned into sound, they need to be grouped, queued and pre-loaded.
1. First, all entries are aggregated when they hold filesystem entries.
Given you have a playlist with 10 entries, the first 4 are consisting of files, the next two
of a a stream and a live source. The last 4 are files again. These entries are now
aggregated into 4 groups: one for the files, one for the stream, one for the live entry
and another one for files. For each group a timer for executing the next step is created.
2. Now, the playlist entries are going to be "pre-loaded". This means that filesystem
entries are queued and pre-loaded and entries which are based on audio streams are buffered.
This is required to allow a seamless play-out, when its time to do so (in the next stage).
Due to their nature, playlist entries which hold live audio sources are not affected by
this stage at all.
Args:
schedule (Schedule): The schedule this entries belong to
entries ([PlaylistEntry]): The playlist entries to be scheduled for playout
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment