developer-guide.md 8.89 KB
Newer Older
1
# Aura Engine Development Guide
David Trattnig's avatar
David Trattnig committed
2
3
4

This page gives insights on extending Aura Engine internals or through the API.

David Trattnig's avatar
David Trattnig committed
5
<!-- TOC -->
David Trattnig's avatar
David Trattnig committed
6

7
- [Aura Engine Development Guide](#aura-engine-development-guide)
David Trattnig's avatar
David Trattnig committed
8
    - [AURA Componentes](#aura-componentes)
David Trattnig's avatar
David Trattnig committed
9
    - [Engine Components](#engine-components)
10
    - [API](#api)
David Trattnig's avatar
David Trattnig committed
11
    - [Engine Startup Phases](#engine-startup-phases)
12
    - [Scheduling](#scheduling)
13
14
    - [More infos for debugging](#more-infos-for-debugging)
        - [Debugging Liquidsoap](#debugging-liquidsoap)
David Trattnig's avatar
David Trattnig committed
15
        - [Tips on configuring the audo interface](#tips-on-configuring-the-audo-interface)
16
    - [Read more](#read-more)
David Trattnig's avatar
David Trattnig committed
17
18
19

<!-- /TOC -->

David Trattnig's avatar
David Trattnig committed
20
## AURA Componentes
David Trattnig's avatar
David Trattnig committed
21
22
23
24
25
26

AURA Engine as part of the AURA Radio Suite uses an modulear architecture
based on a REST API. All external information is retrieved using JSON data-structures.

To get the basic architectural overview, visit the [Aura Meta](https://gitlab.servus.at/autoradio/meta) repository.

David Trattnig's avatar
David Trattnig committed
27
28
29
30
31
32
Starting development of engine can be quite tedious, as it requires all most all other AURA components to be up and running.

For example:

    - Steering, to get the main incredient of an play-out engine: schedules (or "timeslots" in Steering terms), 
      which hold the actual information on playlists and their entries.
33
    - Dashboard, to have a neat interface, being able to programm the timeslots
David Trattnig's avatar
David Trattnig committed
34
35
36
37
38
39
40
41
42
43
44
45
46
    - Tank, to get the references to audio files and other audio sources. Plus the actual files.

If you need to test and develop against the Engine's API you'll also need to get the `engine-api` project running.

For a start it's recommended to create a general `aura` project folder. In there you start cloning all the sub-projects.
After having all the sub-projects configured, and verified that they are working, take a look at the AURA `meta` project.

There's a convenience script to start all of the three main dependencies (Steering, Dashboard, Tank) all at once:

```bash
    ~/code/aura/meta$ ./run.sh aura local
```

David Trattnig's avatar
David Trattnig committed
47
## Engine Components
48
49


50
*...TBD...*
51
52
53
54


## API

David Trattnig's avatar
David Trattnig committed
55
You can find the AURA API definition here: https://gitlab.servus.at/autoradio/meta/blob/master/api-definition.md
David Trattnig's avatar
David Trattnig committed
56

David Trattnig's avatar
David Trattnig committed
57
OpenAPI definition for Engine API: https://app.swaggerhub.com/apis/AURA-Engine/engine-api/
David Trattnig's avatar
David Trattnig committed
58

David Trattnig's avatar
David Trattnig committed
59
## Engine Startup Phases
David Trattnig's avatar
David Trattnig committed
60

David Trattnig's avatar
David Trattnig committed
61
When you start Engine the following is happening:
David Trattnig's avatar
David Trattnig committed
62

63
1. Python `run.py`: Initializes `src/core/engine.py` (The virtual mixer; class for remote-controlling Liquidsoap), Scheduler
64
2. Python `run.py`: Start Liquidsoap.
65
3. Liquidsoap: When Liquidsoap finished its startup, it creates a socket file as configured in `socket_dir` of `engine.ini`.
66
4. Python `src/core/liquidsoap/client.py`: Connects to that socket file.
67
5. Python `src/schedulung/scheduler.py`: Continously loads timeslots from the API endpoints, stores them in the local database and starts the playout as per the timeslots.
68

69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
## Scheduling

Scheduling is split into multiple phases. Below you see a timeline with one timeslot planned at a certain
point in time and the involved phase before:

```ascii
========================================= [                  Scheduling Window               ] ===========
=======================================================  [        Timeslot Play-out                 ] ====

== (FILESYSTEM A) ========================== [ Preload ] [  Play 4  ] ====================================
== (STREAM A) ========================================== [ Preload ] [ Play 1 ] ==========================
== (LIVE 1) ====================================================== [ Preload ] [ Play 1 ] ================
== (FILESYSTEM B) ========================================================== [ Preload ] [  Play 4  ] ====
```

- **Scheduling Window**: Within the scheduling window any commands for controlling
    the mixer of the soundsystem are prepared and queued.

    Only until the start of the window, timeslots can be updated or removed via external API Endpoints
    (e.g. using Steering or Dashboard). Until here any changes on the timeslot itself will be reflected
    in the actual play-out. This only affects the start and end time of the "timeslot" itself.
    It does not involve related playlists and their entries. Those can still be modified after the
    scheduling window has started.

    The start and the end of the window is defined by the start of the timeslot minus
    a configured amount of seconds (see `scheduling_window_start` and `scheduling_window_end`
    in `engine.ini`). The actual start of the window is calcuated by (timeslot start - window start)
    and the end by (timeslot end - window end)

    During the scheduling window, the external API Endpoints are pulled continiously, to
    check for updated timeslots and related playlists. Also, any changes to playlists and
    its entries are respected within that window (see `fetching_frequency` in `engine.ini`).

    > Important: It's vital that the the scheduling window is wider than the fetching frequency.
    Otherwise one fetch might never hit a scheduling window, hence not being able to schedule stuff.

    > Note: If you delete any existing timeslot in Dashboard/Steering this is only reflected in Engine until the start
    of the scheduling window. The scheduling window is defined by the start of the timeslot minus a configured offset
    in seconds. This is limitation is required to avoid corrupted playout in case audio content has been
    preloaded or started playing already.

- **Queuing and Pre-Loading**: Before any playlist entries of the timeslot can be turned into
    sound, they need to be queued and pre-loaded. Ideally the pre-loading happens somewhat before
    the scheduled play-out time to avoid any delays in timing. Set the maximum time reserved for
    pre-loading in your configuration (compare `preload_offset`in `engine.ini`).

    If there is not enough time to reserve the given amount of time for preloading (i.e. some entry
    should have started in the past already) the offset is ignored and the entry is played as soon as possible.

    > Important: To ensure proper timings, the offset should not exceed the time between the start of
    the scheduling-window and the start of the actual timeslot playout. Practically, of course there
    are scenario where playout start later than planned e.g. during startup of the engine during a timeslot
    or due to some severe connectivity issues to some external stream.

- **Play-out**: Finally the actual play-out is happening. The faders of the virtual mixers are pushed
    all the way up, as soon it's "time to play" for one of the pre-loaded entries.
    Transitions between playlist entries with different types of sources (file, stream and analog
    inputs) are performed automatically. At the end of each timeslot the channel is faded-out,
    no matter if the total length of the playlist entries would require a longer timeslot.

    If for some reason the playout is corrupted, stopped or too silent to make any sense, then
    this <u>triggers a fallback using the silence detector</u>.


David Trattnig's avatar
David Trattnig committed
133
## More infos for debugging
134

135
### Debugging Liquidsoap
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159

Connect to Liquidsoap via Telnet

    telnet 127.0.0.1 1234

List available commands

    help

List all available channels

    list

List all input channels connected to the mixer

    mixer.input

Set the volume of mixer `input 0` to `100%`

    mixer.volume 0 100

Push some audio file to the filesystem `queue 0`

    in_filesystem_0.push /path/to/your/file.mp3
160
161


David Trattnig's avatar
David Trattnig committed
162
163
164
165
166
167
168
169
170
171
172
173
### Tips on configuring the audo interface

Configure your audio device in the `[soundcard]` section of `engine.ini`.

You can configure up to **five** line IN and OUT stereo channels. Your hardware should
support that. When you use JACK, you will see the additional elements popping up when
viewing your connections (with e.g. Patchage).

**Pulse Audio:** When using Ubuntu, Pulse Audio is selected by default. This is convenient,
as you won't have the need to adapt any Engine setting to get audio playing initially.

**ALSA:** When you use ALSA, you will have to play around with ALSA settings. In the folder
174
`./src/liquidsoap` is a scipt called alsa_settings_tester.liq. You can start it
David Trattnig's avatar
David Trattnig committed
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
with 'liquidsoap -v --debug alsa_settings_tester.liq'. Changing and playing with
settings may help you to find correct ALSA settings.

**Jack Audio**: Install the JACK daemon and GUI:

```bash
    sudo apt-get install jackd qjackctl 
```

Please ensure to enable "*realtime process priority*" when installing JACK to keep latency low.
Now, you are able to configure your hardware settings using following command:

```bash
    qjackctl
```

Next you need to install the JACK plugin for Liquidsoap:

```bash
sudo apt install \
    liquidsoap-plugin-jack
```


199
200
201
202
203
204
## Read more

- [Overview](/README.md)
- [Installation for Development](installation-development.md)
- [Installation for Production](installation-production.md)
- [Running with Docker](running-docker.md)
205
- [Setup the Audio Store](docs/setup-audio-store.md)
206
207
- [Developer Guide](developer-guide.md)
- [Engine Features](engine-features.md)
David Trattnig's avatar
David Trattnig committed
208
- [Frequently Asked Questions (FAQ)](docs/frequently-asked-questions.md)