engine-core issueshttps://gitlab.servus.at/aura/engine-core/-/issues2024-03-26T15:37:18+01:00https://gitlab.servus.at/aura/engine-core/-/issues/74crash related to jack2024-03-26T15:37:18+01:00Chris Pastlcrash related to jackI noticed a crash while liquidsoap tries to connect to the jack audio server. For some reason, the audio device could not be accessed although it was connected to the host. No chance to bring up engine-core again using `docker compose up...I noticed a crash while liquidsoap tries to connect to the jack audio server. For some reason, the audio device could not be accessed although it was connected to the host. No chance to bring up engine-core again using `docker compose up -d` - the only workaround was to reboot the system.
I assume that a socket was still open and did not close correctly after a previous crash.
```
engine-core | 2024-03-21T21:08:45.509414430Z 2024/03/21 22:08:45 [fallback_folder:4] Queued 1 requests
engine-core | 2024-03-21T21:08:45.637452463Z ERR: jack_wrapper.c::JACK_OpenDevice(804) jack server not running?
engine-core | 2024-03-21T21:08:45.637469226Z ERR: jack_wrapper.c::JACK_shutdown(750) unable to reconnect with jack
engine-core | 2024-03-21T21:08:45.637472405Z ERR: jack_wrapper.c::JACK_OpenDevice(804) jack server not running?
engine-core | 2024-03-21T21:08:45.637474943Z ERR: jack_wrapper.c::JACK_shutdown(750) unable to reconnect with jack
engine-core | 2024-03-21T21:08:45.637477449Z ERR: jack_wrapper.c::JACK_Read(1296) Device not connected to jack!
```https://gitlab.servus.at/aura/engine-core/-/issues/73termination after closing ssh connection2024-03-26T16:13:03+01:00Chris Pastltermination after closing ssh connectionI logged in via ssh to the aura-playout host to start the docker containers.
Usually, all containers stay running after logging out again.
In some cases, engine-core terminated immediately after closing the ssh connection.
There's no s...I logged in via ssh to the aura-playout host to start the docker containers.
Usually, all containers stay running after logging out again.
In some cases, engine-core terminated immediately after closing the ssh connection.
There's no special termination message, but the logs show no further activity after calling `exit`:
`history`
```
6013 2024-03-21 21:45 exit
```
`cat logs/engine-core.log | grep "2024/03/21 21:4"`
```
2024/03/21 21:45:06 [decoder.video.metadata:4] Unsupported MIME type for "/var/audio/fallback/Sine432.mp3": audio/mpeg!
2024/03/21 21:45:06 [request:5] Resolved to [[/var/audio/fallback/Sine432.mp3]].
2024/03/21 21:45:06 [request:5] Resolving request [[/var/audio/fallback/Sine432.mp3]].
2024/03/21 21:45:06 [request:5] Resolved to [[/var/audio/fallback/Sine432.mp3]].
2024/03/21 21:45:06 [fallback_folder:4] Queued 1 requests
2024/03/21 21:45:06 [lang:3] Successfully posted playlog to Engine API.
2024/03/21 21:45:06 [fallback_folder:5] Got metadata at position 0: calling handlers...
2024/03/21 21:45:06 [mksafe:5] Got metadata at position 0: calling handlers...
2024/03/21 21:45:06 [fallback-source:5] Got metadata at position 0: calling handlers...
➜ aura git:(1.0.0-alpha3) ✗
```
Current workaround: execute the start command inside a `screen` session and keep this session open. Then engine-core stays running after closing the ssh connection.https://gitlab.servus.at/aura/engine-core/-/issues/71Refactor creation of stream endpoints2024-03-13T13:24:02+01:00Ole Binderole@freirad.atRefactor creation of stream endpointsAt the moment the code for the stream endpoints (`s0,s1,s2,s3,s4`) is just copy-pasted code. In addition only the endpoint `s0` has environment variables mapped to its settings.At the moment the code for the stream endpoints (`s0,s1,s2,s3,s4`) is just copy-pasted code. In addition only the endpoint `s0` has environment variables mapped to its settings.Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/70Upgrade to Liquidsoap 2.3+2024-03-08T13:59:34+01:00David TrattnigUpgrade to Liquidsoap 2.3+Not yet released, but compare: https://github.com/savonet/liquidsoap/blob/main/CHANGES.mdNot yet released, but compare: https://github.com/savonet/liquidsoap/blob/main/CHANGES.md1.0-alpha7Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/68Simplify way to choose device channels when using AutoConnect2024-02-22T15:45:01+01:00David TrattnigSimplify way to choose device channels when using AutoConnectFinding the correct device took me a couple of tries, since the listed alias, was not hinting the actual product name of the audio device. Here the device is named `Analog Stereo 1/2:playback_FR`, while it is called `alsa_output.usb-Nati...Finding the correct device took me a couple of tries, since the listed alias, was not hinting the actual product name of the audio device. Here the device is named `Analog Stereo 1/2:playback_FR`, while it is called `alsa_output.usb-Native_Instruments_Komplete_Audio_6_76C51E0D-00.analog-stereo-out-ab` when using `pw-link` or `qpwgraph`.
```json
{ ["object.path"] = alsa:pcm:1:hw:1,0,0:playback:playback_1,
["port.physical"] = true,
["object.serial"] = 2364,
["port.name"] = playback_FR,
["object.id"] = 77,
["port.terminal"] = true,
["port.direction"] = in,
["format.dsp"] = 32 bit float mono audio,
["port.id"] = 1,
["port.alias"] = Analog Stereo 1/2:playback_FR,
["audio.channel"] = FR,
["node.id"] = 32,
}
```
Also extracting the correct name and channel ID is not too intuitive. This is what actually worked:
```ini
# Audio Device Settings
AURA_ENGINE_OUTPUT_DEVICE=Analog Stereo 1/2:playback
AURA_ENGINE_OUTPUT_CHANNEL_LEFT=FL
AURA_ENGINE_OUTPUT_CHANNEL_RIGHT=FR
```1.0-alpha5Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/67Warning when running `/etc/wireplumber/scripts/ls-ports.lua`2024-03-07T16:16:53+01:00David TrattnigWarning when running `/etc/wireplumber/scripts/ls-ports.lua`When running the command `docker compose run engine-core wpexec /etc/wireplumber/scripts/ls-ports.lua` I get a warning:
```shell
WARN[0000] a network with name auranet exists but was not created for project "aura-playout".
Set `external...When running the command `docker compose run engine-core wpexec /etc/wireplumber/scripts/ls-ports.lua` I get a warning:
```shell
WARN[0000] a network with name auranet exists but was not created for project "aura-playout".
Set `external: true` to use an existing network
```1.0-alpha7Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/66Better and consistent naming for the channels, like `aura_engine_line_in_0` a...2024-03-18T12:13:14+01:00David TrattnigBetter and consistent naming for the channels, like `aura_engine_line_in_0` and `aura_engine_line_out_0`The PipeWire channels currently used for AURA Playout are named:
- `in_line_0`
- `lineout_0`
This can be a little bit confusing, for example when trying to find the correct channels with `qpwgraph`, since there is no indication what b...The PipeWire channels currently used for AURA Playout are named:
- `in_line_0`
- `lineout_0`
This can be a little bit confusing, for example when trying to find the correct channels with `qpwgraph`, since there is no indication what belongs to AURA:
![Screenshot_from_2024-02-22_11-15-03](/uploads/00eaee3c921fd0e2f846946dca82898f/Screenshot_from_2024-02-22_11-15-03.png){width=45%}
For consistency and that they are easier to identify, I suggest this naming scheme:
- `aura_engine_line_in_0`
- `aura_engine_line_out_0`
Maybe there are also other metadata settings possible, to give more details when calling the following?
```bash
docker compose run engine-core wpexec /etc/wireplumber/scripts/ls-ports.lua
```1.0-alpha4 — Raving Raccoon 🤪🦝Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/63log: 'Unable to decode "/var/audio/fallback/.gitignore'2024-03-21T18:11:54+01:00Chris Pastllog: 'Unable to decode "/var/audio/fallback/.gitignore'engine-core seems to attempt loading the `.gitignore` file, which results in an endless loop producing these messages:
```bash
engine-core | 2024/01/18 00:57:06 [decoder:3] Available decoders cannot decode "/var/a...engine-core seems to attempt loading the `.gitignore` file, which results in an endless loop producing these messages:
```bash
engine-core | 2024/01/18 00:57:06 [decoder:3] Available decoders cannot decode "/var/audio/fallback/.gitignore" as {audio=pcm(stereo),video=none,midi=none}
engine-core | 2024/01/18 00:57:06 [decoder:3] Unable to decode "/var/audio/fallback/.gitignore" using image decoder(s)!
```
Removing `.gitignore` fixed the error.1.0-alpha7Chris PastlChris Pastlhttps://gitlab.servus.at/aura/engine-core/-/issues/60Refactor creation of line in sources2024-03-18T12:41:21+01:00Ole Binderole@freirad.atRefactor creation of line in sourcesAt the moment the number of line in sources is hard coded. The creation of these sources is also hard coded in liquidsoap. I propose a list based approach when we switch to yaml based configuration files. A line in could then look someth...At the moment the number of line in sources is hard coded. The creation of these sources is also hard coded in liquidsoap. I propose a list based approach when we switch to yaml based configuration files. A line in could then look something like this:
```yaml
input_devices:
- "default"
- "default"
```
## Proposal
<mark>To be discussed and defined more clearly; affects all kinds of channels</mark>
## Dependencies
- https://gitlab.servus.at/aura/engine-core/-/issues/41+1.0-alpha7Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/53Support for Windows/OSX Audio (PortAudio)2024-02-07T19:05:30+01:00David TrattnigSupport for Windows/OSX Audio (PortAudio)Liquidsoap provides PortAudio capabilities. Instead of `input.alsa`, Engine Core should also work with `input.portaudio`.
At the moment, we are primarily supporting Debian Linux derivatives. But there are already users testing AURA with...Liquidsoap provides PortAudio capabilities. Instead of `input.alsa`, Engine Core should also work with `input.portaudio`.
At the moment, we are primarily supporting Debian Linux derivatives. But there are already users testing AURA with Docker on Windows or Macs. So as a lower priority feature, we can also support this user-base.
Maybe we should state, that we cannot provide full support on these platforms.
Note: While PortAudio enables audio support for additional operating systems, it also provides support for Linux Audio. There is a chance, that it improves latency and provides ease of use in contrast to the present ALSA configuration. See ticket #50+ for more details.
# Sub Tasks
- [ ] Integrate `input.portaudio` and provide required config options (`ini`, `docker.env` in engine-core and `env` in the aura repo)
- [ ] Do tests with different audio interfaces on Windows and Mac
- [ ] Test how PortAudio behaves on Linux/Debian
- [ ] Extend all affected documentation1.1Chris PastlChris Pastlhttps://gitlab.servus.at/aura/engine-core/-/issues/47[EPIC] Add Liquidsoap server functions to perform fading (fade in, fade out, ...2023-03-28T17:28:25+02:00David Trattnig[EPIC] Add Liquidsoap server functions to perform fading (fade in, fade out, crossfade of channels)Currently fading is implemented by remote controlling the `mixer.volume` using Python.
Using native Liquidsoap features to control fading between channels is more elegant. This probably improves performance and eventual side-effects, as...Currently fading is implemented by remote controlling the `mixer.volume` using Python.
Using native Liquidsoap features to control fading between channels is more elegant. This probably improves performance and eventual side-effects, as the Unix socket is less busy during fades.1.1https://gitlab.servus.at/aura/engine-core/-/issues/46Telnet authentication2024-03-26T12:11:49+01:00Ole Binderole@freirad.atTelnet authenticationAt the moment the telnet server is reachable without authentication for everyone in the `AURA_ENGINE_TELNET_HOST` range. Everybody with access can connect to the telnet server and therefore mess with the playout.
```bash
telnet aura.lo...At the moment the telnet server is reachable without authentication for everyone in the `AURA_ENGINE_TELNET_HOST` range. Everybody with access can connect to the telnet server and therefore mess with the playout.
```bash
telnet aura.local 1234
```
Our sample configuration for `AURA_ENGINE_TELNET_HOST` is `0.0.0.0`. Should this be `127.0.0.1` instead? Then only users on the host machine of `engine-core` can access the telnet server.
It would be great to have some kind of authentication for the telnet client. Liquidsoap itself does not provide any authentication method but has a workaround with a socket and a custom shell.
> Rather than doing our own home-made secure acces, we believe that our users should be able to define their own secure access to the command server, taking advantage of a mainstream authentication mechanism, for instance HTTP or SSH login
More information about that [here](https://www.liquidsoap.info/doc-1.4.3/server.html#securing-the-server).
We currently provide the option to disable the telnet server completely. Maybe it is sufficient to disable it and use a socket, just like the python client does. @david do you think a light custom shell is doable? And do you think this authentication is necessary at all?
I do like the option to access liquidsoap through a direct control shell / telnet server. This way an admin can quickly fix wrongly scheduled timeslots or broken audio files.1.0-alpha7Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/41Config consolidation: Use YAML instead of INI file2024-03-18T12:30:14+01:00David TrattnigConfig consolidation: Use YAML instead of INI fileTo be done at a later milestone, because there is no straight-forward way to parse YAML in Liquidsoap.
This requires engine-core#36+, which provides native YAML support.To be done at a later milestone, because there is no straight-forward way to parse YAML in Liquidsoap.
This requires engine-core#36+, which provides native YAML support.1.0-alpha5Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/38Use Liquidsoap DEB as default for native installation2024-03-21T18:32:15+01:00David TrattnigUse Liquidsoap DEB as default for native installationI've clarified with Romain, maintainer of Liquidsoap the pros and cons with OPAM vs DEP installation.
For end users and development flow (updates), it's much more straightforward using the DEB build provided on GitHub.
Asking about the...I've clarified with Romain, maintainer of Liquidsoap the pros and cons with OPAM vs DEP installation.
For end users and development flow (updates), it's much more straightforward using the DEB build provided on GitHub.
Asking about the disadvantages I got this answer:
> The real potential issue with the build is that they enable virtually all available features. This increases the risk for potential issues such as segfault, memleak and etc.
I'd say, if your priority is limiting user friction then by any means go for it. If you can, however, identify a stable sub-set of features that you will need then either install via opam or perhaps build your own .deb.
It wouldn't be too, too crazy to enable specific GH workflows with limited subsets of optional deps built in either, you could fork our main repo and do that. I'll see if I can find a way to parametrize ours to make it easier.
Since the Docker build is also enabling all default features, I'd opt for UX/DX priority at first. As soon we face any issues we might rethink that approach.1.0-alpha6Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/36[EPIC] Upgrade to Liquidsoap 2.2 or later2024-03-13T15:05:19+01:00David Trattnig[EPIC] Upgrade to Liquidsoap 2.2 or laterParent: #50+
---
Before updating, carefully review the [Liquidsoap changelog](https://github.com/savonet/liquidsoap/blob/main/CHANGES.md) for breaking changes or any features and issues which are related to our use. It is also helpful t...Parent: #50+
---
Before updating, carefully review the [Liquidsoap changelog](https://github.com/savonet/liquidsoap/blob/main/CHANGES.md) for breaking changes or any features and issues which are related to our use. It is also helpful to check the Liquidsoap bug-tracker for known issues.
### Breaking changes
#### 2.2
- BREAKING: all timeout settings and parameters are now float values and in seconds (#2809)
- BREAKING: in output.{shoutcast,icecast}:
- Old icy_metadata renamed to send_icy_metadata and changed to a nullable bool. null means guess.
- New icy_metadata now returns a list of metadata to send with ICY updates.
- Added icy_song argument to generate default "song" metadata for ICY updates. Defaults to <artist> - <title> when available, otherwise artist or title if available, otherwise null, meaning don't add the metadata.
- Cleanup, removed parameters that were irrelevant to each operator, i.e. icy_id in output.icecast and etc.
- Make mount mandatory and name nullable. Use mount as name when name is null.
- Removed support for %define variables, superseded by support for actual variables in encoders.
- Removed confusing let json.stringify in favor of json.stringify()
- Renamed {get,set}env into environment.{get,set}
- Deprecated string_of in favor of string (#2700).
- Deprecated string_of_float in favor of string.float (#2700).
- Added id argument to replaygain operator (#2537). Compare [ID fix](https://github.com/savonet/liquidsoap/discussions/2537) for the new `replaygain` operator
### Fixed & Improvements
#### 2.2
- `The randomization function list.shuffle used in playlist was incorrect and could lead to incorrectly randomized playlists (#2507, #2500).`
#### 2.1.4
- Added buffer_length method to buffer operator.
- Always display error backtrace when a fatal exception is raised in the streaming loop.
- Fixed memory leak in http requests (#2935)
- Space trim in interactive variables set on telnet (#2785)
- Fixed internal streaming logic in max_duration and crossfade.
- Make sure that there's at most one metadata at any given frame position (#2786)
- Fixed input polling stop (#2769)
- Fixed parsed error report in %include directives (#2775)
#### 2.1.3
- Add support for song metadata (mapped to title) and url (mapped to metadata_url) in input.harbor (#2676)
- Fixed request metadata escaping (#2732)
- Fixed http response body on redirect (#2758)
#### 2.1.2
- Fixed order of playlist.next returned requests.
- Fixed infinite loop when reloading a failed playlist (#2576)
- Prevent initial start for autostart and fallible sources.
### Features
#### 2.2 and before
- [YAML support](https://github.com/savonet/liquidsoap/issues/2853), required for https://gitlab.servus.at/aura/engine-core/-/issues/41+
- Added log colors! :-)
- Reimplemented harbor http handler API to be more flexible. Added a new node/express-like registration and middleware API (#2599).
- Added device_id and latency options to input.portaudio and output.portaudio to be able to choose the requested device. Use liquidsoap --list-portaudio-devices to see the list of devices (#2733)
- Added disconnect method to input.harbor, making it possible to disconnect a source client programmatically, including when a new client is trying to connect.
### To check if it is relevant
#### 2.2 and before
- Added support for less memory hungry audio formats, namely pcm_s16 and pcm_f32 (#3008)
- Added "metadata_url" to the default list of exported metadata (#2946)
- Changed default character encoding in output.harbor, output.icecast output.shoutcast to UTF-8 (#2704)
- Cancel pending append when skipping current track on append source.
- Switched default persistence for cross and fade-related overrides to follow documented behavior. By default, "liq_fade_out", "liq_fade_skip", "liq_fade_in", "liq_cross_duration" and "liq_fade_type" now all reset on new tracks. Use persist_overrides to revert to previous behavior (persist_override for cross/crossfade) (#2488).
- Changed self_sync in input.ffmpeg to be a boolean getter, changed self_sync in input.http to be a nullable boolean getter. Set self_sync to true in input.http when an icecast or shoutcast server can be detected.
- Add buffer_length method to input.external.rawaudio and input.external.wav (#2612).
- Fixed metadata parsing in server.insert_metadata (#2619)
- Fixed extract_replaygain path (#2624, @parnikkapore)1.0-alpha4 — Raving Raccoon 🤪🦝Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/27Use variables in encoders to simplify outgoing streams2023-07-17T12:08:16+02:00David TrattnigUse variables in encoders to simplify outgoing streamsLiquidsoap 2.1 [now supports variables in encoders](https://github.com/savonet/liquidsoap/issues/1569). This means the implementation of [outgoing streams](https://gitlab.servus.at/aura/engine-core/-/tree/master/src/outgoing_streams) can...Liquidsoap 2.1 [now supports variables in encoders](https://github.com/savonet/liquidsoap/issues/1569). This means the implementation of [outgoing streams](https://gitlab.servus.at/aura/engine-core/-/tree/master/src/outgoing_streams) can be drastically simplified.1.1https://gitlab.servus.at/aura/engine-core/-/issues/26Metadata Handling: Some streams continuously update metadata remotely, apply ...2023-07-17T12:08:10+02:00David TrattnigMetadata Handling: Some streams continuously update metadata remotely, apply advanced merge-strategyThis applies e.g. to the [Ö1 Stream](https://orf-live.ors-shoutcast.at/oe1-q2a) is updating the title approximately every 20 seconds.
This can be seen as a feature, as audio players are constatly updating the title with new information....This applies e.g. to the [Ö1 Stream](https://orf-live.ors-shoutcast.at/oe1-q2a) is updating the title approximately every 20 seconds.
This can be seen as a feature, as audio players are constatly updating the title with new information.
It also can be seen as a huge source of noise, as new playlogs are generated constantly.1.1https://gitlab.servus.at/aura/engine-core/-/issues/25Feature: Merge title of remote streams into playlog (`icy-title`)2024-03-21T18:32:46+01:00David TrattnigFeature: Merge title of remote streams into playlog (`icy-title`)1.0-alpha5Ole Binderole@freirad.atOle Binderole@freirad.athttps://gitlab.servus.at/aura/engine-core/-/issues/24Disabled fallback source triggers additional track updates (playlog POST)2023-07-17T12:07:48+02:00David TrattnigDisabled fallback source triggers additional track updates (playlog POST)Possibly this happens due to `source.skip` implementation.
To reproduce:
1. Play a fallback source: Metadata updates trigger correctly from this source
2. Switch to a queue source: Metadata updates trigger correctly from this source
3....Possibly this happens due to `source.skip` implementation.
To reproduce:
1. Play a fallback source: Metadata updates trigger correctly from this source
2. Switch to a queue source: Metadata updates trigger correctly from this source
3. Switch back to the fallback source: Two metadata updates get triggered1.0-alpha5https://gitlab.servus.at/aura/engine-core/-/issues/23Fallback channel loudness to high in contrast to other channels2024-02-27T09:43:02+01:00David TrattnigFallback channel loudness to high in contrast to other channelsWhen scheduling audio files which are normalized via Tank, the are way to silent in contrast to the (not normalized) fallback music.
Find out if this is because of normalization or if loudness of fallback audio is too high in general.When scheduling audio files which are normalized via Tank, the are way to silent in contrast to the (not normalized) fallback music.
Find out if this is because of normalization or if loudness of fallback audio is too high in general.1.0-alpha5Ole Binderole@freirad.atOle Binderole@freirad.at