Commit 81c72e83 authored by Gottfried Gaisbauer's avatar Gottfried Gaisbauer
Browse files

reactivated fallback and callback from lqs, fixed multiple lineins on engine...

reactivated fallback and callback from lqs, fixed multiple lineins on engine level, having the problem that fallback does not start
parent 109388a0
......@@ -26,11 +26,53 @@ sudo apt install \
git \
python3 python3-pip \
redis-server \
liquidsoap liquidsoap-plugin-alsa liquidsoap-plugin-flac liquidsoap-plugin-icecast liquidsoap-plugin-pulseaudio \
liquidsoap liquidsoap-plugin-icecast \
mariadb-server libmariadbclient-dev \
quelcom
```
##### Liquidsoap Plugins
###### Soundcard
How liquidsoap is using your soundcard is depending on what you are going to use:
with ALSA:
```bash
sudo apt install \
liquidsoap-plugin-alsa liquidsoap-plugin-pulseaudio
```
With pulseaudio:
```bash
sudo apt install \
liquidsoap-plugin-pulseaudio
```
with jack:
```bash
sudo apt install \
liquidsoap-plugin-jack
```
###### File Formats
Depending on what stream you are going to send, and what recordings you are going to use:
```bash
sudo apt install \
liquidsoap-plugin-aac # for aac support
liquidsoap-plugin-flac # for flac support
liquidsoap-plugin-lame liquidsoap-plugin-mad # for mp3 support
liquidsoap-plugin-opus # for opus support
liquidsoap-plugin-vorbis # for ogg support
```
###### Simple
```bash
sudo apt install \
liquidsoap-plugin-all
```
#### Python Packages
......@@ -61,7 +103,7 @@ GRANT ALL PRIVILEGES ON aura_engine.* TO 'aura'@'localhost';
##### phpmyadmin / adminer way
Log into your phpmyadmin or adminer with correct privileges and create a database and a user for the aura engine.
Log into your phpmyadmin or adminer with correct privileges, create a database and a user for the aura engine.
#### Files and Folders
......@@ -86,33 +128,62 @@ The commandline tool for interacting with the server. Also provides the communic
#### Liquidsoap
The heart of AURA Engine. It uses the built in mixer, to switch between different sources.
The heart of AURA Engine. It uses the built in mixer, to switch between different sources. It records everything and streams everything depending on your settings in aura.ini.
#### Find Help
LiquidSoap Reference: http://savonet.sourceforge.net/doc-svn/reference.html
Python3.5 Reference: https://docs.python.org/3.5/
##### Liquidsoap
Reference: \
http://savonet.sourceforge.net/doc-svn/reference.html
##### Python
Reference: \
https://docs.python.org/3.5/
#### Interfaces
##### From Aura Engine
_Soundserverstate_ \
Returns true and false values of the internal In- and Outputs \
/api/v1/soundserver_state
_Trackservice_ \
/api/v1/trackservice/<selected_date> \
/api/v1/trackservice/
##### To Aura Engine
Interfaces are needed from pv/steering to engine and from tank to engine. More informations you can find here: https://gitlab.servus.at/autoradio/meta/blob/master/api-definition.md
### Hardware
#### Soundcard
AURA Engine ist tested with a ASUS Xonar DGX and a Roland Duo-Capture Ex. Both work well with jack and pulseaudio. For good experience with ALSA, you may need better hardware.
AURA Engine is tested with a ASUS Xonar DGX, a Roland Duo-Capture Ex and also on an Onboard Soundcard (HDA Intel ALC262). Both work well with jack and pulseaudio. For a good experience with ALSA, you may need better hardware.
#### Hard/Soft
When you use ALSA, you will have to play around with ALSA settings. In the folder ./modules/liquidsoap is a scipt called alsa_settings_tester.liq. You can start it with 'liquidsoap -v --debug alsa_settings_tester.liq'. Changing and playing with settings can help you to find correct ALSA settings.
When you use ALSA, you will have to play around with ALSA settings. In the folder ./modules/liquidsoap is a scipt called alsa_settings_tester.liq. You can start it with 'liquidsoap -v --debug alsa_settings_tester.liq'. Changing and playing with settings may help you to find correct ALSA settings.
#### Line In
You can configure up to **five** line ins. Your hardware should support that. When you use JACK, you will see the additional elements popping up when viewing your connections (with e.g. Patchage).
#### Recordings
You can configure up to **five** recorders. You find the settings in the main config file engine.ini. You can choose between different output formats.
### Recordings
#### Streams
You can configure up to five recorders. You find the settings in the main config file engine.ini. You can choose between different output formats.
You can configure up to **five** streams. You find the settings in the engine.ini. You can choose between different streaming formats.
### Streams
### Troubleshooting
You can configure up to five streams. You find the settings in the engine.ini. You can choose between different streaming formats.
**If you cannot find correct ALSA settings** \
Well, this is - at least for me - a hard one. I could not manage to find correct ALSA settings for the above mentioned soundcards. The best experience i had with the ASUS Xonar DGX, but still very problematic (especially the first couple of minutes after starting liquidsoap). Since i enabled JACK support i only use that. It is also a bit of trial and error, but works pretty much out of the box.
If you experience 'hangs' on the stream
* reduce the quality or
**If you experience 'hangs' or other artefacts on the output signal**
* reduce the quality (especially, when hangs are on the stream) or
* install the realtime kernel with
```bash
apt install linux-image-rt-amd64
......
......@@ -75,7 +75,7 @@ class Aura(AuraLogger):
self.messenger.liquidsoapcommunicator = self.liquidsoapcommunicator
self.diskspace_watcher = DiskSpaceWatcher(self.config, self.logger, self.liquidsoapcommunicator)
self.diskspace_watcher.run()
self.diskspace_watcher.start()
def receive_signal(signum, stack):
print("received signal")
......@@ -83,12 +83,14 @@ class Aura(AuraLogger):
signal.signal(signal.SIGUSR1, receive_signal)
# wait for redis message
# and finally wait for redis message
self.join_comm()
# start the web service
self.start_web_service()
def join_comm(self):
# start listener thread
self.messenger.start()
......
......@@ -23,13 +23,15 @@ mail_user=""
mail_pass=""
# if you want to send multiple adminmails, make them space separated
admin_mail="gogo@servus.at gottfried@servus.at"
#
# with from mailadress should be used
from_mail="monitor@aura.py"
# The beginning of the subject. With that you can easily apply filter rules
mailsubject_prefix="[AURA]"
[dataurls]
# the url of pv/steering
calendarurl="http://localhost:8000/api/v1/playout"
# the url of tank
importerurl="http://localhost:8008/api/v1/groups/_public/playlists/"
# how often should the calendar be fetched in seconds (This determines the time of the last change before a specific show)
fetching_frequency=3600
......@@ -38,7 +40,7 @@ fetching_frequency=3600
# LiquidSoap Settings #
#######################
# all these settings here require a restart of the liquidsoap server
# all these settings from here to the bottom require a restart of the liquidsoap server
[liquidsoap]
# the user and group under which this software will run
......@@ -78,7 +80,7 @@ soundsystem="jack"
# with pulse and jack => an non empty value means it is used
# devices with empty string are ignored and not used
input_device_0="y"
input_device_1=""
input_device_1="y"
input_device_2=""
input_device_3=""
input_device_4=""
......
......@@ -72,6 +72,7 @@ class Guru(AuraConfig):
print("...result: ")
if p.stringreply != "":
print(p.stringreply)
if p.stringreply[len(p.stringreply)-1] == "\n":
print(p.stringreply[0:len(p.stringreply) - 1])
else:
......
......@@ -161,6 +161,7 @@ class ScheduleEntry(DB.Model, AuraDatabaseModel):
@orm.reconstructor
def reconstructor(self):
self.calc_unix_times()
self.define_clean_source()
self.set_entry_type()
def define_clean_source(self):
......@@ -193,7 +194,16 @@ class ScheduleEntry(DB.Model, AuraDatabaseModel):
if self.source.startswith("pool") or self.source.startswith("playlist") or self.source.startswith("file"):
self.type = ScheduleEntryType.FILESYSTEM
if self.source.startswith("live") or self.source.startswith("linein"):
self.type = ScheduleEntryType.LIVE
if self.cleansource == "0":
self.type = ScheduleEntryType.LIVE_0
elif self.cleansource == "1":
self.type = ScheduleEntryType.LIVE_1
elif self.cleansource == "2":
self.type = ScheduleEntryType.LIVE_2
elif self.cleansource == "3":
self.type = ScheduleEntryType.LIVE_3
elif self.cleansource == "4":
self.type = ScheduleEntryType.LIVE_4
# ------------------------------------------------------------------------------------------ #
@staticmethod
......@@ -219,8 +229,6 @@ class ScheduleEntry(DB.Model, AuraDatabaseModel):
entry.programme_index = cnt
cnt = cnt + 1
return all_entries
# ------------------------------------------------------------------------------------------ #
......@@ -245,8 +253,8 @@ class ScheduleEntry(DB.Model, AuraDatabaseModel):
# ------------------------------------------------------------------------------------------ #
@staticmethod
def upcoming(datefrom=datetime.datetime.now()):
upcomingtracks = DB.session.query(ScheduleEntry).filter(ScheduleEntry.start > datefrom).all()
def select_upcoming(datefrom=datetime.datetime.now()):
upcomingtracks = DB.session.query(ScheduleEntry).filter(ScheduleEntry.entry_start > datefrom).order_by(ScheduleEntry.entry_start).all()
return upcomingtracks
# ------------------------------------------------------------------------------------------ #
......@@ -263,7 +271,7 @@ class ScheduleEntry(DB.Model, AuraDatabaseModel):
if self.type == self.type.FILESYSTEM:
return "fs"
if self.type == self.type.LIVE:
if self.type == self.type.LIVE_0 or self.type == self.type.LIVE_1 or self.type == self.type.LIVE_2 or self.type == self.type.LIVE_3 or self.type == self.type.LIVE_4:
return "aura_linein_"+self.cleansource # .cleanprotocol[8]
if self.type == self.type.STREAM:
......
......@@ -60,6 +60,11 @@ class RedisChannel(Enum):
class ScheduleEntryType(Enum):
# enumeration with names of liquidsoap inputs
FILESYSTEM = "fs"
STREAM = "http"
LIVE = "live"
\ No newline at end of file
LIVE_0 = "aura_linein_0"
LIVE_1 = "aura_linein_1"
LIVE_2 = "aura_linein_2"
LIVE_3 = "aura_linein_3"
LIVE_4 = "aura_linein_4"
\ No newline at end of file
......@@ -171,6 +171,8 @@ class LiquidSoapCommunicator(ExceptionLogger):
else:
self.recorder_start_one(num)
self.disable_transaction()
# ------------------------------------------------------------------------------------------ #
def recorder_start_all(self):
self.enable_transaction()
......@@ -188,62 +190,75 @@ class LiquidSoapCommunicator(ExceptionLogger):
self.__send_lqc_command__(self.client, "recorder", str(num), "start")
# ------------------------------------------------------------------------------------------ #
def activate(self, entry):
active_type = self.scheduler.get_active_entry().type
def activate(self, new_entry):
# grab the actual active entry
old_entry = self.scheduler.get_active_entry()
# determine its type
old_type = old_entry.type
try:
# enable transaction
self.enable_transaction()
if active_type == entry.type:
if old_type == new_entry.type:
# push something to active channel
self.activate_same_channel(entry)
self.activate_same_channel(new_entry)
else:
# switch to another channel
self.activate_different_channel(entry, active_type)
self.activate_different_channel(new_entry, old_type)
# disable conn
self.disable_transaction()
# insert playlist entry
self.insert_track_service_entry(entry)
self.insert_track_service_entry(new_entry)
except LQConnectionError as e:
# we already caught and handled this error in __send_lqc_command__, but we do not want to execute this function further
# we already caught and handled this error in __send_lqc_command__, but we do not want to execute this function further and pass the exception
pass
# ------------------------------------------------------------------------------------------ #
def activate_same_channel(self, entry, silent=False):
if not silent:
def activate_same_channel(self, entry, activate_different_channel=False):
if not activate_different_channel:
self.logger.info(TerminalColors.PINK.value + entry.type.value + " already active!" + TerminalColors.ENDC.value)
# push to fs or stream
if entry.type == ScheduleEntryType.FILESYSTEM:
self.playlist_push(entry.source)
if entry.type == ScheduleEntryType.STREAM:
self.http_start_stop(True)
self.set_http_url(entry.source)
self.http_start_stop(True)
# nothing to do when we are live => just leave it as is
# set active channel to wanted volume
if not activate_different_channel:
self.channel_volume(entry.type.value, entry.volume)
# ------------------------------------------------------------------------------------------ #
def activate_different_channel(self, entry, active_type):
self.logger.info(TerminalColors.PINK.value + "LiquidSoapCommunicator is activating " + entry.type.value + " & deactivating " + active_type.value + "!" + TerminalColors.ENDC.value)
# reuse of this function, because activate_same_channel and activate_different_channel are doing pretty the same except setting of the volume
self.activate_same_channel(entry, True)
# set others to zero volume
# set other channels to zero volume
others = self.all_inputs_but(entry.getChannel())
for o in others:
self.channel_volume(o, 0)
# set active channel to wanted volume
self.channel_volume(entry.type.value, entry.volume)
# ------------------------------------------------------------------------------------------ #
def insert_track_service_entry(self, schedule_entry):
# create trackservice entry
trackservice_entry = TrackService()
# set foreign keys
trackservice_entry.playlist_id = schedule_entry.playlist_id
trackservice_entry.entry_num = schedule_entry.entry_num
trackservice_entry.source = schedule_entry.source
# store
trackservice_entry.store(add=True, commit=True)
# ------------------------------------------------------------------------------------------ #
......@@ -323,7 +338,7 @@ class LiquidSoapCommunicator(ExceptionLogger):
return message
except (AttributeError, ValueError) as e: #(LQConnectionError, AttributeError):
self.disable_transaction(force=True)
self.logger.critical("Ran into exception when setting volume of channel " + channel + ". Reason: " + str(e))
self.logger.error("Ran into exception when setting volume of channel " + channel + ". Reason: " + str(e))
# ------------------------------------------------------------------------------------------ #
def auraengine_state(self):
......@@ -351,7 +366,10 @@ class LiquidSoapCommunicator(ExceptionLogger):
@param uri: Die Uri
"""
return self.__send_lqc_command__(self.client, "fs", "push", uri)
# self.notifyClient()
# ------------------------------------------------------------------------------------------ #
def playlist_seek(self, seconds_to_seek):
return self.__send_lqc_command__(self.client, "fs", "seek", seconds_to_seek)
# ------------------------------------------------------------------------------------------ #
def version(self):
......
......@@ -24,6 +24,7 @@
import time
import logging
import datetime
import threading
from libraries.enum.auraenumerations import ScheduleEntryType
......@@ -43,7 +44,7 @@ class LiquidSoapInitThread(threading.Thread):
# ------------------------------------------------------------------------------------------ #
def run(self):
try:
# sleep needed, because the socket is created to slow by liquidsoap
# sleep needed, because the socket is created too slow by liquidsoap
time.sleep(1)
self.logger.info("Waited 1s for liquidsoap. Jez soit a si gspian")
......@@ -53,16 +54,16 @@ class LiquidSoapInitThread(threading.Thread):
# reset channels and reload them
channels = self.liquidsoapcommunicator.reload_channels()
# set every volume to 0
# for all available channels
for c in channels:
# set volume to zero
self.liquidsoapcommunicator.channel_volume(c, "0")
# and activate this channel
self.liquidsoapcommunicator.channel_activate(c, True)
# select all channels
# for c in channels:
# self.liquidsoapcommunicator.channel_activate(c, True)
# setting init params
# setting init params like a blank file..
self.liquidsoapcommunicator.playlist_push(self.liquidsoapcommunicator.config.get("install_dir") + "/configuration/blank.flac")
# .. or the radio fro stream (it is overwritten as soon as one http overtake is planned)
self.liquidsoapcommunicator.set_http_url("http://stream.fro.at/fro-128.ogg")
# wait another second. lqs really starts slow..
......@@ -73,9 +74,25 @@ class LiquidSoapInitThread(threading.Thread):
self.logger.info("LiquidSoapInitThread sets activechannel: "+str(self.active_entry))
channel = self.active_entry.type
# have to seek?
if channel == ScheduleEntryType.FILESYSTEM:
# calc how many seconds were missed
now_unix = time.mktime(datetime.datetime.now().timetuple())
seconds_to_seek = now_unix - self.active_entry.entry_start_unix
# and seek these seconds forward
self.liquidsoapcommunicator.playlist_seek(seconds_to_seek)
# finally make something hearable :-)
if channel != "" and channel is not None:
# activate http stream if needed
self.liquidsoapcommunicator.http_start_stop(channel == ScheduleEntryType.STREAM)
# finally set the volume up
self.liquidsoapcommunicator.channel_volume(channel.value, self.active_entry.volume)
else:
self.logger.error("Channel is NULL or empty! Cannot set ")
else:
self.logger.warning("No active entry in the scheduler! Is a programme loaded?")
......
......@@ -69,6 +69,9 @@ class LiquidSoapPlayerClient(LiquidSoapClient):
if command == "push":
return self.fs_push(*args)
if command == "seek":
return self.fs_seek(*args)
return "LiquidSoapPlayerClient does not understand fs." + command + str(args)
# ------------------------------------------------------------------------------------------ #
......@@ -88,6 +91,11 @@ class LiquidSoapPlayerClient(LiquidSoapClient):
self.command('fs', 'push', uri)
return self.message
# ------------------------------------------------------------------------------------------ #
def fs_seek(self, uri):
self.command('fs', 'seek', uri)
return self.message
# ------------------------------------------------------------------------------------------ #
def set_http_url(self, uri):
self.command('http', 'url', uri)
......
......@@ -22,7 +22,6 @@
# along with engine. If not, see <http://www.gnu.org/licenses/>.
#
import os
import smtplib
from email.message import EmailMessage
from libraries.exceptions.auraexceptions import MailingException
......
......@@ -41,7 +41,6 @@ from libraries.enum.auraenumerations import RedisChannel, TerminalColors
# ------------------------------------------------------------------------------------------ #
class ServerRedisAdapter(threading.Thread, RedisMessenger):
debug = False
# logger = None
pubsub = None
config = None
redisdb = None
......@@ -53,12 +52,9 @@ class ServerRedisAdapter(threading.Thread, RedisMessenger):
# ------------------------------------------------------------------------------------------ #
def __init__(self):
#super(ServerRedisAdapter, self).__init__()
threading.Thread.__init__(self)
RedisMessenger.__init__(self)
# self.logger = logging.getLogger("AuraEngine")
# init
threading.Thread.__init__ (self)
self.shutdown_event = Event()
......@@ -95,7 +91,7 @@ class ServerRedisAdapter(threading.Thread, RedisMessenger):
try:
self.work(item)
except RedisConnectionException as rce:
self.logger.error(rce)
self.logger.error(str(rce))
if not self.shutdown_event.is_set():
self.logger.info(TerminalColors.ORANGE.value + "waiting for REDIS message on channel " + self.channel + TerminalColors.ENDC.value)
......
......@@ -292,7 +292,7 @@ class RedisMessenger():
next = self.rstore.db.get('next_'+playlisttype+'_file')
if next is None:
return ""
next = b""
return next.decode('utf-8')
......
......@@ -53,16 +53,17 @@ inputs = ref []
mixer = mix(id="mixer", list.append([input_fs, input_http], !inputs))
# output source with fallbacks
# stripped_stream = strip_blank(max_blank=fallback_max_blank, min_noise=fallback_min_noise, threshold=fallback_threshold, mixer)
ignore(fallback_max_blank)
ignore(fallback_min_noise)
ignore(fallback_threshold)
ignore(timeslot_fallback)
ignore(station_fallback)
ignore(show_fallback)
stripped_stream = strip_blank(max_blank=fallback_max_blank, min_noise=fallback_min_noise, threshold=fallback_threshold, mixer)
# ignore(fallback_max_blank)
# ignore(fallback_min_noise)
# ignore(fallback_threshold)
# ignore(timeslot_fallback)
# ignore(station_fallback)
# ignore(show_fallback)
# enable fallback
output_source = mixer # fallback(id="fallback", track_sensitive=false, [mksafe(stripped_stream), timeslot_fallback, show_fallback, station_fallback])
# output_source = mixer
output_source = fallback(id="fallback", track_sensitive=false, [mksafe(stripped_stream), timeslot_fallback, show_fallback, station_fallback])
##################
# create outputs #
......@@ -84,6 +85,6 @@ output_source = mixer # fallback(id="fallback", track_sensitive=false, [mksafe(s
# start initialization #
########################
# system('#{list.assoc("install_dir", ini)}/guru.py --init-player --quiet')
system('#{list.assoc("install_dir", ini)}/guru.py --init-player --quiet')
......@@ -98,8 +98,7 @@ def fallback_create(~skip=true, name, requestor)
log("Creating channel #{name}")
# Create the request.dynamic source
# Set conservative to true to queue
# several songs in advance
# Set conservative to true to queue several songs in advance
#source = request.dynamic(conservative=true, length=50., id="pool_"^name, requestor, timeout=60.)
source = request.dynamic(length=50., id="pool_"^name, requestor, timeout=60.)
......@@ -109,7 +108,7 @@ def fallback_create(~skip=true, name, requestor)
# Skip blank when asked to
source =
if skip then
skip_blank(source, max_blank=10., threshold=-40.)
skip_blank(max_blank=fallback_max_blank, min_noise=fallback_min_noise, threshold=fallback_threshold, source)
else
source
end
......
......@@ -44,6 +44,7 @@ class DiskSpaceWatcher(threading.Thread):
# ------------------------------------------------------------------------------------------ #
def __init__(self, config, logger, liquidsoapcommunicator):
threading.Thread.__init__(self)
self.liquidsoapcommunicator = liquidsoapcommunicator
self.config = config
self.logger = logger
......@@ -60,19 +61,21 @@ class DiskSpaceWatcher(threading.Thread):
except:
seconds_to_wait = 600
# while True:
while not self.exit_event.is_set():
# calc next time
next_time = datetime.datetime.now() + datetime.timedelta(seconds=seconds_to_wait)
try:
# calc next time
next_time = datetime.datetime.now() + datetime.timedelta(seconds=seconds_to_wait)
# write to logger
self.logger.info("Diskspace watcher every " + str(seconds_to_wait) + "s started. Going to start next time " + str(next_time))
# write to logger
self.logger.info("Diskspace watcher every " + str(seconds_to_wait) + "s started. Going to start next time " + str(next_time))
# check disk space
self.check_disk_space()
# check disk space
self.check_disk_space()
# and wait
self.exit_event.wait(seconds_to_wait)
# and wait
self.exit_event.wait(seconds_to_wait)
except BrokenPipeError as e:
self.logger.critical("Cannot check if recorder is running. It seems LiquidSoap is not running. Reason: " + str(e))