[EPIC] Docker Compose: Modular and simplified configuration
Currently most configuration options of individual services are replicated as environment variables for Docker Compose.
This approach has several drawbacks:
- Optional service configurations are always present: All env vars are always present, even when not all services are used. For example there are always warnings about Icecast vars not set, even though Icecast is not needed in most scenarios.
- No differentiation between app config and container config: App defaults are always set on infrastructure level, even though they are application specific and provide no advantage when passed through Docker Compose.
- Missing meta information: The sheer amount of env vars makes it harder to understand which settings relate to which service. In case an original yaml file, meta information provided by its hierarchy gets lost when using a flat env file.
- Inconsistencies: For some services like Engine all settings got duplicated as env vars, in case of Tank and Engine Core it's still a separate configuration file.
- Missing configuration options: Not all settings are exposed, for example in case of Tank.
- Standard-alone Docker use is hard to not possible: It's a typical approach on e.g. Dockerhub to provide details on a few environment variables, allowing to construct a simple
docker run ...
command. This is currently, not possible in a pragmatic way. Hence individual docker run settings are not documented yet for all images. Also, existing individual run scripts for Docker got broken. This might be still the case for other services. - DX: harder to maintain for developers when app features change and get extended. It's easier to point to a new, optional configuration option in a config, without any need to extend .env and docker-compose files in addition. Same for noting required steps in the release notes on how to update .env files already in use. There are additional problems with potentially re-defining defaults on app-config and container-config level.
Possible solutions
(A) Differentiate between app and infrastructure config (suggested)
Summary: Only provide config settings as ENV variables, which are required for starting the container. Hide app specific config options which are never or rarely changed in relevant config files. Those config file can be bind-mounted into the container.
Mount app configuration files
It is quite common to have app config in a specific file and (bind-) mount it into the container. Some popular examples are:
-
NGINX:
-v /host/path/nginx.conf:/etc/nginx/nginx.conf:ro
-
Prometheus:
-v /path/to/prometheus.yml:/etc/prometheus/prometheus.yml
-
Postgres:
-v "$PWD/my-postgres.conf":/etc/postgresql/postgresql.conf
-
Icecast2:
--volume /path/to/your/icecast.xml:/etc/icecast2/icecast.xml
When there are infrastructure settings required on container level, then those settings can be set using an env var. A good example for this approach is the Tank Config YAML. Here settings like database credentials or URLs are provided by Docker envs, while other app settings are set in the config directly.
Provide sample configuration files
Each service should provide a sample configuration file in e.g. meta/config
allowing a simple start of services. In the future those sample file could be updated using a script, also matching the versions of services in the docker-compose config.
Service developers provide sensible defaults on app level
Service developers are advised to provide meaningful default settings. Ideally admins do not even need to mount an app config for most scenarios. Only if they deviate from the standard installation, they can pass additional configuration.
An example for inspiration is the current Engine Core configuration, where most settings are even commented out, hence using the internal defaults.
(B) Keep the current approach, duplicating all config file options as ENV variables
App developers review all service configuration files, add additional mappings where required.
App developers are advised to add sensible defaults where possible. Optional ENV variables should not be required when starting with Docker or Docker Compose. Additional improvements as discussed below can be applied e.g. splitting of .env
files per service.
Note, that this doesn't solve the admin and developer challenges mentioned above. Also, 3rd party services like NGINX (which we use), will still have their bind-mounted config. As a consequence we are still mixing approaches.
(C) Any better suggestions
Is there a better option using a low-level approach?
Note, we want to avoid extending the technology stack with enterprise-grade configuration management tools. This ticket is mainly about the mapping between config options between service implementation and Docker/Docker Compose.
Sub Tasks
-
Update Docker Compose to deal with optional ser... (#132 - closed) -
Docker Compose: Tank configuration cannot be mo... (#109 - closed) -
Docker: Engine is not able to run standalone an... (engine#113 - closed) -
Modify Dockerfile to use included `sirv` server... (dashboard-clock#20 - closed) -
Sample configuration with meaningful defaults f... (engine-recorder#20 - closed) -
Sample configuration with meaningful defaults f... (engine#121 - closed) -
Sample configuration with meaningful defaults f... (engine-core#39 - closed) -
Sample configuration with meaningful defaults f... (engine-api#32 - closed) -
Optional/tbd: Expose settings.py in form of a YAML configurat... (steering#132)