Hi folks.
I am running a HP ProLiant N40L microserver at home as the core of my media-setup. It acts as a usenet-indexer, download manager, storage server with a ZFS RAID, OpenVPN server and some other services.
The only thing that was missing was a solution to stream all the media files to devices outside of the network. At home the server just acts as a samba server for my Kodi-equipped FireTV and FireStick, which works absolutely brilliant. But after I didn’t renew my Plex Premium Pass last year, I had no further option to stream my media over the net. I still could use the VPN connection to connect to my samba shares, but thats not very practicable. With limited upload (narf) you can’t stream HD series over my connection without reencocoding it to a more upstream-friendly bitrate.
So I dediced to give „Emby“ a go.
Emby is kinda the open source brother of Plex and, although offering paid premium options, the core functions (App for iOS and Android, streaming/reencoding to my iPad, Notebook over the internet) seem to be sufficient for my use case.
But instead of just installing Emby and messing with my server installation, I took the Docker approach.
Here is what I did. Luckily the guys at Emby provide a „ready to roll“ Docker image, I only had to mount a volume for the configuration and my media folder into it (I did this „read only“ as I don’t want Emby to mess with my metadata).
This command is all you need to get Emby running:
docker run -d --name emby -v /path/to/emby/configdir:/config -v /path/to/your/mediafiles:/media:ro -P emby/embyserver
The „-P“ switch tells Docker to expose all defined ports of the Emby-Dockerimage, which in my case looked like this:
0.0.0.0:32777->1900/udp, 0.0.0.0:32776->7359/udp, 0.0.0.0:32800->8096/tcp, 0.0.0.0:32799->8920/tcp
So Docker is opening the defined ports on arbitrary ports on your host. In this case you could reach the webinterface of emby on „localhost:32800“.
To make it accessible from the internet you need a webserver/reverse proxy, in my case I already have an nginx running on my server, handling the access to a bunch of webinterfaces. So that is what I am going to use. Of course you can define the host-port under which emby should be reachable when running the docker-container with the „-p“ switch and then just use the port in your nginx upstream configuration. But that again is not how I wanted to do this.
I wanted Docker to be able to dynamically assign the ports without having to rewrite my nginx config after every restart/reboot.
In order to archieve this, I made use of a combination of Hashicorp’s Consul and Consul-Template and a modified version of the great „Registrator“ tool written by Progrium from Gliderlabs.
As preparation for getting Consul and Registrator see each other I used the new „network“ feature available since Docker 1.9
With
docker network create consul
I created a network in which both services could talk to each other. Then I started the Consul container with
docker run -d --name consul --net=consul -p 8400:8400 -p 8500:8500 -p 8600:53/udp -h node1 progrium/consul -server -bootstrap
The Consul image is also provided by Progrium and is in this case running in single server and bootstrap mode since I don’t run a cluster and don’t need any consistency. When the container is killed, just fire up another on and bootstrap a new Consul server. The whole range of exposed ports isn’t really necessary, but I wanted to be able to check the Consul service with the Consul binary from my host server so I just exposed them all. You could probably try out which ones are needed for the binary and the Registrator and just expose them, but as a quick solution I went full monty.
As you can see, the container is named „consul“ and the „–net=consul“ switch tells the container to use the previously created docker network consul.
So with that in place it is easy to start the Registrator container, assign it to the same consul network and let him talk to the consul endpoint with this command:
docker run -v /var/run/docker.sock:/tmp/docker.sock -h $hostname --name registrator --net=consul -d codemonauts/registrator -ip $YourInternalHostIp -ttl 600 -ttl-refresh 300 -resync 600 -cleanup consul://consul.consul:8500
This command mounts the docker.sock into Registrator, so it can listen to docker events on the host and assigns it to „–net=consul“. The switches added after the registrator container are used to configure the service itself.
I set the IP manually to my internal host ip, because that is what I want to use as my upstream address in nginx.
The ttl, refresh and resync switches tell Registrator in which intervals to refresh the existing, and to clean up dangling, unresponsive services. That is an extra feature which is (was?) not yet included in the original Registrator, so I used the modified version in my own image.
The endpoint is „consul:consul.consul:8500“ which roughly translates to „search for a consul endpoint on the host named consul in the network named consul on this port.“ Which, of course, points to our previously started Consul container.
Now you can check with „docker logs registrator“ if it can talk to the consul service and finds any running docker services with exposed ports. As the Emby server is already running, it should now be found and registered as a service in Consul.
From the logs on my machine you see the output of the registrator „refresh“ with two instances of emby running (again with all ports exposed, which probably can be optimized). Important is the port 8096 which exposes the webinterface of emby.
2015/11/26 11:49:06 refreshed: a24012cdf43e batcave:emby:8920 2015/11/26 11:49:06 refreshed: a24012cdf43e batcave:emby:1900:udp 2015/11/26 11:49:06 refreshed: a24012cdf43e batcave:emby:7359:udp 2015/11/26 11:49:06 refreshed: a24012cdf43e batcave:emby:8096 2015/11/26 11:49:06 refreshed: 2b65a593330d batcave:emby_1:7359:udp 2015/11/26 11:49:06 refreshed: 2b65a593330d batcave:emby_1:8096 2015/11/26 11:49:06 refreshed: 2b65a593330d batcave:emby_1:8920 2015/11/26 11:49:06 refreshed: 2b65a593330d batcave:emby_1:1900:udp
This translates to the following services in Consul. On my host I can access Consul with:
consul watch -type services
and get the output
"embyserver-8096": [], "embyserver-8920": [] }
So we see there is a service entry for every exposed port in emby. What we want are the addresses and ports where the webinterface can be reached, so with
consul watch -type service -service embyserver-8096
we get the following response:
[
{
"Node": {
"Node": "node1",
"Address": "172.18.0.2"
},
"Service": {
"ID": "batcave:emby_1:8096",
"Service": "embyserver-8096",
"Tags": null,
"Port": 32800,
"Address": "$YourInternalHostIP"
},
...
{
"Node": {
"Node": "node1",
"Address": "172.18.0.2"
},
"Service": {
"ID": "batcave:emby:8096",
"Service": "embyserver-8096",
"Tags": null,
"Port": 32798,
"Address": "$YourInternalHostIP"
}
]
We are mostly interested in the port value, as we know that the service address is hardcoded as our host ip, but when you use that infrastructure in a cluster, the address is also likely to change regularly.
So now we have an infrastructure that dynamically picks up docker event (new containers) and registers the services to consul sorted by the exposed port. How can we translate the values from consul to a dynamic NGINX configuration?
The answer is consul-template. A tool that watches the consul endpoint for changes, rewrites that changes with help of a template to a nginx configuration and the reload nginx itself.
You can also run consul-template in an docker container, but in my case I just downloaded the linux binary from Hashicorp and used this. After creating the necessary folder structure I created the following configuration file:
consul = "127.0.0.1:8500"
syslog {
enabled = true
facility = "LOCAL5"
template {
source = "/etc/consul-template/mediacenter.tmpl"
destination = "/etc/nginx/sites-available/mediacenter"
command = "/etc/init.d/nginx reload"
}
Again, as I expose all Consul ports to my host machine, consul-template is able to talk to Consul via localhost.
It tells consul-template which template to use (mediacenter.tmpl), which config to create (mediacenter) and what command to fire after changes are made (reload nginx).
The corresponding template in my case looks like this (just the parts that are interesting, no need to expose my mediaserver to the public):
upstream emby {
ip_hash; {{ range services }}{{ if .Name | regexMatch "embyserver-8096" }}{{ range service .Name "passing" }}
server {{ .Address }}:{{ .Port }}; {{ end }} {{ end }} {{ end }}
}
server {
...
}
...
location /emby/ {
proxy_pass http://emby/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-for $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $remote_addr;
proxy_set_header X-Forwarded-Protocol $scheme;
proxy_redirect off;
# websocket configuration
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
This template looks over the range of (healthy aka passing) services that matches the given regex (embyserver-8096 as this is the webinterface) and creates for every occurence a server entry with the address and port values taken from Consul.
I borrowed the location configuration for emby from Xsteadfastx’s Blog, especially the websocket part, so many thanks for that, !
Now for usability I have this small upstart script for consul-template in /etc/init:
# Consul Template (Upstart unit) description "Consul Template" start on (local-filesystems and net-device-up IFACE!=lo) stop on runlevel [06] exec /usr/local/bin/consul-template -config=/etc/consul-template/config >> /var/log/consul-template.log 2>&1 respawn respawn limit 10 10 kill timeout 10
With that in place you can control the consul-template binary quite comfortable.
So all you have to do now is fire up „start consul-template“ and check the log for errors. If everything is set up correctly, your log should look like this:
2015/11/24 17:41:14 [DEBUG] (config) merging with "/etc/consul-template/config/consul-template.conf" 2015/11/24 17:41:14 [DEBUG] (logging) enabling syslog on LOCAL5 * Reloading nginx configuration nginx ...done.
No error messages, consul-template is running and nginx reloaded without complains.
Now its time to check the created config file if everything is in place and if this is the case you should be able to reach the Emby webinterfaces via $yourip/domain/emby.
Right now I use this setup to handle two running containers with Emby and a Lychee installation based on a MySQL and a nginx container, managed by docker-compose and it works nice and stable so far.