«

»

Mar 29

Swarming Raspberry Pi, Part 2: Registry & Mirror

This episode will consist of a quick aside to build a Docker Registry Mirror.

Previous: Part 1

Why Do We Need a Mirror??

The Docker Registry is, to my mind, one of the greatest contributors to the rapid growth of Docker. Having a central location for images encourages sharing and re-use. Obviously some images are more useful than others, but it is a wonderful resource for learning and leveraging the work of others. I like to look at Dockerfiles written by others; I learn a lot that way. It also helps me to determine which image to use where there are more than one providing a service. (Hint: I prefer to avoid those who do not share their Dockerfile).

In the Raspberry Pi Swarm, there are containers which will be pulled by all of the hosts. It takes a while to pull down images, even in parallel (at which point network congestion can cause problems). If an image is 300MB, unless we have a local mirror the entirety of the image is going to be pulled down across the internet by each host. It’s a lot quicker to cache the image locally and pull it once. Also, without a local mirror, there is a dependency on internet connectivity — something I am attempting to avoid with my Pi Swarm.

Configuring the Mirror Host

Requires:
1. Raspberry Pi
2. Hypriot Image (or other image supporting recent Docker)
3. External Disk (see below)

As before, I am starting from a Hypriot install. I have attached an external disk (in my case I’m using a SSD with LVM) to /opt/docker-cache to serve as my mirror. I’m not sure that using a micro-sd for the cache is the best way to go — it’s slow, for one thing. I’m not certain that the benefit of a less internet traffic and latency is overcome by the slow disk.

I’ve uploaded a Docker Registry image for the Raspberry Pi to the Docker Hub. You can pull it via:

The Dockerfile follows — there are a few differences from the stock Dockerfile:

If you want to build it yourself, you can do the following (on a pi with Docker):

Running the Mirror

Once you’ve built (or pulled) the registry image, it can be started on the registry host. In order for the mirroring to work properly, modifications to the other Docker hosts will need to be made. That follows below.

I am assuming that the external disk is mounted at: /opt/docker-cache.

There are a couple arguments of interest:

Argument Notes
-p 80:5000 The registry is exposed on port 80. This serves two purposes. First, if you wish to have a local registry of private images running on host registry, you can specify registry/project/image instead of registry:5000/project/image when pushing or pulling an image. Additionally, should you need to delete an image, docker is (at the moment) unable to delete images of the form HOST:PORT/PROJECT/IMAGE.
-v /opt/docker-cache:/tmp/registry This points to the location outside of the container which is used for the mirror’s cache. If you don’t specify a location, it’s kept within the container and once the container ends you’ve lost the benefits of a mirror.
-e GUNICORN_OPTS=["--preload"] This is to workaround a bug. The registry will not start up correctly without it. See Issue #892 · docker/docker-registry for a discussion of it.

Using the Mirror

Docker hosts which you wish to use the mirror need extra arguments passed into the daemon upon start. Assuming you are running a debian based version, the default configuration file is located at /etc/default/docker.

The DOCKER_OPTS variable needs to be edited to add --registry-mirror=http://REGISTRY_MIRROR. Replace REGISTRY_MIRROR with either the name or IP address of the host on which the mirror is running.

Once the file is edited, restart docker:

sudo service docker restart

To test, pull in a docker image you’ve not pulled before, remove it, then pull in again: (you may briefly see a message saying it’s using the mirror)

We went from ~44 to ~16 seconds total, almost 1/3 the time.

Limitations

Ultimately the limiting factor for improvement is I/O, namely the speed of the USB Bus and the bandwidth of the local network. In this case, the mirror is plugged into a 10/100Mb switch. The host performing the pull is connected over wireless. Kidzilla is watching Netflix.

As a quick test of network speed:

Ok, so in this totally unscientific test, I can fill 48Mb/s. Let’s try the same, but wired:

A little bit better; so at this point either the 10/100Mbs switch is being flooded or we’ve hit the limit of the USB bus. There are other processes running on the Pi, too, some of which are chatty.

Next Time

Next time is using consul and registrator along with swarm. If you’d like to play in the mean time, the images are out on docker hub:

Service Image Notes
consul nimblestratus/rpi-consul Port of progrium/consul
registrator nimblestratus/rpi-registrator Port of gliderlabs/registrator
swarm nimblestratus/rpi-swarm Port of docker/swarm. Details on how to use it at: Swarming Raspberry Pi Part 1

4 comments

2 pings

Skip to comment form

  1. Dieter Reuter (@Quintus23M)

    Matt, you’re using a SSD connected to the RPi. Could you just give some details on how to attach the SSD and maybe some performance comparison between SD cards and SSD in your setup? I’d like to get an idea how big the improvement is, one could gain when using a SSD attached via USB2.0.

  2. Matt Williams

    Definitely.

    I’ll work on that tonight.

    I’m just about finished with writeup on bonding NICs. I didn’t see much difference (in the artificial test I did) with the B+, but defintitely difference on the Pi 2B.

    And discovered something potentially ugly about cron on the B+ with the Hypriot version from the beginning of March… The short form is that cron is eating almost 1/2 the CPU…. even with no crontabs.

  3. Steve Woolley

    Hi Matt,
    Spelling on the rip-consul image table wrong on blog post. Should be “nimblestratus”. Only way I noticed was tried to cut and past this into docker hub search bar.

    Thanks for the excellent posts.

  4. Matt Williams

    Thanks for the catch!

    I’ll fix it right away.

    I’ve been having lots of fun with them, truth be told!

  1. Swarming Raspberry Pi: Private Registry for Swarm Images » Ramblings

    […] need to have a registry running. Swarming Raspberry Pi Part 2: Registry and Mirror has […]

  2. 350+ ???????? ????????, ???? ? ???????????? ??? ?????? ? Docker — ? ???? ??

    […] Matthewkwilliams: ????????????? Swarm ??? Raspberry Pi – ????????? ? ???????; […]

Leave a Reply

%d bloggers like this: