Using Docker and a Private Registry with VPN On Windows
Wasn’t that a very specific title? Docker has a very good documentation and reading that alone is enough for most of the straightforward tasks we might want to do. But as always some practical tasks are not straightforward, hence this blog. What we are going to see here today is how to setup docker toolbox on a Windows machine, make it work even when VPN is connected, make it talk to a private, insecure docker registry (that is why VPN) and configure it so it can run docker compose and see how we can set this config as a one-time activity. That’s quite a mouthful, but yes this is what we are going to do. All ready? Let us begin then.
Install Docker Toolbox
Go and download the docker toolbox and install it. That should create a shortcut called “Docker Quickstart Terminal”. Run it. That should show you an error about virtualization.
Enable Virtualization
Restart your machine, enter the BIOS settings and enable virtualization. It may be under advanced settings. On this Laptop, it is under the advanced settings -> device configurations and is named as: “Virtualization Technology (VTx)”. Whatever be the name, enable it.
Docker requires a Linux kernel, and since Windows machines lack it (of course!), docker toolbox runs a lightweight Linux distro called boot2docker in a virtualbox, hence the virtualization setting.
A Handy Tip
This tutorial will require you to copy and paste quite some shell commands, it is better we make that easy. Exit the quickstart terminal. Right click the shortcut, click properties -> options and enable ‘Quick Edit’ mode and save. It might ask for permission. This should now enable paste just by right clicking the mouse, to copy just select the text with mouse. While we are at that, also consider increasing the buffer and window size to suite your taste.
Start Up the VM
Make sure you are not connected to VPN and use the Quickstart Terminal shortcut again, this time it should proceed to validate if the boot2docker image is latest, or it shall pull the latest image, then it shall create a VM, get an IP, setup some ssh keys and finally the whale should appear with a terminal. Run the following commands to get a hang of docker running on windows:
docker -v docker version then docker run hello-world docker images docker ps -a
(And do read the output of hello-world, it describes how docker works).
The Disappointment
Feeling happy? Now for a little disappointment, connect VPN and try again. Errors errors everywhere. Disconnect VPN. What happened: Docker is running in a virtualbox on your machine, which gets an IP in local range (normally: 192.168.99.100), and you are talking to it over ssh. Once VPN is up, it sets the new routes and sends the 192.168.* range traffic out over VPN and your commands never reach your VM running docker. The most popular solution to this is setting a port forwarding and is documented on many blogs/githubissues. Let’s just do that.
A new Beginning
Ensure you are not on VPN and remove the default VM, not necessary, but reduces the confusion. So in the quickstart terminal:
docker-machine rm default
And confirm. We are now going to create a new VM, let us call it ‘custom’. So type in:
docker-machine create -d virtualbox custom eval "$(docker-machine env custom)"
It might take a couple of minutes, it is almost the same process as the first time. What we did is created a VM named custom and setup the environment to talk to this VM instead of the default. Mark this step, cause if anything goes wrong in the following steps, this is the one you should get back to to start over. Just be sure to use a new name, docker currently does not allow reusing names for VMs, so next time you may not be able to create a VM called custom. A new name should work just fine.
Battling With VPN
Now we shall create the a port forwarding on the virtual machine, binding the default docker port (2376) on localhost/127.0.0.1 to forward to this VM, whatever the ip of it.
docker-machine stop custom /c/Program Files/Oracle/VirtualBox/VBoxManage.exe modifyvm "custom" --natpf1 "docker,tcp,,2376,,2376" docker-machine start custom docker ps -a
If you changed the location of virtualbox installation, please use appropriate path to vboxmanage. Assuming it was successful, last command should show you a table with all containers. You can use UI to do that as well: Open VirtualBox, stop the VM, open settings -> network -> NAT adapter -> advanced -> Port forwarding. Click add rule and use the same values as above (comma separates columns). If the command was successful, you should see the rule listed at the same location. Also, this is the place to add an entry if you need any port exposed from a docker container and use it with VPN enabled; for example your application’s tomcat port.
We are not done yet, a few more commands:
export DOCKER_HOST="tcp://localhost:2376" export DOCKER_TLS_VERIFY="0" alias docker="docker --tlsverify=false"
Kudos to this smart guy for that alias. In other posts, you might find IP of the VM (which does not work), public IP of your machine, or even loopback IP (127.0.0.1) being used, which might work but I would advise against that. Use ‘localhost’ instead; this and the TLS setting has to do with running docker-compose.
Now enable VPN and enjoy docker. This is where your journey ends if you are not using a private registry; but if you are, then continue.
Configuring Private Insecure Registry
Ensure that VPN is down, and ssh into the docker-machine. We want to enable it to talk to an insecure registry. A private docker registry does not need a name, but docker images in a non-docker-hub registry require that they be tagged with the URL of the registry prefixed to the usual repository name. They say it is for transparency, helps in identifying where the image originates from. Hence, it would be advisable to have a host-name even if your registry is private and has a static IP. That way even if you change the IP of the registry for whatever reason, you do not have to update all images/tags/compose ymls, shell scripts and whatever else is using them. Let us say our registry is hosted at: dockerregistry.example.com, on port 5000 and this being insecure, of course, is accessible only over VPN.
This step is intentionally manual, to avoid risks of breaking something else:
docker-machine ssh custom sudo vi /var/lib/boot2docker/profile
In the EXTRA_ARGS, before the closing quote, add this line: --insecure-registry=dockerregistry.example.com:5000
(I would ensure a blank line before the quote, as there already was) Save the file and exit vi (:wq). We now need to restart the docker daemon for changes to take effect:
sudo /etc/init.d/docker stop Ensure service is down: sudo /etc/init.d/docker status sudo /etc/init.d/docker start Ensure service is up: sudo /etc/init.d/docker status
Exit the VM by typing exit in terminal. (BTW, there is restart command too)
Using the registry
Now let us try pushing and pulling from this registry. In the quickstart terminal:
docker tag hello-world dockerregistry.example.com:5000/hello-world docker push dockerregistry.example.com:5000/hello-world docker rmi dockerregistry.example.com:5000/hello-world docker run dockerregistry.example.com:5000/hello-world
What we did is tagged an image with the registry, pushed it to the private registry, removed the local copy and run the image by pulling from this registry.
Docker Compose
Next step is to get docker compose up and running with this setup. Actually, we are already ready, everything that we need to run docker-compose is taken care of in the previous steps. Most importantly docker-host configuration. You see, the TLS certs allow only for docker-machine IP and localhost to be used even when we disable verification, but we have already taken that into account and we have already configured our private registry. All set. Just connect VPN, navigate to your directory with docker-compose.yml file and hit: docker-compose up. You should see the images in compose file getting pulled and executed.
Starting the quickstart terminal second time
When you restart the quickstart terminal you might find that it recreates the ‘default’ VM and configures the environment to use it. That is okay, it does not bother us. But what does bother us is that none of the docker commands are working with VPN again. Please keep reading..
Consecutive starts of quickstart terminal
Well, we have to reconfigure the terminal every time to use our VM of choice. Here is how to do it:
Always make sure that you start the terminal when VPN is down. Starting with VPN up has never worked for me; and then run these commands:
eval "$(docker-machine env custom)" export DOCKER_HOST="tcp://localhost:2376" export DOCKER_TLS_VERIFY="0" alias docker="docker --tlsverify=false"
Yes, every time you start the terminal. There is a way to avoid this, read on.
One Time Setup: For The Brave Among Us
From this point on, you are entering undocumented territory and are on your own. If something breaks, do not come looking for me. 🙂 And before making any modifications, take a backup.
If you notice, the shortcut points to a shell script called ‘start.sh’. We are going to modify this script to auto-configure our environment every time it is called. Navigate to docker installation directory (directory that quickstart shortcut is pointing to) and open the start.sh (After creating a backup) file in a text editor.
Change 1: On line number 10 which looks like: VM=${DOCKER_MACHINE_NAME-default}
change that line to: VM=custom. Custom here is the name of our VM. This saves you from typing the eval line every time.
Change 2: On line 66/67, in “Setting Env” step, after the existing eval command add the following lines:
eval "DOCKER_HOST="tcp://localhost:2376"" eval "DOCKER_TLS_VERIFY="0"" eval "alias docker=docker --tlsverify=false"
These handle rest of the config. That is all, save and exit the file and we are ready to roll. This may break when an update the docker toolbox is installed which overwrites the file, may not work if the script changes in future, may break things I am not aware of, hence only for the brave. Besides, I do not use a Windows machine daily, so you guys would be first to know if it starts breaking ;). Let me know and we will figure it out.
2 thoughts on “Using Docker and a Private Registry with VPN On Windows”
Gr8 it works like a charm. Thanks a ton buddy. It saved lots of time. One small changes here instead of alias docker=”docker –tlsverify=false” it should be alias docker=”docker –-tlsverify=false” ( double hyphen instead of single hyphen).
Ah, thanks for bringing it to my notice. That is how it was written, but the text formatting ruined it, it should have been code formatting. Probably something that happened during the blog migration. Fixed. (There are still some formatting issues, will come back to them later.)