Tag Archives: containers

Clean up unused objects in docker

The objective of cleaning is not just to clean, but to feel happiness living within that environment

Marie Kondo

Docker is a very useful open platform to run and develop applications – containers. The main features are fast and consistent delivery of your applications, responsive deployment and scaling and running more workloads on same hardware. If you haven’t played around with containers, it is a way to package and run application in loosely isolated environment. The whole ideology of having fast, separated spawned processes with only files you need, has been there for years and you have many implementations, docker being the most widely adopted platform. If you want to check, how the engine works, check out docs here.

There are a lot of articles with different techniques, how to remove unused objects, I’ll summarize most effective ones as I did encounter different challenges with running them on Linux.

Remove unused objects

Docker generally doesn’t remove unused objects automatically (images, containers, networks, volumes and networks). That is left up to you, but it gives you for each of the object different ways of removing items you don’t need to save space.

As of now, PC storage is not that expensive (you can get 1TB for +100€), but if you are hosting your containers and running them on some hosted platform (especially if you are doing CI/CD), you can quickly run out of bought space and that can result in additional money spent.

So, how to tackle cleaning up unused objects?

Remove it all in one shot?

Docker provides a prune command. The easiest way to run the prune is to use following command:

docker <object> prune 

More useful option is to run with –filter command – that enables you to define your own expressions, f.e. remove only images with specific labels or time frame. 

For example:

docker image prune -a --force --filter "until=240h" (remove images created more than 10 days ago)
docker image prune -a --force --filter "until=2021-01-01T00:00:00"' (remove images created before year 2021)
docker image prune --filter="label=maintainer=bojan" (remove all images with the label maintainer that is set to bojan)

Upper examples are with images, but you can easily use it on other objects as well:

docker container prune --force --filter "until=5m"

or you can use docker ps -a command to get all command and then execute rm command on top of that:

docker rm $(docker ps --filter=status=exited --filter=status=created -q)

Check prune documentation for more.

If you don’t want to manually remove each of the objects, you can call

docker system prune

which will remove everything –> stopped containers, network not used by at least one container, all dangling images and build caches.

It won’t prune volumes though (as they might contain data). You need to provide –volumes flag to include them in cleaning. If you add -f (–force) flag, you can skip the prompt dialog. 

If you want to automate prune command, you can create a cron job in Linux or Task Scheduler in Windows. How to do that, nicely explained here.

Most common task – remove unused images

Most common operations (besides containers), is to build an image and then work with different versions, updating it regularly and building new variations frequently. If you use prune on images, it will also delete unreferenced images. Which can be challenging, if you want to have some images, which doesn’t have any containers or usage, but want to have them present on your system (downloaded). Some of images can be big in size – for example full blow Windows Server image can result in 4.7 GB in size. Especially as in free tier you have limits on download quota. Also, on slow internet speed it helps to have images present locally.

Docker has a notion of dangling images. A dangling image is one that is not tagged and is not referenced by any container. You can create new image and if you don’t add the name, that can become “dangling image”. Those are the ones that are untagged (displays <none>), when you run docker images command.

Check this conversation on StackOverFlow. It has good tips, I summarized them below.

To check, what images are dangling, you can check the label with dangling enabled on:

docker images --filter=dangling=true
Dangling command execution

You can combine upper command with pipping and xargs:

docker images --quiet --filter=dangling=true | xargs --no-run-if-empty docker rmi

you can also shorten the call with:

docker rmi $(docker images -f "dangling=true" -q)

or use classic grep:

docker images | grep "<none>"
Grep call with docker dangling

If you combine that with awk, you can get images and you can execute that with xargs –no-run-if-empty docker rmi

docker images | grep "<none>" | awk '{print $3}' | xargs --no-run-if-empty docker rmi
Run with awk

If you are using this regularly, it makes sense to add it to your shell startup file as alias (how to do it here) or you to define command with your own understanding.

Conclusion

Docker has a nice built commands to take advantage of to have your docker environment tidy and performant. I do recommend having task scheduler or script with your command to have that cleaning automated. At least create alias for you to run on regular basic without knowing complex command. It is easier to call clean-images rather than piping 4 times. 

Sometimes creativity is needed. Combining piping with grep and awk, you can easily achieve whatever you need without affecting objects you don’t want to be affected without knowing inner works of docker.

If you are always running latest and greatest, purge is enough (usually in production), but if you are selective, scripts can do wonders, don’t be afraid to use it (of course if they are tested before running them in production).