After constantly battling with my system over free space at a crucial time when deadlines had to be met, I finally decided to spend time figuring out what the heck was going on.
docker-compose up was failing with errors like
INTERNAL ERROR: cannot create temporary directory! and on checking,
/root partition indeed had 0% free space. As a stopgap, I deleted unused kernel images, jdks, apt cache, etc. just so I could continue working.
I was surprised to find
/var/lib/docker/aufs/diff take 52GB of my precious SSD space. Surprised, because I have a cron job run everyday to delete dangling/stale containers and images, and I expected any disk space used by them to be freed up.
du -shc /var/lib/docker to see your damage. Note that this includes files that are used by some of your containers. Our servers also have the same issue, with the directory swelling up to 24GB+. The only difference on the server is it uses
overlay2 instead of
aufs as the storage driver.
docker system prune [docs]
Follow options that make sense in your case.
docker version to ensure this command is available. If this doesn't seem to free enough disk space (as seen in my case), upgrade to
17.06.2-ce+ [info] and run the above command again. This seems to fix the issue properly.
Note that upgrading docker will often affect your containers so take care doing this in production!
Having a couple of extra linux kernel images installed or Oracle Java installed always helps in freeing up disk space when you really need it!