Debian httpredir mirror system unreliable/unusable in Docker?

Question

Short Version

Debian's httpredir.debian.org mirror service causes my Docker builds to fail very frequently because apt-get can't download a package or connect to a server or things like that. Am I the only one having this problem? Is the problem mine, Debian's, or Docker's? Is there anything I can do about it?

Long Version

I have several Dockerfiles built on debian:jessie, and Debian by default uses the httpredir.debian.org service to find the best mirror when using apt-get, etc. Several months ago, httpredir was giving me continual grief when trying to build images. When run inside a Dockerfile, apt-get using httpredir would almost always mess up on a package or two, and the whole build would fail. The error usually looked like a mirror was outdated or corrupt in some way. I eventually stopped using httpredir in all my Dockerfiles by adding the following lines:

# don't use httpredir.debian.org mirror as it's very unreliable
RUN echo deb http://ftp.us.debian.org/debian jessie main > /etc/apt/sources.list

Today went back to trying httpredir.debian.org again because ftp.us.debian.org is out of date for a package I need, and sure enough it's failing on the Docker Hub:

Failed to fetch http://httpredir.debian.org/debian/pool/main/n/node-retry/node-retry_0.6.0-1_all.deb  Error reading from server. Remote end closed connection [IP: 128.31.0.66 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

Here's the apt-get command I'm running in this case, though I've encountered it with many others:

RUN apt-get update && apt-get install -y \
  build-essential \
  chrpath \
  libssl-dev \
  libxft-dev \
  libfreetype6 \
  libfreetype6-dev \
  libfontconfig1 \
  libfontconfig1-dev \
  curl \
  bzip2 \
  nodejs \
  npm \
  git

Thanks for any help you can provide.

Answer 1

I just had the same problem today, when rebuilding a Dockerfile I had not build in a while.

Adding this line before the apt-get install seems to do the trick:

RUN apt-get clean

Got the idea here:

• https://github.com/docker/hub-feedback/issues/556
• https://github.com/docker-library/buildpack-deps/issues/40
• https://github.com/Silverlink/buildpack-deps/commit/be1f24eb136ba87b09b1dd09cc9a48707484b417

Answer 2

From the discussion on this question, and my experience dealing with this issue repeatedly over a number of months, apt-get clean seems to not in and of itself help, but the fact you're rebuilding (i.e. httpredir usually picks a different mirror) gets it to work. Indeed without exception manually triggering a rebuild or two has resulted in a successful build.

That is obviously not a viable solution, though. So, no, I don't have a solution, but I also don't have enough reputation to mark this as a duplicate.