docker local registry "exec: \"htpasswd\": executable file not found in $PATH"

Question

Until recently this worked fine

docker run --entrypoint htpasswd registry:2 -Bbn myuser  mypwd  > /my/registry2/reg/hub/auth/htpasswd

now its erroring out with

docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"htpasswd\": executable file not found in $PATH": unknown.

this is on Ubuntu 18.04 and 20.04 with docker

docker version
Client: Docker Engine - Community
 Version:           19.03.11
 API version:       1.40
 Go version:        go1.13.10
 Git commit:        42e35e61f3
 Built:             Mon Jun  1 09:12:22 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.11
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       42e35e61f3
  Built:            Mon Jun  1 09:10:54 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Answer 1

It will continue to work if you pin the local docker registry to

registry:2.7.0

instead of picking up the latest version 2 by just using registry:2 which is sadly broken

for details see https://github.com/docker/distribution-library-image/commit/ab00e8dae12d4515ed259015eab771ec92e92dd4 (they removed package apache2-utils) and https://github.com/GoogleContainerTools/jib/pull/2538/commits/f816c837e34eb389c2cdee1bc9a2918c5d2e33e3 and https://github.com/GoogleContainerTools/jib/pull/2539 as referenced in https://github.com/docker/distribution-library-image/issues/106

alternatively, instead of executing htpasswd from inside registry:2 you can install binary htpasswd using

apt-get install apache2-utils # thankfully this is NOT the apache server

and use syntax

htpasswd -Bbn myuser  mypwd  > /my/registry2/reg/hub/auth/htpasswd

on Ubuntu 18.04 or 20.04

PS here are all files which come from package apache2-utils ... just some utilities not any server

dpkg -L apache2-utils 

/.
/usr
/usr/bin
/usr/bin/ab
/usr/bin/checkgid
/usr/bin/fcgistarter
/usr/bin/htcacheclean
/usr/bin/htdbm
/usr/bin/htdigest
/usr/bin/htpasswd
/usr/bin/logresolve
/usr/bin/rotatelogs
/usr/sbin
/usr/sbin/check_forensic
/usr/sbin/httxt2dbm
/usr/sbin/split-logfile
/usr/share
/usr/share/doc
/usr/share/doc/apache2-utils
/usr/share/doc/apache2-utils/changelog.Debian.gz
/usr/share/doc/apache2-utils/copyright
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/ab.1.gz
/usr/share/man/man1/htdbm.1.gz
/usr/share/man/man1/htdigest.1.gz
/usr/share/man/man1/htpasswd.1.gz
/usr/share/man/man1/httxt2dbm.1.gz
/usr/share/man/man1/logresolve.1.gz
/usr/share/man/man8
/usr/share/man/man8/check_forensic.8.gz
/usr/share/man/man8/checkgid.8.gz
/usr/share/man/man8/fcgistarter.8.gz
/usr/share/man/man8/htcacheclean.8.gz
/usr/share/man/man8/rotatelogs.8.gz
/usr/share/man/man8/split-logfile.8.gz

for good measure I booked docker a ticket on this https://github.com/docker/docker.github.io/issues/11060

Answer 2

The current Docker documentation describes a simple way to generate a secret with htpasswd:

 mkdir auth
 docker run \
  --entrypoint htpasswd \
  httpd:2 -Bbn testuser testpassword > auth/htpasswd

The newly generated file auth/htpasswd can later be used in the registry image:

docker run -d \
  -p 5000:5000 \
  --restart=always \
  --name registry \
  -v "$(pwd)"/auth:/auth \
  -e "REGISTRY_AUTH=htpasswd" \
  -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \
  -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \