How to fix locking failed in pipenv?

Question

I'm using pipenv inside a docker container. I tried installing a package and found that the installation succeeds (gets added to the Pipfile), but the locking keeps failing. Everything was fine until yesterday. Here's the error:

(app) root@7284b7892266:/usr/src/app# pipenv install scrapy-djangoitem
Installing scrapy-djangoitem…
Adding scrapy-djangoitem to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock (6d808e) out of date, updating to (27ac89)…
Locking [dev-packages] dependencies…
Building requirements...
Resolving dependencies...
✘ Locking Failed! 
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pipenv/resolver.py", line 807, in <module>
    main()
  File "/usr/local/lib/python3.7/site-packages/pipenv/resolver.py", line 803, in main
    parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
  File "/usr/local/lib/python3.7/site-packages/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/usr/local/lib/python3.7/site-packages/pipenv/resolver.py", line 758, in resolve_packages
    results = clean_results(results, resolver, project)
  File "/usr/local/lib/python3.7/site-packages/pipenv/resolver.py", line 634, in clean_results
    reverse_deps = project.environment.reverse_dependencies()
  File "/usr/local/lib/python3.7/site-packages/pipenv/project.py", line 376, in environment
    self._environment = self.get_environment(allow_global=allow_global)
  File "/usr/local/lib/python3.7/site-packages/pipenv/project.py", line 366, in get_environment
    environment.extend_dists(pipenv_dist)
  File "/usr/local/lib/python3.7/site-packages/pipenv/environment.py", line 127, in extend_dists
    extras = self.resolve_dist(dist, self.base_working_set)
  File "/usr/local/lib/python3.7/site-packages/pipenv/environment.py", line 122, in resolve_dist
    deps |= cls.resolve_dist(dist, working_set)
  File "/usr/local/lib/python3.7/site-packages/pipenv/environment.py", line 121, in resolve_dist
    dist = working_set.find(req)
  File "/root/.local/share/virtualenvs/app-lp47FrbD/lib/python3.7/site-packages/pkg_resources/__init__.py", line 642, in find
    raise VersionConflict(dist, req)
pkg_resources.VersionConflict: (importlib-metadata 2.0.0 (/root/.local/share/virtualenvs/app-lp47FrbD/lib/python3.7/site-packages), Requirement.parse('importlib-metadata<2,>=0.12; python_version < "3.8"'))
(app) root@7284b7892266:/usr/src/app# 

What could be wrong?

EDIT After removing Pipfile.lock and trying to install a package, I got:

(app) root@ef80787b5c42:/usr/src/app# pipenv install httpx
Installing httpx…
Adding httpx to Pipfile's [packages]…
✔ Installation Succeeded 
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Building requirements...
Resolving dependencies...
✔ Success! 
Locking [packages] dependencies…
Building requirements...
⠏ Locking...Resolving dependencies...
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 252, in install
    site_packages=state.site_packages
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 2202, in do_install
    skip_lock=skip_lock,
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1303, in do_init
    pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1113, in do_lock
    keep_outdated=keep_outdated
  File "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 1323, in venv_resolve_deps
    c = resolve(cmd, sp)
  File "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 1136, in resolve
    result = c.expect(u"\n", timeout=environments.PIPENV_INSTALL_TIMEOUT)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/delegator.py", line 215, in expect
    self.subprocess.expect(pattern=pattern, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/pexpect/spawnbase.py", line 344, in expect
    timeout, searchwindowsize, async_)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/pexpect/spawnbase.py", line 372, in expect_list
    return exp.expect_loop(timeout)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/pexpect/expect.py", line 181, in expect_loop
    return self.timeout(e)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/pexpect/expect.py", line 144, in timeout
    raise exc
pexpect.exceptions.TIMEOUT: <pexpect.popen_spawn.PopenSpawn object at 0x7f81e99bec90>
searcher: searcher_re:
    0: re.compile('\n')
<pexpect.popen_spawn.PopenSpawn object at 0x7f81e99bec90>
searcher: searcher_re:
    0: re.compile('\n')
(app) root@ef80787b5c42:/usr/src/app# 

Answer 1

Here are my debugging notes. Still not sure which package is causing the problem, but this does seem to fix it.

The error you get when you first run pipenv install with pipenv version 2020.8.13.

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/cli/command.py", line 252, in install
    site_packages=state.site_packages
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 1928, in do_install
    site_packages=site_packages,
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 580, in ensure_project
    pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 512, in ensure_virtualenv
    python=python, site_packages=site_packages, pypi_mirror=pypi_mirror
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 999, in do_create_virtualenv
    project._environment.add_dist("pipenv")
  File "/usr/local/lib/python3.6/site-packages/pipenv/environment.py", line 135, in add_dist
    self.extend_dists(dist)
  File "/usr/local/lib/python3.6/site-packages/pipenv/environment.py", line 127, in extend_dists
    extras = self.resolve_dist(dist, self.base_working_set)
  File "/usr/local/lib/python3.6/site-packages/pipenv/environment.py", line 122, in resolve_dist
    deps |= cls.resolve_dist(dist, working_set)
  File "/usr/local/lib/python3.6/site-packages/pipenv/environment.py", line 121, in resolve_dist
    dist = working_set.find(req)
  File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 642, in find
    raise VersionConflict(dist, req)
pkg_resources.VersionConflict: (importlib-metadata 2.0.0 (/usr/local/lib/python3.6/site-packages), Requirement.parse('importlib-metadata<2,>=0.12; python_version < "3.8"'))

If you run pip install -U pipenv it seems to change the importlib-metadata version:

Installing collected packages: importlib-metadata
  Attempting uninstall: importlib-metadata
    Found existing installation: importlib-metadata 2.0.0
    Uninstalling importlib-metadata-2.0.0:
      Successfully uninstalled importlib-metadata-2.0.0
Successfully installed importlib-metadata-1.7.0

Now if you run pipenv install -d --skip-lock it will finish. It seems like a library is requiring a version >= importlib-metadata 2.0.

When I pinned the following dependencies it didn't work at first when running pipenv lock, however, if I removed the lock file (rm Pipenv.lock) then it worked when I ran pipenv lock again.

virtualenv = "==20.0.31"
importlib-metadata = "==1.7.0"