Can I retest all installed CPAN modules?

Question

If a module A relies on module B and module B is upgraded, A may break due to changes. My idea is to retest both A and B after upgrading B.

I think the easiest way is just to retest everything that can be retested: download every installed module from CPAN and execute its test scripts.

Is there a way to download and retest?

If there is no way, are there any helpers/API so I can implement such tool?

I basically need to

• query what is installed (including version numbers)
• download and unpack exact versions
• execute tests

Answer 1

The cpan tool that ships with core Perl includes a -l option, that directs it to provide a list of installed modules. For example, the last 10 items in the list on my system:

$ cpan -l 2>/dev/null |tail -10
Test2::Event::Encoding  1.302073
Test2::Event::Bail  1.302073
Test2::Event::Exception 1.302073
Test2::Event::Subtest   1.302073
Test2::Event::Skip  1.302073
Test2::Event::Info  1.302073
Test2::Event::Diag  1.302073
Test2::Event::TAP::Version  1.302073
JSON::PP    2.27400_02
JSON::PP::Boolean   undef

As demonstrated here you get a list of modules and version numbers. Sometimes the tool doesn't find the version in META, and therefore will return undef for the version numbers. CPAN authors should be on the lookout for these sorts of mistakes, as they make it harder for tools that wish to identify the version without compiling the module itself.

Once you have the modules and version numbers you can use the cpanm tool (provided by App::cpanm) with its --test-only option to pull down a module of a specific version and test it. You can request a specific version like this:

cpanm Some::[email protected]

(Pulls down only version 0.9990 of the target module).

Where things get tricky are the following: Perl ships with a bunch of modules, and some of them also receive updates via CPAN. The cpan -l tool will list all installed modules, including the ones that ship with Perl.

Also some of the modules listed are simply part of a larger distribution.

Another tool that can be useful to you, which has been bundled with Perl since 5.8.9, is corelist. If you run this:

corelist File::Temp

You will get: "File::Temp was first released with perl v5.6.1"

If you do this:

corelist JSON

You will get: "JSON was not in CORE (or so I think)"

So it is pretty simple to determine if a module you're looking at in your list is one that ships with Perl or not. Use that information as you see fit.

Another thing you'll have to solve for is what to do about shared dependencies. If the first thing you test is a Moose upgrade, you'll pull in half of CPAN (that's an exaggeration), and that will dirty your environment for testing other modules. To mitigate this effect you have a few choices. One is to leverage App::perlbrew and its lib option to set up throwaway library space. That way you can install a module and its dependencies in a destination designated by perlbrew lib and perlbrew use, and then throw it away when done to move on to the next library for testing.

However, there may be a better solution, of which I am not adequately familiar to document here: The toolchain used by CPAN smoke testers. See CPAN::Testers if you wish to pursue this strategy. The smoke testers have worked out relatively lightweight approaches to pulling down and testing modules with their dependencies in an automated fashion.

Finally, another issue you will encounter is that CPAN authors are the ones who decide which versions of their modules live on CPAN and which ones get deleted. A few years ago CPAN authors were encouraged to keep their CPAN repositories clean by deleting old versions. I don't know if this practice is still encouraged, but it does mean that you cannot count on a specific version number still existing. To solve for this problem you should probably maintain your own repository of the tarballs for all versions you have installed at a given moment in time. The CPAN module framework Pinto helps with keeping versions of a module, pinning some to not update, and other useful tricks.

Answer 2

If you download and extract distribution B and cd into that directory, you can use the brewbuild binary from my Test::BrewBuild (NOTE: requires Perlbrew or Berrybrew) with the -R aka --revdep flag to test B as well as all distributions that require B:

~/repos/Mock-Sub $ brewbuild -R

reverse dependencies: Test::Module::CheckDep::Version, App::RPi::EnvUI, RPi::DigiPot::MCP4XXXX, Devel::Examine::Subs, PSGI::Hector, File::Edit::Portable, Devel::Trace::Subs

Test::Module::CheckDep::Version
5.26.1 :: PASS

App::RPi::EnvUI
5.26.1 :: FAIL

RPi::DigiPot::MCP4XXXX
5.26.1 :: FAIL

Devel::Examine::Subs
5.26.1 :: PASS

PSGI::Hector
5.26.1 :: PASS

File::Edit::Portable
5.26.1 :: PASS

Devel::Trace::Subs
5.26.1 :: PASS

I do this every time I update one of my CPAN distributions that have reverse dependencies, so I don't break any consumers of my dists.