What is a good way to manage luarocks rockspec files and why?

Question

I'm working on my first lua package, and I am deeply confused as to what to name my rockspec(s) and where to put them. Every popular lua package I look at seems to deal with rockspecs differently. This is very different from, say, Ruby, where every gem just has a single gemspec. Here are some examples:

• lua-cliargs: a single rockspec in the project root, (e.g. lua_cliargs-3.0-1.rockspec)
• ldoc: a single rockspec in the project root, named with scm instead of a version number (e.g. ldoc-scm-2.rockspec)
• lua-MessagePack: multiple, version-named rockspecs stored in a top-level rockspec/ directory (e.g. lua-messagepack-0.1.0-1.rockspec, ... lua-messagepack-0.3.4-1.rockspec), followed by some with an extra lua version inserted before the package version (e.g. lua-messagepack-lua53-0.3.6-1.rockspec). Sometimes there are two rockspecs for the same package version, one containing lua53 and one not.
• luafilesystem: multiple, version-named rockspecs stored in a top-level /rockspec directory (e.g. luafilesystem-1.3.0-1.rockspec, ... luafilesystem-1.6.1-1.rockspec), followed by some that have cvs instead of a package version number inserted before the package version (e.g. luafilesystem-cvs-1.rockspec, luafilesystem-cvs-2.rockspec).
• lpeg: no rockspec at all
• luasocket: one rockspec in the root containing scm instead of a package number (e.g. luasocket-scm-0.rockspec), and a /rockspec directory containing a single rockspec with a package number (e.g. luasocket-3.0rc2-1.rockspec)

Now, this question explains why one would have a "working" rockspec file and a separate directory full of release-versioned rockspecs, but I still have several questions:

1. What are rockspec revisions for? The luarocks wiki says that package releases should be tagged in git with the version number, and the example it gives does not include the rockspec revision. If my rockspec is in VCS, and I tag a particular commit, then doesn't that effectively fix the rockspec(s) included in that commit and therefore version? A later revision to the rockspec couldn't possibly be in the tagged commit.
2. How can lpeg be listed on luarocks but lack a rockspec?
3. The luarocks wiki says that scm should be used in lieu of a package version number if you want your rockspec to refer to HEAD. But since HEAD is continually changing, and the rockspec has to list all of the files, it would seem that one would need to continually increment the revision number of the scm rockspec to keep up with any added or deleted files. One could get around this by not using a revision number at all for scm rockspecs, and yet the ones I see have low revision numbers (e.g. ldoc-scm-2.rockspec). Is this a mistake?
4. Are any of the above examples considered a best practice?

Answer 1

What are rockspec revisions for?

The rockspec revision is the versioning of the rockspec file itself. Suppose you release Foo version 1.0; you create a rockspec foo-1.0-1.rockspec. Later you learn that to get Foo 1.0 to compile in FreeBSD, you need to pass an extra -D flag; the source code needs no changes at all. You edit the rockspec adding a platform override section and re-submit it to luarocks.org as foo-1.0-2.rockspec.

If my rockspec is in VCS, and I tag a particular commit, then doesn't that effectively fix the rockspec(s) included in that commit and therefore version?

Yes. But that is not a problem, because the rockspec used when building is not the one included with the source distribution. A .src.rock file is an archive containing the submitted .rockspec file and the source code tarball (or a source checkout in a subdir if the rockspec uses an SCM protocol such as git://).

Indeed, this leads to a chicken-and-egg problem in that the latest rockspec for Foo 1.0 is not there when one checks out tag v1.0 manually from Github, but that is not the expected workflow: when using LuaRocks, a user will normally use luarocks install foo; and if they want to checkout the rockspecs for a project, they would either visit the page for Foo at luarocks.org or they would look at the rockspecs stored in HEAD, which is where people stop by first anyway.

Note that a similar chicken-and-egg situation happens if one wanted to distribute a source tarball containing a rockspec and then wants to set the tarball source.url and its corresponding source.md5 in the rockspec. Having an MD5 there implies that the rockspec itself cannot be inside the tarball. One way around it is to simply avoid the source.md5 field, or to skip the rockspec when packaging the tarball. This is the same situation that would happen in one would include Linux-distribution package metadata in an upstream tarball; it's more evident here because upstream and packager tend to be the same person.

How can lpeg be listed on luarocks but lack a rockspec?

Probably because this is a rare case where upstream and packager are not the same person. Roberto Ierusalimschy released lpeg, but as of December 2016, the rockspecs for it are uploaded by Gary Vaughan.

[...] the ones I see have low revision numbers (e.g. ldoc-scm-2.rockspec). Is this a mistake?

This could have many reasons and it could be a mistake or not.

• If a rockspec uses the make builtin, then the scm rockspec will likely not change when files are added or removed;
• Some projects just don't change their set of files all that often, so the revision remains low;
• Some developers keep their scm rockspecs at -0 (as in "not a released revision") and never upload them to luarocks.org (keeping only released versions there). So, the one you get when you fetch a git revision is the valid one for that snapshot. Incrementing revisions is an expected practice for rockspecs released to luarocks.org;
• The rockspec could be indeed out of date and then that's a mistake.

Are any of the above examples considered a best practice?

Objectively speaking, since the rockspec file used when one runs luarocks install foo is the one bundled inside the .src.rock file, stored separately from the source archives, there are no major practical consequences as to where exactly in the source tree a developer stores their rockspecs. It's a matter of personal organization.

Keeping the latest scm rockspec at the root has the slight advantage that it is automatically picked up by luarocks make if one wants to make a build from a local checked-out tree.

But as long as rockspecs are uploaded to the server with luarocks upload foo, the end-user experience will be the same, regardless where the rockspecs are in the source tree.