Publishing large NPM packages

Question

I'm trying to publish large npm package to a private Nexus 3 nmp repository, but the operation fails with the following error

npm ERR! node v7.7.4
npm ERR! npm  v4.1.2

npm ERR! "toString()" failed
npm ERR! 
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

The problem is at V8 level and appears when building the request for publishing to Nexus.

It's not a good practice to publish large npm packages, but it's a third party plugin, and we need it for the project.

Is there a way to configure/patch V8 to support bigger file size ?

What is the request format for uploading npm package to nexus, so I can try to transform the package to encoded string using some other tool ?

Answer 1

I have published the package using CURL. The format of the request is

curl -H "Content-Type: application/json"\
-H "Authorization: Basic ${authorization_token}"\
-H "Content-Length: ${actual size of the request body in bytes (just check the size of the request body file)}"\
-H "Accept-Encoding: gzip"\
-H "version: 8.1.1"\
-H "Accept: application/json"\
-H "npm-session: a972cb330cbacab5"\
-H "npm-in-ci: false"\
-H "User-Agent: node/v7.7.4"\
-H "Host: nexus.example.com:8081"\
-X PUT\
--data-binary @path_to_request_body_file\
--verbose\
http://nexus.example.com:8081/nexus/repository/npmrepo/ExamplePackage

nexus.example.com:8081 - are the actual nexus server host and port
authorization_token - is nexus authorization token
npmrepo - is a npm repository created through the Nexus Repository Manager, where the ExamplePackage will be published
path_to_request_body_file - should be file path to the request body file with format of the content as follows

{
  "_id": "ExamplePlugin",
  "name": "ExamplePlugin",
  "description": "Example Plugin",
  "dist-tags": {
    "latest": "0.1.0"
  },
  "versions": {
    "0.1.0": {
      "name": "ExamplePlugin",
      "version": "0.1.0",
      "cordova_name": "ExamplePlugin",
      "description": "Example Plugin",
      "license": "Apache 2.0",
      "keywords": [
        "ExamplePlugin",
        "StackoverflowExample"
      ],
      "platforms": [
        "ios",
        "android"
      ],
      "engines": [],
      "dependencies": {},
      "maintainers": [
        {
          "name": "example_nexus_user",
          "email": "[email protected]"
        }
      ],
      "_id": "[email protected]",
      "dist": {
        "shasum": "${shasum of the package archive file}",
        "tarball": "http://nexus.example.com:8081/nexus/repository/npmrepo/ExamplePlugin/-/ExamplePlugin-0.1.0.tgz"
     }
    }
  },
  "readme": "",
  "access": "public",
  "maintainers": [
    {
      "name": "example_nexus_user",
      "email": "[email protected]"
    }
  ],
  "_attachments": {
    "ExamplePlugin-0.1.0.tgz": {
      "content_type": "application/octet-stream",
      "data": "${Base64 encoded content of the npm package archive}",
      "length": "${actual size of the package archive in bytes}"
    }
  }
}

If the package archive file doesn't already exist, you can use npm pack to generate it.
I have used openssl to generate base64 encoded string for the package archive openssl base64 -in ExamplePlugin-0.1.0.tgz -out ExamplePluginEncoded, but any other tool that supports large files may be used.
I have generated the shasum for the package archive using shasum -a 1 ExamplePlugin-0.1.0.tgz, some other tool may also be used.

I got the format of the request and the JSON payload debugging the npm-registry-client, some of the fields are probably not required, some more options may also be available.