Running `grunt` on VM Shared Directory

Question

Using Vagrant with a Windows host and Linux guest, grunt returned the following error when trying to run a job.

As I understand, this file path exceeds the 255 character limit of Windows when on a host-guest shared directory.

npm ERR! Error: EPERM, open '/u01/aa/bb/build/share/app-core/app-
ui/node_modules/grunt-contrib-imagemin/node_modules/pngquant-bin/node_modules/bin-
wrapper/node_modules/download/node_modules/request/node_modules/form-
data/node_modules/combined-stream/test/integration/test-delayed-streams-and-buffers-
and-strings.js'

I could develop on a non-shared directory on my guest VM, but I'd prefer to use a shared directory since I use an IDE on the host.

How can I fix this issue so that I can run grunt on the shared directory?

Answer 1

I experienced a similar issue while running yeoman with angular-generator. The solution I found right now is a little trick leveraging the way npm handles it's dependencies and folder structure.

Please consider that the problem you are experiencing is probably only affecting a few files (and probably only the imagemin module).

For Yeoman my solution was to first generate, skipping the automatically started installation:

yo angular appname --skip-install

Starting bower installation manually:

bower install

Starting from here it should be interesting to you: Install "request" npm module as first level module (instead of a depencdency deeply resolved inside imagemin later):

npm install request --save-dev

So if you now run the regular npm install to fetch the remaining modules, the already existing request module is used.

npm install --no-bin-links

Please note that using --no-bin-links is also a good idea if you don't want to run VirtualBox as an Administrator.

So, to answer your question. Running grunt should work if you correctly setup your packages and call npm install before. The module "request" is an arbitrary module that prevents the big path of the test-delayed-streams-and-buffers-and-strings.js file.

Answer 2

If you are using the default vboxsf synced folder type, you may consider using the new rsync based synced folder to work around the issue.

rsync type will monitor the folders between host and guest, copy files directories to keep them in sync.

The difference between default vboxsf and rsync is that the latter will cause the guest Linux to run the application (grunt in your case) from its native file system (e.g. ext4) instead of vboxsf (provided by VirtualBox Guest Additions, after all the files are on Windows).

rsync synced folder basically make use of rsync CLI utility to copy / sync files and directories between hosts. So there are 2 copies of everything. When you run your grunt against the file system within the guest, it is native Linux file system and should not be affected by long path (but ext4 does have a max filename length of 255 bytes).

By default, Vagrant only syncs the folders on vagrant up or vagrant reload. But you can run vagrant rsync to manually force sync or vagrant rsync-auto to monitor changes and sync in real time.

Reference: http://docs.vagrantup.com/v2/synced-folders/rsync.html

You can set synced folder type to rsync in your Vagrantfile below

Vagrant.configure("2") do |config|
  config.vm.synced_folder ".", "/vagrant", type: "rsync",
    rsync__exclude: ".git/"
end

Update

Tested creating the directory tree structure and a file, no problem opening it from a Linux perspective, the file system is ext4. I suspect this is a problem with npm.

terry@arch:/$ uname -a
Linux arch.apac.lab 3.14.1-1-ARCH #1 SMP PREEMPT Mon Apr 14 20:40:47 CEST 2014 x86_64 GNU/Linux
terry@arch:/$ cat /u01/aa/bb/build/share/app-core/app-ui/node_modules/grunt-contrib-imagemin/node_modules/pngquant-bin/node_modules/bin-wrapper/node_modules/download/node_modules/request/node_modules/form-data/node_modules/combined-stream/test/integration/test-delayed-streams-and-buffers-and-strings.js
test

Update - rsync synced folder test

I did the previous test on a physical Linux installation. The following is on a quick test on Mac, guest is the same as above.

rsync synced folder configuration

config.vm.synced_folder ".", "/u01/aa/bb/build/share/app-core/app-ui/           node_modules/grunt-contrib-imagemin/node_modules/pngquant-bin/node_modules/bin-   wrapper/node_modules/download/node_modules/request/node_modules/form-data/        node_modules/combined-stream/test/integration", type: "rsync 

NOTE: I chown -R vagrant:vagrant /u01/ recursively beforehand to avoid permission issues when syncing.

sync on vagrant up

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 => 2222 (adapter 1)
==> default: Running 'pre-boot' VM customizations...
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
    default: Warning: Connection timeout. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Rsyncing folder: /ops/arch64/ => /u01/aa/bb/build/share/app-core/app-ui/node_modules/grunt-contrib-imagemin/node_modules/pngquant-bin/node_modules/bin-wrapper/node_modules/download/node_modules/request/node_modules/form-data/node_modules/combined-stream/test/integration
==> default: Mounting shared folders...
    default: /vagrant => /ops/arch64
==> default: VM already provisioned. Run `vagrant provision` or use `--provision` to force it

$ vagrant ssh
Last login: Sun Apr 27 10:36:10 2014 from 10.0.2.2
vagrant@archlinux:~$ cd /u01/aa/bb/build/share/app-core/app-ui/node_modules/grunt-contrib-imagemin/node_modules/pngquant-bin/node_modules/bin-wrapper/node_modules/download/node_modules/request/node_modules/form-data/node_modules/combined-stream/test/integration
vagrant@archlinux:/u01/aa/bb/build/share/app-core/app-ui/node_modules/grunt-contrib-imagemin/node_modules/pngquant-bin/node_modules/bin-wrapper/node_modules/download/node_modules/request/node_modules/form-data/node_modules/combined-stream/test/integration$ ls -l
total 12
-rw-r----- 1 vagrant vagrant   14 Apr 27 10:35 test-delayed-streams-and-buffers-and-strings.js
-rw-r----- 1 vagrant vagrant 5018 Apr 27 10:22 Vagrantfile
vagrant@archlinux:/u01/aa/bb/build/share/app-core/app-ui/node_modules/grunt-contrib-imagemin/node_modules/pngquant-bin/node_modules/bin-wrapper/node_modules/download/node_modules/request/node_modules/form-data/node_modules/combined-stream/test/integration$ cat test-delayed-streams-and-buffers-and-strings.js
stackoverflow

Result is the same. As mentioned earlier, I suspect that this may be a npm issue.

Answer 3

For an actual solution to this, check out this comment, or continue reading here:

config.vm.provider "virtualbox" do |v|
    v.customize ["sharedfolder", "add", :id, "--name", "www", "--hostpath", (("//?/" + File.dirname(__FILE__) + "/www").gsub("/","\\"))]
end

config.vm.provision :shell, inline: "mkdir /home/vagrant/www"
config.vm.provision :shell, inline: "mount -t vboxsf -o uid=`id -u vagrant`,gid=`getent group vagrant | cut -d: -f3` www /home/vagrant/www", run: "always"

In the code above, I am appending \\?\ to the current directory absolute path. This will actually force the Windows API to allow an increase in the MAX_PATH variable (normally capped at 260). Read more about max path. This is happening during the sharedfolder creation which is intentionally handled by VBoxManage and not Vagrant's "synced_folder" method. The last bit is pretty self-explanatory; we create the new shared folder and then make sure it's mounted each time the machine is accessed or touched since Vagrant likes to reload its mounts/shared folders on each load.

I hope this helps!