Heroku Slug Size After Multiple Deployments

Question

I have a Ruby on Rails application that was very close to the slug size limit (300mb). I have reduced the size as much as possible with .slugignore but haven't got the results I'd like.

On a whim I tried creating a new Heroku application and deployed the same git repository to it. Lo and behold it comes in well below (only 80mb vs 208mb). When accessing bash and checking the size on each server I noticed a few differences between the servers:

du -hs *
130M (old) vs 1.4M (new) public
23M (old) vs 5.3M (new) tmp
...

What gives? On Heroku are user's expected to destroy and recreate applications after enough deployments pass? I looked through the public directory and it is keeping all sorts of old removed assets and junk. How can I cleanup my existing app without destroy it and starting from scratch?

Answer 1

This is probably somewhat related to Rail's attempt to avoid problems during deploys. To goal is to avoid downtime and/or errors during deploy. Your code isn't deployed instantly to all servers. Rails solution is to allow multiple versions of the assets to exist simultaneously. The typically process is:

%> rake assets:precompile  # To build new assets
%> rake assets:clean       # Remove old assets

Rails keeps 2 previous versions of the assets (by default). Depending on the application you may need old assets for a while. For example if you have a heavy JavaScript application that fetches resources once then is dynamically updated through XHR. The old code running in the client may refer to other old resources. Even with a non-client heavy app you may have several seconds where some nodes are referring to old assets and some are referring to new assets.

This would explain some of the reason public is larger on your old app. It essentially has three versions of all your assets while your new deploy only has one. It sounds like something must be messed up with your build environment though because even if all your assets occupy the 1.4MB in your new app, three times that size should only be about 5MB and not the 130MB in your old app.

In addition to the have three versions of your assets, there is also some accumulation in the tmp directory. The assets compiling process caches some information in the tmp directory. Rails has a rake tmp:cache:clear that you can periodically run to git rid of cruft there.

Heroku automatically runs rake assets:clean. So that should keep it to just three versions of the assets. But Heroku doesn't actually run rake tmp:cache:clear. Instead they have some custom code that deletes cache files until the cached data is under 50MB. I'm assuming they do this to keep as much cache info as possible while still capping things. Keeping as much cached data probably ensures asset compilation runs fast. This means your tmp directory will continue to grow until it reaches 50MB.

If your public/assets directory is growing beyond just 3 versions or 50MB in your tmp directory is too much then creating a new app is a decent way to clear things out. You could also create a custom slug. Just running rake assets:clean or rake tmp:cache:clear on a heroku console won't work. That will just clean things on a dyno and not your slug. So your cleaning will be thrown away when a new dyno is created.