Working with huge files in VIM

Question

I tried opening a huge (~2GB) file in VIM but it choked. I don't actually need to edit the file, just jump around efficiently.

How can I go about working with very large files in VIM?

Answer 1

I had a 12GB file to edit today. The vim LargeFile plugin did not work for me. It still used up all my memory and then printed an error message :-(. I could not use hexedit for either, as it cannot insert anything, just overwrite. Here is an alternative approach:

You split the file, edit the parts and then recombine it. You still need twice the disk space though.

• Grep for something surrounding the line you would like to edit:

  grep -n 'something' HUGEFILE | head -n 1 

• Extract that range of the file. Say the lines you want to edit are at line 4 and 5. Then do:

  sed -n -e '4,5p' -e '5q' HUGEFILE > SMALLPART 

  • The -n option is required to suppress the default behaviour of sed to print everything
  • 4,5p prints lines 4 and 5
  • 5q aborts sed after processing line 5

• Edit SMALLPART using your favourite editor.

• Combine the file:

  (head -n 3 HUGEFILE; cat SMALLPART; sed -e '1,5d' HUGEFILE) > HUGEFILE.new  

  • i.e: pick all the lines before the edited lines from the HUGEFILE (which in this case is the top 3 lines), combine it with the edited lines (in this case lines 4 and 5) and use this combined set of lines to replace the equivalent (in this case the top 5 lines) in the HUGEFILE and write it all to a new file.

  HUGEFILE.new will now be your edited file, you can delete the original HUGEFILE.