Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to clean a codebase, trailing whitespace, new lines etc

I have a code base that is driving me nuts with conflicts due to trailing whitespace. I'd like to clean it up.

I'd want to:

  • Remove all trailing whitespace
  • Remove any newline characters at the end of files
  • Convert all line endings to unix (dos2unix)
  • Convert all leading spaces to tabs, ie 4 spaces to tabs.

  • While ignoring the .git directory.

I'm on OSX Snow Leopard, and in zsh.

so far, i have:

sed -i "" 's/[ \t]*$//' **/*(.)

which works great, but sed adds a new line to the end of every file it touches, which is no good. I dont think sed can be stopped from doing this, so how can i remove these new lines? Theres probably some awk magic to be applied here.

(Complete answers also welcome)

like image 635
jhogendorn Avatar asked Feb 16 '11 01:02

jhogendorn


People also ask

How do you solve trailing whitespace?

Type M-x delete-trailing-whitespace to delete all trailing whitespace. This command deletes all extra spaces at the end of each line in the buffer, and all empty lines at the end of the buffer; to ignore the latter, change the variable delete-trailing-lines to nil .

How do I delete a trailing space in Vim?

One way to make sure to remove all trailing whitespace in a file is to set an autocmd in your . vimrc file. Every time the user issues a :w command, Vim will automatically remove all trailing whitespace before saving.

What is trailing whitespace in code?

Trailing whitespace. Description: Used when there is whitespace between the end of a line and the newline.


1 Answers

[EDIT: Fixed whitespace trimming]
[EDIT #2: Strip trailing blank lines from end of file]

perl -i.bak -pe 'if (defined $x && /\S/) { print $x; $x = ""; } $x .= "\n" x chomp; s/\s*?$//; 1 while s/^(\t*)    /$1\t/; if (eof) { print "\n"; $x = ""; }' **/*(.)

This strips trailing blank lines from the file, but leaves exactly one \n at the end of the file. Most tools expect this, and it will not show up as a blank line in most editors. However if you do want to strip that very last \n, just delete the print "\n"; part from the command.

The command works by "saving up" \n characters until a line containing a non-blank character is seen -- then it prints them all before processing that line.

Remove .bak to avoid creating backups of the original files (use at your own risk!)

\s*? matches zero or more whitespace characters non-greedily, including \r, which is the first character of the \r\n DOS line-break syntax. In Perl, $ matches either at the end of the line, or immediately before a final \n, so combined with the fact that *? matches non-greedily (trying a 0-width match first, then a 1-width match and so on) it does the right thing.

1 while s/^(\t*) /$1\t/ is just a loop that repeatedly replaces any lines beginning with any number of tabs followed by 4 spaces with one more tab than there was, until this is no longer possible. So it will work even if some lines have been partially converted to tabs already, provided all \t characters start at a column divisible by 4.

I haven't seen the **/*(.) syntax before, presumably that's a zsh extension? If it worked with sed, it will work with perl.

like image 181
j_random_hacker Avatar answered Oct 01 '22 07:10

j_random_hacker