sed: Argument list too long

Question

I have created a script in Unix environment. In the script, I used the sed command as shown below to delete some lines from the file. I want to delete a specified set of lines, not necessarily a simple range, from the file, specified by line numbers.

sed -i "101d; 102d; ... 4930d;" <file_name>

When I execute this it shows the following error:

sed: Arg is too long

Can you please help to resolve this problem?

Answer 1

If you want to delete a contiguous range of lines, you can specify a range of line numbers:

sed -i '101,4930d' file

If you want to delete some arbitrary set of lines that can't easily be expressed as a range, you can put the commands in a file rather than on the command line, and use sed -f.

For example, if foo.sed contains:

2d
4d
6d
8d
10d

then this:

sed -i -f foo.sed file

will delete lines 2, 4, 6, 8, and 10 from file. Putting the commands in a file rather than on the command line avoids limits on command line length.

If there's some pattern to the lines you want to delete, you might consider using a more sophisticated tool such as Awk or Perl.

Answer 2

I had this exact same problem.

I originally put the giant sed command sed -i "101d; 102d; ... 4930d;" <file_name> in a file and tried to execute as a bash script.

To fix - put only the deletion commands in a file and run that file as a sed script. I was able to execute 18,193 deletion commands that had failed to run before.

sed -i -f to_delete.sed input_file

to_delete.sed:

101d;102d;...4930d