Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Too many open files error while running awk command

Tags:

linux

shell

awk

I am trying to split a big file based on a pattern. I am using awk command for this. After creating certain number of files it gives error: Too many open files.

Command is:

awk '/pattern here/{i++}{print > "file"i}' /input file

Can someone tell me how to close these files? I tried following but it gives error.

    awk '/pattern here/{i++}{print > "file"i}' /input file | close("file"i)
like image 900
Keval Shah Avatar asked Oct 01 '15 01:10

Keval Shah


People also ask

What causes too many open files error?

The "Too many open files" message means that the operating system has reached the maximum "open files" limit and will not allow SecureTransport, or any other running applications to open any more files. The open file limit can be viewed with the ulimit command: The ulimit -aS command displays the current limit.

What is awk '{ print $2 }'?

awk '{ print $2; }' prints the second field of each line. This field happens to be the process ID from the ps aux output. xargs kill -${2:-'TERM'} takes the process IDs from the selected sidekiq processes and feeds them as arguments to a kill command.


1 Answers

Before starting on the next file, close the previous one:

    awk '/pattern here/{close("file"i); i++}{print > "file"i}' InputFile
like image 169
John1024 Avatar answered Sep 18 '22 17:09

John1024