Menu
I have an pages.txt file with 100 URLs inside. I want to check them one by one and fail on the first problem. This is what I'm doing:
cat pages.txt | xargs -n 1 curl --silent \ --output /dev/null --write-out '%{url_effective}: %{http_code}\n'; echo $? Exit code is 1, but I see it only when the entire file is done. How to stop earlier, on the first problem?
584
GNU parallel is an alternative to xargs that is designed to have the same options, but is line-oriented. Thus, using GNU Parallel instead, the above would work as expected.
xargs will run the first two commands in parallel, and then whenever one of them terminates, it will start another one, until the entire job is done. The same idea can be generalized to as many processors as you have handy. It also generalizes to other resources besides processors.
The xargs command is used in a UNIX shell to convert input from standard input into arguments to a command. In other words, through the use of xargs the output of a command is used as the input of another command.
Overview. xargs (short for “eXtended ARGuments”) is a command on Unix and most Unix-like operating systems. It converts input from standard input (STDIN) into arguments to a command.
xargs -n 1 sh -c '<your_command> $0 || exit 255' < input xargs -n 1 sh -c 'curl --silent --output /dev/null \ --write-out "%{url_effective}: %{http_code}\n" $0 || exit 255' < pages.txt For every URL in pages.txt, executes sh -c 'curl ... $0 || exit 255' one by one (-n 1) forcing to exit with 255 if the command fails.
From man xargs:
If any invocation of the command exits with a status of 255, xargs will stop immediately without reading any further input. An error message is issued on stderr when this happens.
69
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
