Redirecting multiline output to multiple files

Question

I have a list of URLs, and would like to identify what is a directory and what is not:

https://www.example.com/folder/
https://www.example.com/folder9/
https://www.example.com/folder/file.sh
https://www.example.com/folder/text

I can use grep -e /$ to find which is which, but I'd like to do an inline command where I can redirect the output based on that logic.

I understand that awk may have the answer here, but don't have enough experience in awk to do this.

Something like:

cat urls | if /$ matches write to folders.txt else write to files.txt

I could drop it all to a file then read it twice but when it gets to thousands of lines I feel that would be inefficient.

Answer 1

Yes, awk is a great choice for this:

awk '/\/$/ { print > "folders.txt"; next }
           { print > "files.txt" }' urls.txt

• /\/$/ { print > "folders.txt"; next } if the line ends with a /, write it to folders.txt and skip to the next line
• { print > "files.txt" } write all other lines to files.txt

You may want to use the expression /\/[[:space:]]*$/ instead of /\/$/ in case you have trailing spaces in your file.

Answer 2

All you need is:

awk '{print > ((/\/$/ ? "folders" : "files")".txt")}' urls.txt