Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using awk to process a database

I have a directory on my computer which contains an entire database I found online for my research. This database contains thousands of files, so to do what I need I've been looking into file i/o stuff. A programmer friend suggested using bash/awk. I've written my code:

    #!/usr/bin/env awk
    ls -l|awk'
    BEGIN {print "Now running"}
    {if(NR == 17 / $1 >= 0.4 / $1 <= 2.5)
    {print $1 > wavelengths.txt;
    print $2 > reflectance.txt;
    print $3 > standardDev.txt;}}END{print "done"}'

When I put this into my console, I'm already in the directory of the files I need to access. The data I need begins on line 17 of EVERY file. The data looks like this:

some number    some number    some number
some number    some number    some number
    .              .              .
    .              .              .
    .              .              .

I want to access the data when the first column has a value of 0.4 (or approximately) and get the information up until the first column has a value of approximately 2.5. The first column represents wavelengths. I want to verify they are all the same for each file later, so I copy them into a file. The second column represents reflectance and I want this to be a separate file because later I'll take this information and build a data matrix from it. And the third column is the standard deviation of the reflectance.

The problem I am having now is that when I run this code, I get the following error: No such file or directory

Please, if anyone can tell me why I might be getting this error, or can guide me as to how to write the code for what I am trying to do... I will be so grateful.

like image 952
user1723196 Avatar asked Sep 19 '26 11:09

user1723196


1 Answers

The main problem is that you need to quote the names of the output file names as they are strings not variables. Use:

print $1 > "wavelengths.txt"

instead of:

print $1 > wavelengths.txt
like image 95
Ed Morton Avatar answered Sep 21 '26 00:09

Ed Morton