Bash: Trying to append to a variable name in the output of a function

Question

this is my very first post on Stackoverflow, and I should probably point out that I am EXTREMELY new to a lot of programming. I'm currently a postgraduate student doing projects involving a lot of coding in various programs, everything from LaTeX to bash, MATLAB etc etc.

If you could explicitly explain your answers that would be much appreciated as I'm trying to learn as I go. I apologise if there is an answer else where that does what I'm trying to do, but I have spent a couple of days looking now.

So to the problem I'm trying to solve: I'm currently using a selection of bioinformatics tools to analyse a range of genomes, and I'm trying to somewhat automate the process.

I have a few sequences with names that look like this for instance (all contained in folders of their own currently as paired files):

SOL2511_S5_L001_R1_001.fastq
SOL2511_S5_L001_R2_001.fastq
SOL2510_S4_L001_R1_001.fastq
SOL2510_S4_L001_R2_001.fastq

...and so on...

I basically wish to automate the process by turning these in to variables and passing these variables to each of the programs I use in turn. So for example my idea thus far was to assign them as wildcards, using the R1 and R2 (which appears in all the file names, as they represent each strand of DNA) as follows:

#!/bin/bash

seq1=*R1_001*
seq2=*R2_001*

On a rudimentary level this works, as it returns the correct files, so now I pass these variables to my first function which trims the DNA sequences down by a specified amount, like so:

# seqtk is the program suite, trimfq is a function within it, 
# and the options -b -e specify how many bases to trim from the beginning and end of     
# the DNA sequence respectively. 

seqtk trimfq -b 10 -e 20 $seq1 > 
seqtk trimfq -b 10 -e 20 $seq2 > 

So now my problem is I wish to be able to append something like "_trim" to the output file which appears after the >, but I can't find anything that seems like it will work online.

Alternatively, I've been hunting for a script that will take the name of the folder that the files are in, and create a variable for the folder name which I can then give to the functions in question so that all the output files are named correctly for use later on.

Many thanks in advance for any help, and I apologise that this isn't really much of a minimum working example to go on, as I'm only just getting going on all this stuff!

Joe

EDIT

So I modified @ghoti 's for loop (does the job wonderfully I might add, rep for you :D ) and now I append trim_, as the loop as it was before ended up giving me a .fastq.trim which will cause errors later.

Is there any way I can append _trim to the end of the filename, but before the extension?

Answer 1

Explicit is usually better than implied, when matching filenames. Your wildcards may match more than you expect, especially if you have versions of the files with "_trim" appended to the end!

I would be more precise with the wildcards, and use for loops to process the files instead of relying on seqtk to handle multiple files. That way, you can do your own processing on the filenames.

Here's an example:

#!/bin/bash

# Define an array of sequences
sequences=(R1_001 R2_001)

# Step through the array...
for seq in ${sequences[@]}; do

  # Step through the files in this sequence...
  for file in SOL*_${seq}.fastq; do
    seqtk trimfq -b 10 -e 20 "$file" > "${file}.trim"
  done

done

I don't know how your folders are set up, so I haven't addressed that in this script. But the basic idea is that if you want the script to be able to manipulate individual filenames, you need something like a for loop to handle the that manipulation on a per-filename basis.

Does this help?

UPDATE:

To put _trim before the extension, replace the seqtk line with the following:

    seqtk trimfq -b 10 -e 20 "$file" > "${file%.fastq}_trim.fastq"

This uses something documented in the Bash man page under Parameter Expansion if you want to read up on it. Basically, the ${file%.fastq} takes the $file variable and strips off a suffix. Then we add your extra text, along with the suffix.

You could also strip an extension using basename(1), but there's no need to call something external when you can use something built in to the shell.