How to loop over files in different directories

Question

I want to loop over multiple files and apply a function to them. The problem is that these files are all in different, but similarly named, directories. The pathway pattern is similar, but the number changes according to what family it's up to.

For example, I have code written as this:

for(i in 1:numfiles) {
    olddata <- read.table(paste("/home/smith/Family", i, "/Family", i, ".txt", sep="\t"),
                          header=T)      
    # FUNCTION GOES HERE

    write.table(newdata,
                paste("/home/smith/Family", i, "/Family", i, "test.txt",
                      sep = ",", quote=F, row.names=F)
}

The problem I have is that the family numbers don't go in numeric order. Some are labeled just with a number (ex: 2) and others have a letter appended to that number (ex: 1a)

In each family subdirectory (ie Family i) I want to call in the same file (the file name is exactly the same but with the number (i) changed according to what family it refers to). I want to loop over these particular files. For example... For family 1a the file is here: "/home/smith/Family1a/Family1a.txt" but for family 2 the file is here: "/home/smith/Family2/Family2.txt".

Also, R doesn't like my use of numfiles.

Answer 1

Have a look at ?list.files and ?dir, e.g.:

files <- list.files("/home/smith", pattern="Family[[:alnum:]]+.txt", recursive=TRUE, full.names=TRUE)

for (currentFile in files) {
  olddata <- read.table(currentFile, header=TRUE)
  ## some code
  write.table(newdata, file=sub(pattern=".txt$", replacement="test.txt", x=currentFile))
}

Or:

dirs <- dir("/home/smith", pattern="Family[[:alnum:]]+$")

fileName <- file.path(dirs, paste0(dirs, ".txt"))
testFileName <- file.path(dirs, paste0(dirs, "_test.txt"))

for (i in seq(along=fileName))

  olddata <- read.table(fileName[i], header=TRUE)
  ## some code
  write.table(newdata, file=testFileName[i])
}