Python - Calculating the second column from the first one in a file

Question

I am a beginner in Python and cannot cope with one of the moments of my project, so I would be glad you to help me:)

Lets's imagine, I have a *.txt file with only one column which looks like:

         Column-1

row-1    0
row-2    25.00
row-3    27.14
row-4    29.29
row-5    31.43
row-6    33.57

*Column with rows is added here just to simplify the explanation.

I need to calculate the second column which takes as input data from the Column-1 and outputs the result of subtraction of this row value from the previous one (if the row-1(Column-1) value is 0, than it should be 0 in the row-1(Column-2) as well). It should be like:

• row-2(Column-2) = row-2(Column-1) - row-1(Column-1)
• row-3(Column-2) = row-3(Column-1) - row-2(Column-1) and so on.

Let me show you how the output file shoud be like:

         Column-1  Column-2

row-1    0         0
row-2    25.00     25.00
row-3    27.14     2.14
row-4    29.29     2.15
row-5    31.50     2.21
row-6    33.57     2.07

For now I am only here with my programming:

import sys

with open('file.txt', "r") as f:
    sys.stdout = open('%s (calc).txt' % f.name, 'a')
    for line in f:
        column = line.strip().split()
        Column_1 = float(column[0])
        column.extend([None])

I wonder what to do next. Maybe, numpy is right for this task? I am not strong in it (basically, I don't know it at all), should I learn it?

Anyway, I would be really thankful for you contribution.

Answer 1

I believe this would do what you asked:

INPUT = 'file.txt'
OUTPUT = 'calc.txt'

def main():
    with open(INPUT, 'r') as reader, open(OUTPUT, 'a') as writer:
        last_value = 0
        for line in reader:
            column_1, *remaining_columns = map(float, line.split())
            column_2 = column_1 - last_value
            last_value = column_1
            print(column_1, column_2, sep='\t', file=writer)

if __name__ == '__main__':
    main()

Answer 2

One approach could be the following: Say you have two lists:

a = [1,2,3,0]
b = [0,1,2,3]

You can then subtract one list from the other using the following step:

import operator
map(operator.sub, a, b)

To do this, you'd need to read in your file as an array (using array.append(value) to get all the data in.

Then make a copy of your data, and offset it by one (the list length needs to be the same). How you handle the beginning and end of the array depends on how important these values are to you (maybe you can afford lose one value).

Answer 3

Here is a solution using list comprehensions and a zip:

#!/usr/bin/env python3

with open('file.txt', "r") as f:

    # read column one into a list
    column_1 = [float(line.strip()) for line in f]
    # compute differences of neighbouring values
    column_2 = [now - last for now, last in zip(column_1, [0.0]+column_1[:-1])]

    with open("result.txt", "w") as outfile:
        for c1, c2 in zip(column_1, column_2):
            print("{:.2f}\t{:.2f}".format(c1, c2), file=outfile)

What's happening here is we first create a list of all entries from the input file. By using zip we can create tuples from two (or more) iterables. We have to create a second list with the values shifted by 1 and adding a 0.0 at the front to act as a sentinel for the first entry.

Now we can zip these two lists together and compute the differences between the value pairs using the second list comprehension.

Learning numpy is always a good idea, but I think it would be overkill for this task.