Numpy time based vector operations where state of preceding elements matters - are for loops appropriate?

Question

What do numpy arrays provide when performing time based calculations where state matters. In other words, where what has occurred in earlier or later in a sequence is important.

Consider the following time based vectors,

TIME = np.array([0.,   10.,  20.,  30.,  40.,  50.,  60.,  70.,  80.,  90.])
FLOW = np.array([100., 75.,  60.,  20.0, 60.0, 50.0, 20.0, 30.0, 20.0, 10.0])
TEMP = np.array([300., 310., 305., 300., 310., 305., 310., 305., 300., 295.0])

Let's say that an exponential decay in TEMP should be applied once the FLOW falls below 30 without raising again above 50. In the data above, a function would be applied at TIME=60 above and the last two values of TEMP would be updated by this secondary function which would began with the corresponding TEMP value.

There is a need to "look ahead" to determine if the FLOW rises above 50 in the elements after the <30 condition is requested. It does not seem that the numpy functions are aimed at time based vectors where state is important and the traditional method of nested for loops perhaps remains the way to go. But given given my newness to numpy and the fact that I have to perform alot of these types of state based manipulations, I would appreciate direction or affirmation.

Answer 1

While Joe Kington's answer is certainly correct (and quite flexible), it is rather more circuitous than need be. For someone trying to learn Numpy, I think a more direct route might be easier to understand.

As I noted under your question (and as Joe also noticed), there seems to be an inconsistency between your description of the code's behaviour and your example. Like Joe, I'm also going to assume you describe the correct behaviour.

A couple notes:

1. Numpy works well with filter arrays to specify to which elements an operation should be applied. I make use of them several times.
2. The np.flatnonzero function returns an array of indices specifying the locations at which the given array is non-zero (or True).

The code uses the example arrays you provided.

import numpy as np

TIME = np.array([0.,   10.,  20.,  30.,  40.,  50.,  60.,  70.,  80.,  90.])
FLOW = np.array([100., 75.,  60.,  20.0, 60.0, 50.0, 20.0, 30.0, 20.0, 10.0])
TEMP = np.array([300., 310., 305., 300., 310., 305., 310., 305., 300., 295.0])

last_high_flow_index = np.flatnonzero(FLOW > 50)[-1]
low_flow_indices = np.flatnonzero(FLOW < 30)
acceptable_low_flow_indices = low_flow_indices[low_flow_indices > last_high_flow_index]
apply_after_index = acceptable_low_flow_indices[0]

We now have the index after which the function should be applied to TEMP. If I am reading your question correctly, you would like the temperature to begin decaying once your condition is met. This can be done as follows:

time_delta = TIME[apply_after_index:] - TIME[apply_after_index]
TEMP[apply_after_index:] = TEMP[apply_after_index:] * np.exp(-0.05 * time_delta)

TEMP has been updated, so print TEMP outputs

[ 300.          310.          305.          300.          310.          305.
  310.          184.99185121  110.36383235   65.82339724]

Alternatively, you can apply an arbitrary Python function to the appropriate elements by first vectorizing the function:

def myfunc(x):
    ''' a normal python function that acts on individual numbers'''
    return x + 3

myfunc_v = np.vectorize(myfunc)

and then updating the TEMP array:

TEMP[apply_after:] = myfunc_v(TEMP[apply_after:])