Hidden Markov in PyMC3

Question

I have a multivariate Monte-Carlo Hidden Markov problem to solve:

   x[k] = f(x[k-1]) + B u[k]
   y[k] = g(x[k])

where:

x[k] the hidden states (Markov dynamics)
y[k] the observed data
u[k] the stochastic driving process

Is PyMC3 already mature enough to handle this problem or should I stay with version 2.3? Secondly, any references to HM models in a PyMC framework would be much appreciated. Thanks.

-- Henk

Answer 1

I did something similar with PyMC 2.x. My u was not time dependent though. Here is my example.

# we're using `some_tau` for the noise throughout the example.
# this should be replaced with something more meaningful.
some_tau = 1 / .5**2

# PRIORS
# we don't know too much about the velocity, might be pos. or neg. 
vel = pm.Normal("vel", mu=0, tau=some_tau)

# MODEL
# next_state = prev_state + vel (and some gaussian noise)
# That means that each state depends on the prev_state and the vel.
# We save the states in a list.
states = [pm.Normal("s0", mu=true_positions[0], tau=some_tau)]
for i in range(1, len(true_positions)):
    states.append(pm.Normal(name="s" + str(i),
                            mu=states[-1] + vel,
                            tau=some_tau))

# observation with gaussian noise
obs = pm.Normal("obs", mu=states, tau=some_tau, value=true_positions, observed=True)

I guess you need to model you vel as a list of RV. They prob have some dependence as well.

Here is the original question: PyMC: Parameter estimation in a Markov system

Here is the full example as IPython notebook: http://nbviewer.ipython.org/github/sotte/random_stuff/blob/master/PyMC%20-%20Simple%20Markov%20Chain.ipynb