Why is it impossible to Applicative-traverse arrays? (Or is it?)

Question

While pondering how to best map, i.e. traverse, an a -> Maybe a-Kleisli over an unboxed vector, I looked for an existing implementation. Obviously U.Vector is not Traversable, but it does supply a mapM, which for Maybe of course works just fine.

But the question is: is the Monad constraint really needed? Well, it turns out that even boxed vectors cheat for the Traversable instance: they really just traverse a list, which they convert from/to:

instance Traversable.Traversable Vector where
  {-# INLINE traverse #-}
  traverse f xs = Data.Vector.fromList Applicative.<$> Traversable.traverse f (toList xs)

mono-traversable does the same thing also for unboxed vectors; here this seems even more gruesome performance-wise.

Now, I wouldn't be surprised if vector was actually able to fuse many of these hacked traversals into a far more efficient form, but still – there seems to be a fundamental problem, preventing us from implementing a traversal on an array right away. Is there any “deep reason” for this inability?

Answer 1

After reading through the relevant source of vector and trying to make mapM work with Applicative I think the reason why Data.Vector.Unboxed.Vector doesn't have a traverse :: (Applicative f, Unbox a, Unbox b) -> (a -> f b) -> Vector a -> f (Vector b) function and Data.Vector.Vector doesn't have a native traverse is the fusion code. The offender is the following Stream type:

-- Data/Vector/Fusion/Stream/Monadic.hs  Line: 137

-- | Result of taking a single step in a stream
data Step s a where
  Yield :: a -> s -> Step s a
  Skip  :: s -> Step s a
  Done  :: Step s a

-- | Monadic streams
data Stream m a = forall s. Stream (s -> m (Step s a)) s

This is used internally to implement mapM. The m will be the same as from your initial call to Data.Vector.Unboxed.mapM. But because the spine of this stream is inside the m functor, it is not possible to work with it if you only have an applicative for m.

See also this issue on the vector GitHub repo: Weaken constraint on mapM.

Disclaimer: I don't really know how fusion works. I don't know how vector works.