Why does Stream provide convenience methods on an extension trait instead of the trait itself?

Question

Consider the Iterator trait from the standard library:

pub trait Iterator {
    type Item;

    // required
    pub fn next(&mut self) -> Option<Self::Item>;

    // potentially advantageous to override
    pub fn size_hint(&self) -> (usize, Option<usize>) { ... }
    pub fn count(self) -> usize { ... }
    pub fn last(self) -> Option<Self::Item> { ... }
    pub fn advance_by(&mut self, n: usize) -> Result<(), usize> { ... }
    pub fn nth(&mut self, n: usize) -> Option<Self::Item> { ... }

    // convenience
    pub fn step_by(self, step: usize) -> StepBy<Self> { ... }
    pub fn chain<U>(self, other: U) -> Chain<Self, U::IntoIter> { ... }
    pub fn zip<U>(self, other: U) -> Zip<Self, U>::IntoIter> { ... }
    pub fn map<B, F>(self, f: F) -> Map<Self, F> { ... }
    pub fn for_each<F>(self, f: F) { ... }
    ...
}

And consider the Stream and StreamExt traits from the futures crate:

pub trait Stream {
    type Item;

    // required
    fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>>;

    // potentially advantageous to override
    fn size_hint(&self) -> (usize, Option<usize>) { ... }
}

pub trait StreamExt: Stream {
    // convenience
    pub fn next(&mut self) -> Next<'_, Self> { ... }
    pub fn into_future(self) -> StreamFuture<Self> { ... }
    pub fn map<T, F>(self, f: F) -> Map<Self, F> { ... }
    pub fn enumerate(self) -> Enumerate<Self> { ... }
    pub fn filter<Fut, F>(self, f: F) -> Filter<Self, Fut, F> { ... }
    ...
}

impl<T> StreamExt for T where T: Stream { ... }

They have a lot of similarities, given that Stream is essentially an async version of Iterator. However, I'd like to draw attention to their differences.

Why are they structured differently?

The only benefit I see of splitting a trait is that the StreamExt methods can't be overridden. This way they are guaranteed to behave as intended, whereas the convenience methods for Iterator could be overridden to behave inconsistently. However, I can't imagine this being a common issue to consider guarding against it. And this difference comes at the cost of accessibility and discoverability, requiring users to import StreamExt to use them and to know they exist in the first place.

Given that Stream came after Iterator it is obvious the split was a deliberate decision, but what was the motivation? Surely its more than what I've thought up. Is there something bad about the Iterator design?

^{Extension traits are certainly required when provided by another crate but this question isn't about that.}

Answer 1

The biggest advantage of the split is that the trait implementing the convenience methods can be implemented in a different crate from the core method. This was important for the Future vs FutureExt trait, as it allowed the core methods of the Future trait to be moved into std without needing to standardize the FutureExt convenience methods.

This had two advantages: Firstly, Future could go into core as the core methods do not depend on having an allocator, whilst some of the convenience methods might. Secondly it reduced the surface area of standardization to minimize the cost of standardizing a feature that was a high priority in order to standardize async/await. Instead the convenience methods could be continued to be iterated on in the futures crate for now in FutureExt.

So why is Future vs FutureExt relevant for Stream vs StreamExt? Whilst firstly, Stream is an extension of Future, so there is an argument for just following the same pattern. But even more so, there is an expectation that at some point Stream will be standardized, potentially with some syntax sugar for working with async/await. By splitting now, the cost of migrating that core functionality into std/core is minimised. The long term plan appears to be that the core functionality will move from futures to std/core, whilst futures will become a more rapidly developed/lower risk location for extended functionality.

As a historical note, 0.1.x of futures used the iterator style of convenience methods for Future and Stream. This was changed in 0.2.x as part of the experimentation/iteration leading up to async/await.