How do I implement an HTTP Response Filter to operate on the entire content at once, no chunking

Question

As mentioned in a couple other posts (see References below) I am attempting to create response filters in order to modify content being produced by another web application.

I have the basic string transformation logic working and encapsulated into Filters that derive from a common FilterBase. However, the logic must operate on the full content, not chunks of content. Therefore I need to cache the chunks as they are written and perform the filter when all the writes are completed.

As shown below I created a new ResponseFilter derived from MemoryStream. On Write, the content is cached to another MemoryStream. On Flush, the full content, now in the MemoryStream is converted to a string and the Filter logic kicks in. The modified content is then written back out to the originating stream.

However, on every second request (basically when a new Filter is instantiated over the previous one) the previous filter's Flush method is being executed. At this point the the application crashes on the _outputStream.Write() method as the _cachedStream is empty.

The order of event is as follows:

1. First Request
2. Write method is called
3. Flush method is called
4. Close method is called
5. Close method is called
6. At this point the app returns and the proper content is displayed.
7. Second Request
8. Flush method is called
9. Application crashes on _outputStream.Write. ArgumentOutOfRangeException (offset).
10. Continue through crash (w/ in Visual Studio)
11. Close method is called

There are a couple of questions I have:

1. Why is Close called twice?
2. Why is Flush called after Closed was called?
3. To Jay's point below, Flush may be called before the stream is completely read, where should the filter logic reside? In Close? In Flush but with "if closing"?
4. What is the proper implementation for a Response Filter that works on the entire content at once?

Note: I experience the exact same behavior (minus Close events) if I do not override the Close method.

public class ResponseFilter : MemoryStream
{
    private readonly Stream _outputStream;
    private MemoryStream _cachedStream = new MemoryStream(1024);

    private readonly FilterBase _filter;

    public ResponseFilter (Stream outputStream, FilterBase filter)
    {
        _outputStream = outputStream;
        _filter = filter;
    }

    // Flush is called on the second, fourth, and so on, page request (second request) with empty content.
    public override void Flush()
    {
        Encoding encoding = HttpContext.Current.Response.ContentEncoding;

        string cachedContent = encoding.GetString(_cachedStream.ToArray());

        // Filter the cached content
        cachedContent = _filter.Filter(cachedContent);

        byte[] buffer = encoding.GetBytes(cachedContent);
        _cachedStream = new MemoryStream();
        _cachedStream.Write(buffer, 0, buffer.Length);

        // Write new content to stream
        _outputStream.Write(_cachedStream.ToArray(), 0, (int)_cachedStream.Length);
        _cachedStream.SetLength(0);

        _outputStream.Flush();
    }

    // Write is called on the first, third, and so on, page request.
    public override void Write(byte[] buffer, int offset, int count)
    {
        // Cache the content.
        _cachedStream.Write(buffer, 0, count);
    }

    public override void Close()
    {
        _outputStream.Close();
    }
}

// Example usage in a custom HTTP Module on the BeginRequest event.
FilterBase transformFilter = new MapServiceJsonResponseFilter();
response.Filter = new ResponseFilter(response.Filter, transformFilter);

References:

• How do I deploy a managed HTTP Module Site Wide?
• Creating multiple (15+) HTTP Response filters, Inheritance vs. Composition w/ Injection

Answer 1

Thanks to a tip from Jay regarding Flush being called for incremental writes I have been able to make the Filter work as desired by performing the filtering logic only if the Filter is closing and has not yet closed. This ensures that the Filter only Flushes once when the Stream is closing. I accomplished this with a few simple fields, _isClosing and _isClosed as shown in the final code below.

public class ResponseFilter : MemoryStream
{
    private readonly Stream _outputStream;
    private MemoryStream _cachedStream = new MemoryStream(1024);

    private readonly FilterBase _filter;
    private bool _isClosing;
    private bool _isClosed;

    public ResponseFilter (Stream outputStream, FilterBase filter)
    {
        _outputStream = outputStream;
        _filter = filter;
    }

    public override void Flush()
    {
        if (_isClosing && !_isClosed)
        {
            Encoding encoding = HttpContext.Current.Response.ContentEncoding;

            string cachedContent = encoding.GetString(_cachedStream.ToArray());

            // Filter the cached content
            cachedContent = _filter.Filter(cachedContent);

            byte[] buffer = encoding.GetBytes(cachedContent);
            _cachedStream = new MemoryStream();
            _cachedStream.Write(buffer, 0, buffer.Length);

            // Write new content to stream
            _outputStream.Write(_cachedStream.ToArray(), 0, (int)_cachedStream.Length);
            _cachedStream.SetLength(0);

            _outputStream.Flush();
        }
    }

    public override void Write(byte[] buffer, int offset, int count)
    {
        _cachedStream.Write(buffer, 0, count);
    }

    public override void Close()
    {
        _isClosing = true;

        Flush();

        _isClosed = true;
        _isClosing = false;

        _outputStream.Close();
    }
}

I have not yet found answers to my other questions above so I will not mark this answer as excepted at this time.