How can I append DOM elements as they stream in from the network?

Question

I have an element that I want to populate as a request's HTML streams in, instead of waiting for the complete response. This turned out to be incredibly difficult.

Things I've tried:

1. milestoneNode.insertAdjacentHTML('beforebegin', text)

Be lovely if this worked. Unfortunately, elements with quirky parsing wreck it — such as <p> and <table>. The resulting DOM can be charitably described as Dada.

2. Using a virtual DOM/DOM update management library

Google's incremental-dom seemed most promising, but its patch() operation always restarts from the beginning of the container node. Not sure how to "freeze" it in place.

This also has baggage from doing at least HTML tokenization in JavaScript, and some actual tree-building would have to happen, unless one serves well-formed XHTML5. (Nobody does.) Reimplementing a browser's HTML parser seems like a sign I've gone horribly wrong.

3. document.write()

I was desperate. Ironically, this ancient boogeyman has almost the behavior I need, sans the "throwing away the existing page" thing.

4. Appending to a string, then innerHTMLing it periodically

Defeats the point of streaming, since eventually the entire response gets held in memory. Also has repeated serialization overhead.

On the plus side, it actually works. But surely there's a better way?

Answer 1

Jake Archibald figured out a silly hack to get this behavior in browsers today. His example code says it better than I would:

// Create an iframe:
const iframe = document.createElement('iframe');

// Put it in the document (but hidden):
iframe.style.display = 'none';
document.body.appendChild(iframe);

// Wait for the iframe to be ready:
iframe.onload = () => {
  // Ignore further load events:
  iframe.onload = null;

  // Write a dummy tag:
  iframe.contentDocument.write('<streaming-element>');

  // Get a reference to that element:
  const streamingElement = iframe.contentDocument.querySelector('streaming-element');

  // Pull it out of the iframe & into the parent document:
  document.body.appendChild(streamingElement);

  // Write some more content - this should be done async:
  iframe.contentDocument.write('<p>Hello!</p>');

  // Keep writing content like above, and then when we're done:
  iframe.contentDocument.write('</streaming-element>');
  iframe.contentDocument.close();
};

// Initialise the iframe
iframe.src = '';

Although <p>Hello!</p> is written to the iframe, it appears in the parent document! This is because the parser maintains a stack of open elements, which newly created elements are inserted into. It doesn't matter that we moved <streaming-element>, it just works.