Rendering Partial Markdown from a Token Stream

Part of Streaming AI Responses in the Browser, under Frontend Consumption & Client Patterns.

Markdown halfway through being written is usually invalid markdown, and feeding it to a renderer on every token produces a document that flickers between interpretations. The fix is to render a repaired copy, and never to repair the buffer itself.

Symptom & Developer Intent

  • Text flips between a paragraph and a code block as an opening fence is written and later closed.
  • A partially-typed link renders as literal [text](htt and then jumps into place.
  • Half-written tables render as pipe characters, then rearrange into a table.
  • The page stutters on long answers, with profiles showing time in the markdown parser.
  • Model-generated HTML executes, or a sanitiser strips content the model legitimately produced.

The intent is a stable, safe rendering of an answer that is still arriving.

Root Cause Analysis

Markdown is block-structured and its constructs are delimited. Mid-generation, an opening delimiter has arrived and its closing one has not, so the parser makes the only interpretation available to it — which changes the moment the closing delimiter appears. Every such change is a visible re-layout.

What each unterminated construct renders as Table of markdown constructs mid-generation with what a parser makes of them and the repair that stabilises the render. What each unterminated construct renders as Renders as Repair Unclosed ``` fence swallows the rest append a closing fence Half-written [link]( literal brackets hide until complete Odd number of ` emphasis flips trim the trailing run Table, first row only pipes as text wait for the delimiter
Each row is a visible flicker: the parser has to choose an interpretation, and the choice changes the moment the closing delimiter arrives.

Three specific constructs cause nearly all of the flicker. An unclosed code fence swallows everything after it into a code block. A partial link or image renders as literal brackets until the closing parenthesis arrives. Unbalanced inline markers — a single backtick, an unclosed ** — flip emphasis on and off across the tail of the document.

Tables and lists add a second-order version of the same problem. A table needs its header row and its delimiter row before it can be recognised as a table at all, so the first two lines render as ordinary paragraph text with visible pipe characters and then rearrange. Lists behave better because each item is independently valid, but a nested list re-indents as the deeper items arrive.

The performance problem is separate and compounding: re-parsing the entire document on every token is O(n) work per token, so cost grows quadratically with answer length. A four-kilobyte answer re-parsed eighty times a second is where the frame budget goes. Worse, the cost is concentrated exactly when the answer is longest and the user is most engaged with it.

The safety problem is independent of both. Model output can contain HTML, and rendering it into innerHTML executes it. This is not hypothetical when part of the prompt comes from user input: a prompt-injection payload that reaches the model’s output reaches the DOM by the same path, and the fact that the text came from your own model is no guarantee at all about what it contains.

Step-by-Step Resolution

Repair a copy, keep the buffer pristine Pipeline from the raw buffer through a repaired copy to sanitised HTML, with the buffer itself never modified. Repair a copy, keep the buffer pristine the final render, after done, uses the raw buffer with no repair at all Raw buffer never modified Repaired copy close · trim · hide Markdown → HTML parse the copy Sanitise allow-list copy parse always
Mutating the buffer is what corrupts the final answer: the next delta appends after the closing fence you added.

Step 1 — Repair a copy, never the buffer 

// Return a renderable COPY. Mutating the buffer corrupts the final answer,
// because the next delta appends after whatever you added.
function repairForRender(md) {
  let out = md;

  // Unclosed fenced code block — close it so the remainder is not swallowed.
  const fences = (out.match(/^```/gm) || []).length;
  if (fences % 2 === 1) out += '\n```';

  // Trailing partial link or image — hide until the closing paren arrives.
  out = out.replace(/!?\[[^\]]*\]\([^)]*$/, '');
  // Trailing partial link label.
  out = out.replace(/!?\[[^\]]*$/, '');

  // Unbalanced inline code.
  if ((out.match(/`/g) || []).length % 2 === 1) out = out.replace(/`[^`]*$/, '');

  // Unbalanced strong / emphasis at the very end.
  if ((out.match(/\*\*/g) || []).length % 2 === 1) out = out.replace(/\*\*(?:(?!\*\*)[\s\S])*$/, '');

  return out;
}

Step 2 — Sanitise every rendered frame 

// Model output is untrusted input. Sanitise on the way to the DOM, every time.
function renderMarkdown(el, raw) {
  const html = markdownToHtml(repairForRender(raw));
  el.innerHTML = sanitize(html, {
    ALLOWED_TAGS: ['p', 'br', 'strong', 'em', 'code', 'pre', 'a', 'ul', 'ol', 'li',
                   'h1', 'h2', 'h3', 'h4', 'blockquote', 'table', 'thead', 'tbody',
                   'tr', 'th', 'td', 'hr'],
    ALLOWED_ATTR: ['href', 'title', 'class'],
  });
}

Step 3 — Split completed blocks from the live tail

This is what keeps long answers fast: parse finished blocks once, and re-parse only the trailing block on each frame.

// Everything before the last blank line is settled; only the tail moves.
function splitSettled(md) {
  const cut = md.lastIndexOf('\n\n');
  if (cut === -1) return { settled: '', tail: md };
  return { settled: md.slice(0, cut), tail: md.slice(cut + 2) };
}

function createIncrementalRenderer(container) {
  const settledEl = document.createElement('div');
  const tailEl = document.createElement('div');
  container.append(settledEl, tailEl);

  let renderedSettled = '';

  return (buffer) => {
    const { settled, tail } = splitSettled(buffer);
    if (settled !== renderedSettled) {
      renderMarkdown(settledEl, settled);        // re-parsed only when a block completes
      renderedSettled = settled;
    }
    renderMarkdown(tailEl, tail);                // small, cheap, re-parsed per frame
  };
}

Step 4 — Coalesce onto the animation frame 

const render = createIncrementalRenderer(document.getElementById('answer'));

let buffer = '';
let queued = false;
es.addEventListener('delta', (e) => {
  buffer += JSON.parse(e.data).text;
  if (queued) return;
  queued = true;
  requestAnimationFrame(() => { queued = false; render(buffer); });
});

Step 5 — Render the final answer once, unrepaired

When the generation completes the text is valid markdown and needs no repair — so render it once more from the raw buffer, which also removes any artefact the repair introduced.

es.addEventListener('done', () => {
  es.close();
  renderMarkdown(document.getElementById('answer'), buffer);   // raw, complete, final
});

Validation & Monitoring

Test with inputs designed to be invalid at every intermediate length.

Markdown parse cost per frame as the answer grows Bar chart comparing per-frame parse cost when re-parsing the whole document against parsing only the trailing block, at four answer lengths. Markdown parse cost per frame as the answer grows per rendered frame, whole document versus trailing block only 1 KB, whole doc 5 ms 4 KB, whole doc 20 ms 16 KB, whole doc 80 ms 16 KB, tail only 3 ms parse cost per frame
Splitting settled blocks from the live tail converts a cost that grows with answer length into a constant one — which is what keeps long answers smooth.
// A deterministic replay harness: feed a known answer one character at a time
// and assert the render never throws and never contains a raw script tag.
const sample = "Here is code:\n\n```js\nconst a = 1;\n```\n\nAnd a [link](https://example.com).";

for (let i = 1; i <= sample.length; i++) {
  const partial = sample.slice(0, i);
  const html = markdownToHtml(repairForRender(partial));
  console.assert(!/<script/i.test(html), `script tag leaked at length ${i}`);
  console.assert(typeof html === 'string', `render failed at length ${i}`);
}

Character-by-character replay is the right granularity: it exercises every possible partial state, including the ones a token-boundary replay would skip.

For the performance side, profile a long answer and confirm markdown parsing stays flat rather than climbing with length. The block-splitting in step 3 is what produces that flat line; without it the parse cost per frame grows with the document.

// Cheap production signal: how long the render callback takes at the end of a long answer.
const t0 = performance.now();
render(buffer);
if (performance.now() - t0 > 8) beacon('ai_render_slow_ms', performance.now() - t0);

Production Checklist

Frequently Asked Questions

Why not just wait for the whole answer and render once?

Because incremental rendering is the feature — a user watching text appear perceives the system as far more responsive than one waiting on a spinner for the same total time. The goal is to render continuously and stably, which is what repairing a copy achieves.

Is sanitising really necessary when the text comes from my own model?

Yes. Model output is a function of its input, and some of that input is usually user-supplied — so it is untrusted by construction. Sanitise every frame with an explicit allow-list; the cost is negligible next to the parse.

Why does my final answer contain an extra closing fence?

Because the repair was applied to the buffer instead of to a copy. Once you append the closing fence to the real buffer, the next delta lands after it and the document is corrupt. Repair only the value you hand to the renderer.

How do I stop long answers from stuttering?

Stop re-parsing the whole document each frame. Split at the last blank line: render everything before it once when a new block completes, and re-parse only the trailing block on each frame. That converts a cost that grows with answer length into a constant one.