menu

Node.js

A Spectrum community for Node.js ✨🐢🚀✨

Channels
Team

Caching streaming HTML?

February 2, 2018 at 2:11pm

Caching streaming HTML?

February 2, 2018 at 2:11pm (Edited 1 year ago)

I'm working on moving Spectrum to streaming server-side rendering to help with backpressure on our SSR server. Unfortunately, as I was building it I realized that it completely broke our current Redis caching mechanism, which relies on monkey-patching res.send—but res.send isn't called when streaming a response.

This is what our streaming code looks like (very simplified pseudo code):

// Write the <head> and root <div>
res.write('<html><head>' + metaTags + '</head><body><div id="root>')
// Render the frontend React app
const stream = renderToNodeStream(<Frontend />)
// Pipe that HTML to the response
stream.pipe(res, { end: false });
// When React is finished, clean up the dangling HTML tags
stream.on('end', () => res.end('</div></body></html>'))

Now the question is, how do I cache the generated HTML from that?


February 2, 2018 at 2:12pm

Currently starting to read https://github.com/substack/stream-handbook, maybe I just need a deeper understanding of streams

  • reply
  • like

According to @xander76 on Twitter (https://twitter.com/xander76/status/959429678372188160) the code would have to use a Transform stream, which looks something like this:

  • reply
  • like
let cacheEntry = "";
renderToNodeStream(<Frontend/>)
.pipe(new Transform({
transform(chunk, enc, callback) {
cacheEntry += chunk; callback(chunk);
},
flush(callback) {
redis.set(req.path, cacheEntry);
}
})
.pipe(res);
like-fill
1
  • reply
  • like

Going to try that right away!

like-fill
1
  • reply
  • like

Apparently that specific code will have UTF-8 issues according to resident Node wizard @mafintosh, awaiting further information before proceeding

like-fill
1
  • reply
  • like

February 7, 2018 at 7:30pm

Thanks for the blog post Max!!

  • reply
  • like

Very welcome, glad you liked it!

  • reply
  • like
  • reply
  • like