menu
announcement

Spectrum will become read-only on August 10, 2021. Learn more about the decision in our official announcement.

Node.js

A Spectrum community for Node.js ✨🐢🚀✨

Channels
# All channels
view-forward
# general
view-forward
# community
view-forward
# help
view-forward
# jobs
view-forward
# projects
view-forward
Team

Caching streaming HTML?

February 2, 2018 at 2:11pm

Caching streaming HTML?

February 2, 2018 at 2:11pm (Edited 3 years ago)
I'm working on moving Spectrum to streaming server-side rendering to help with backpressure on our SSR server. Unfortunately, as I was building it I realized that it completely broke our current Redis caching mechanism, which relies on monkey-patching res.send—but res.send isn't called when streaming a response.
This is what our streaming code looks like (very simplified pseudo code):
// Write the <head> and root <div>
res.write('<html><head>' + metaTags + '</head><body><div id="root>')
// Render the frontend React app
const stream = renderToNodeStream(<Frontend />)
// Pipe that HTML to the response
stream.pipe(res, { end: false });
// When React is finished, clean up the dangling HTML tags
stream.on('end', () => res.end('</div></body></html>'))
Now the question is, how do I cache the generated HTML from that?

February 2, 2018 at 2:12pm
Currently starting to read https://github.com/substack/stream-handbook, maybe I just need a deeper understanding of streams
  • reply
  • like
According to on Twitter (https://twitter.com/xander76/status/959429678372188160) the code would have to use a Transform stream, which looks something like this:
  • reply
  • like
let cacheEntry = "";
renderToNodeStream(<Frontend/>)
.pipe(new Transform({
transform(chunk, enc, callback) {
cacheEntry += chunk; callback(chunk);
},
flush(callback) {
redis.set(req.path, cacheEntry);
}
})
.pipe(res);
like-fill
1
  • reply
  • like
Going to try that right away!
like-fill
1
  • reply
  • like
Apparently that specific code will have UTF-8 issues according to resident Node wizard , awaiting further information before proceeding
like-fill
1
  • reply
  • like

February 7, 2018 at 7:30pm
Thanks for the blog post Max!!
  • reply
  • like
Very welcome, glad you liked it!
  • reply
  • like
  • reply
  • like