{ "data": { "post": { "title": "Improving Performance with HTTP Streaming", "createdAt": 1684264387622, "creator": { "id": "e46fded15590", "name": "Victor" }, "content": { "bodyModel": { "paragraphs": [ { "name": "2f94", "text": "Improving Performance with HTTP Streaming", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "7a08", "text": "How HTTP Streaming can improve page performance and how Airbnb enabled it on an existing codebase", "type": "P", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "9dd1", "text": "By: Victor Lin", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://www.linkedin.com/in/victorhlin/", "userId": null, "start": 4, "end": 14, "anchorType": "LINK" }, { "title": null, "type": "STRONG", "href": null, "userId": null, "start": 0, "end": 3, "anchorType": null } ], "iframe": null, "metadata": null }, { "name": "3351", "text": "", "type": "IMG", "href": null, "layout": "INSET_CENTER", "markups": [], "iframe": null, "metadata": { "id": "1*q2A2ZjnULygCKIWuiSBKXg.jpeg", "originalWidth": 1440, "originalHeight": 960 } }, { "name": "715f", "text": "Introduction", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "a054", "text": "You may have heard a joke that the Internet is a series of tubes. In this blog post, we’re going to talk about how we get a cool, refreshing stream of Airbnb.com bytes into your browser as quickly as possible using HTTP Streaming.", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://en.wikipedia.org/wiki/Series_of_tubes", "userId": null, "start": 35, "end": 64, "anchorType": "LINK" } ], "iframe": null, "metadata": null }, { "name": "f9bf", "text": "Let’s first understand what streaming means. Imagine we had a spigot and two options:", "type": "P", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "7ad4", "text": "Fill a big cup, and then pour it all down the tube (the “buffered” strategy)", "type": "ULI", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "4438", "text": "Connect the spigot directly to the tube (the “streaming” strategy)", "type": "ULI", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "f095", "text": "In the buffered strategy, everything happens sequentially — our servers first generate the entire response into a buffer (filling the cup), and then more time is spent sending it over the network (pouring it down). The streaming strategy happens in parallel. We break the response into chunks, which are sent as soon as they are ready. The server can start working on the next chunk while previous chunks are still being sent, and the client (e.g, a browser) can begin handling the response before it has been fully received.", "type": "P", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "d6eb", "text": "Implementing Streaming at Airbnb", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "ca5b", "text": "Streaming has clear advantages, but most websites today still rely on a buffered approach to generate responses. One reason for this is the additional engineering effort required to break the page into independent chunks. This just isn’t feasible sometimes. For example, if all of the content on the page relies on a slow backend query, then we won’t be able to send anything until that query finishes.", "type": "P", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "806e", "text": "However, there’s one use case that’s universally applicable. We can use streaming to reduce network waterfalls. This term refers to when one network request triggers another, resulting in a cascading series of sequential requests. This is easily visualized in a tool like Chrome’s Waterfall:", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://developer.chrome.com/docs/devtools/network/reference/#waterfall", "userId": null, "start": 281, "end": 290, "anchorType": "LINK" }, { "title": null, "type": "STRONG", "href": null, "userId": null, "start": 92, "end": 110, "anchorType": null } ], "iframe": null, "metadata": null }, { "name": "77db", "text": "Chrome Network Waterfall illustrating a cascade of sequential requests", "type": "IMG", "href": null, "layout": "INSET_CENTER", "markups": [], "iframe": null, "metadata": { "id": "1*qhOyK4HxTnhImOTPhSA4DQ.png", "originalWidth": 1592, "originalHeight": 1062 } }, { "name": "bde8", "text": "Most web pages rely on external JavaScript and CSS files linked within the HTML, resulting in a network waterfall — downloading the HTML triggers JavaScript and CSS downloads. As a result, it’s a best practice to place all CSS and JavaScript tags near the beginning of the HTML in the tag. This ensures that the browser sees them earlier. With streaming, we can reduce this delay further, by sending that portion of the tag first.", "type": "P", "href": null, "layout": null, "markups": [ { "title": null, "type": "CODE", "href": null, "userId": null, "start": 285, "end": 291, "anchorType": null }, { "title": null, "type": "CODE", "href": null, "userId": null, "start": 427, "end": 433, "anchorType": null } ], "iframe": null, "metadata": null }, { "name": "ea4f", "text": "Early Flush", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "98d3", "text": "The most straightforward way to send an early tag is by breaking a standard response into two parts. This technique is called Early Flush, as one part is sent (“flushed”) before the other.", "type": "P", "href": null, "layout": null, "markups": [ { "title": null, "type": "CODE", "href": null, "userId": null, "start": 46, "end": 52, "anchorType": null }, { "title": null, "type": "STRONG", "href": null, "userId": null, "start": 133, "end": 144, "anchorType": null } ], "iframe": null, "metadata": null }, { "name": "3146", "text": "The first part contains things that are fast to compute and can be sent quickly. At Airbnb, we include tags for fonts, CSS, and JavaScript, so that we get the browser benefits mentioned above. The second part contains the rest of the page, including content that relies on API or database queries to compute. The end result looks like this:", "type": "P", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "7a7e", "text": "Early chunk:", "type": "P", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "d80f", "text": "\n \n \n \n", "type": "PRE", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "beef", "text": "With this implemented on the server, the only remaining task is to write some JavaScript to detect when our Deferred Data chunk arrives. We did this with a MutationObserver, which is an efficient way to observe DOM changes. Once the Deferred Data JSON element is detected, we parse the result and inject it into our application’s network data store. From the application’s perspective, it’s as though a normal network request has been completed.", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://developer.mozilla.org/en-US/docs/Web/API/MutationObserver", "userId": null, "start": 156, "end": 172, "anchorType": "LINK" } ], "iframe": null, "metadata": null }, { "name": "64b4", "text": "Watch out for `defer`", "type": "P", "href": null, "layout": null, "markups": [ { "title": null, "type": "STRONG", "href": null, "userId": null, "start": 0, "end": 21, "anchorType": null } ], "iframe": null, "metadata": null }, { "name": "3228", "text": "You may notice that some tags are re-ordered from the Early Flush example. The script tags moved from the Early chunk to the Body chunk and no longer have the defer attribute. This attribute avoids render-blocking script execution by deferring scripts until after the HTML has been downloaded and parsed. This is suboptimal when using Deferred Data, as all of the visible content has already been received by the end of the Body chunk, and we no longer worry about render-blocking at that point. We can fix this by moving the script tags to the end of the Body chunk, and removing the defer attribute. Moving the tags later in the document does introduce a network waterfall, which we solved by adding preload tags into the Early chunk.", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script#attributes", "userId": null, "start": 159, "end": 174, "anchorType": "LINK" }, { "title": "", "type": "A", "href": "https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel/preload", "userId": null, "start": 702, "end": 709, "anchorType": "LINK" } ], "iframe": null, "metadata": null }, { "name": "1672", "text": "Implementation Challenges", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "d693", "text": "Status codes and headers", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "254a", "text": "Early Flush prevents subsequent changes to the headers (e.g to redirect or change the status code). In the React + NodeJS world, it’s common to delegate redirects and error throwing to a React app rendered after the data has been fetched. This won’t work if you’ve already sent an early tag and a 200 OK status.", "type": "P", "href": null, "layout": null, "markups": [ { "title": null, "type": "CODE", "href": null, "userId": null, "start": 287, "end": 293, "anchorType": null } ], "iframe": null, "metadata": null }, { "name": "6ddd", "text": "We solved this problem by moving error and redirect logic out of our React app. That logic is now performed in Express server middleware before we attempt to Early Flush.", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://expressjs.com/en/guide/using-middleware.html", "userId": null, "start": 111, "end": 136, "anchorType": "LINK" } ], "iframe": null, "metadata": null }, { "name": "802d", "text": "Buffering", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "f77b", "text": "We found that nginx buffer responses by default. This has resource utilization benefits but is counterproductive when the goal is sending incremental responses. We had to configure these services to disable buffering. We expected a potential increase in resource usage with this change but found the impact to be negligible.", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://www.nginx.com/resources/wiki/start/topics/examples/x-accel/#x-accel-buffering", "userId": null, "start": 14, "end": 19, "anchorType": "LINK" } ], "iframe": null, "metadata": null }, { "name": "534f", "text": "Response delays", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "4a34", "text": "We noticed that our Early Flush responses had an unexpected delay of around 200ms, which disappeared when we disabled gzip compression. This turned out to be an interaction between Nagle’s algorithm and Delayed ACK. These optimizations attempt to maximize data sent per packet, introducing latency when sending small amounts of data. It’s especially easy to run into this issue with jumbo frames, which increases maximum packet sizes. It turns out that gzip reduced the size of our writes to the point where they couldn’t fill a packet, and the solution was to disable Nagle’s algorithm in our haproxy load balancer.", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://en.wikipedia.org/wiki/Nagle%27s_algorithm", "userId": null, "start": 181, "end": 198, "anchorType": "LINK" }, { "title": "", "type": "A", "href": "https://en.wikipedia.org/wiki/TCP_delayed_acknowledgment", "userId": null, "start": 203, "end": 214, "anchorType": "LINK" }, { "title": "", "type": "A", "href": "https://en.wikipedia.org/wiki/Jumbo_frame", "userId": null, "start": 383, "end": 395, "anchorType": "LINK" }, { "title": "", "type": "A", "href": "https://www.haproxy.com/documentation/hapee/latest/onepage/#4.2-option%20http-no-delay", "userId": null, "start": 594, "end": 601, "anchorType": "LINK" } ], "iframe": null, "metadata": null }, { "name": "72d0", "text": "Conclusion", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "afbd", "text": "HTTP Streaming has been a very successful strategy for improving web performance at Airbnb. Our experiments showed that Early Flush produced a flat reduction in First Contentful Paint (FCP) of around 100ms on every page tested, including the Airbnb homepage. Data streaming further eliminated the FCP costs of slow backend queries. While there were challenges along the way, we found that adapting our existing React application to support streaming was very feasible and robust, despite not being designed for it originally. We’re also excited to see the broader frontend ecosystem trend in the direction of prioritizing streaming, from @defer and @stream in GraphQL to streaming SSR in Next.js. Whether you’re using these new technologies, or extending an existing codebase, we hope you’ll explore streaming to build a faster frontend for all!", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://web.dev/fcp/", "userId": null, "start": 161, "end": 183, "anchorType": "LINK" }, { "title": "", "type": "A", "href": "https://graphql.org/blog/2020-12-08-improving-latency-with-defer-and-stream-directives/", "userId": null, "start": 638, "end": 667, "anchorType": "LINK" }, { "title": "", "type": "A", "href": "https://nextjs.org/docs/advanced-features/react-18/streaming", "userId": null, "start": 671, "end": 695, "anchorType": "LINK" } ], "iframe": null, "metadata": null }, { "name": "ecb2", "text": "If this type of work interests you, check out some of our related positions here.", "type": "P", "href": null, "layout": null, "markups": [ { "title": "", "type": "A", "href": "https://careers.airbnb.com/", "userId": null, "start": 76, "end": 80, "anchorType": "LINK" } ], "iframe": null, "metadata": null }, { "name": "55ba", "text": "Acknowledgments", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "2e78", "text": "Elliott Sprehn, Aditya Punjani, Jason Jian, Changgeng Li, Siyuan Zhou, Bruce Paul, Max Sadrieh, and everyone else who helped design and implement streaming at Airbnb!", "type": "P", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "1e8d", "text": "****************", "type": "H3", "href": null, "layout": null, "markups": [], "iframe": null, "metadata": null }, { "name": "1416", "text": "All product names, logos, and brands are property of their respective owners. All company, product and service names used in this website are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.", "type": "P", "href": null, "layout": null, "markups": [ { "title": null, "type": "EM", "href": null, "userId": null, "start": 0, "end": 241, "anchorType": null } ], "iframe": null, "metadata": null } ] } } } } }