From AMP To PWA

A few months ago, The Washington Post launched support for Accelerated Mobile Pages (AMP HTML). Since then, we’ve been comparing the performance of AMP articles against their responsive mobile counterparts. The results are staggering. On average, our responsive mobile pages become interactive within 3.5 seconds. By comparison, our AMP pages become interactive in under 1 second.

Our AMP content can be discovered through platforms such as Google News, Twitter, or Pinterest. These initial interactions with Washington Post content benefit from the instant nature of the AMP format, but since AMP HTML is limiting (by design), we needed a way to transition users into a more immersive experience without those limitations. This post describes how we implemented a Service Worker that bootstraps a Progressive Web App (PWA) from AMP articles.

AMP Performance

You can watch Malte Ubl from the Google AMP team talk about how AMP achieves its speed, but here are some key points:

• Restrict JavaScript: AMP doesn’t allow custom JavaScript. JavaScript can block DOM construction and delay page rendering, which would prevent AMP HTML from reaching the performance goals.
• Restrict CSS: CSS blocks page load and the rendering path, so AMP requires CSS to be inline and smaller than 50 kilobytes. This also removes additional HTTP requests from the critical rendering path
• Lay out the page once: All resources are statically sized to determine the position and layout before any resources are downloaded.
• Lazy loading: AMP prioritizes resource loading by only loading elements that the user is likely to see.

These restrictions result in a consistent loading pattern that allows AMP articles to be pre-rendered without wasting unnecessary bandwidth and CPU. When an article appears in the Google News carousel (or any AMP viewer), the content above the fold can be pre-rendered before the user interacts with the article. Without pre-rendering, it is difficult to load a web page on a mobile device in under 1 second. Mobile devices are subject to Radio Resource Control (RRC), lower power states, and wide variability in network performance. Even LTE users have to deal with their devices jumping between network towers and moving between 3G and LTE. Judicious use of pre-connection and pre-rendering is the sleight of hand that makes AMP articles appear to load instantly.

A consistent theme of AMP HTML is limitation in return for consistently fast performance, but publishers and journalists are always looking for new ways to tell a story. Content such as interactive visualizations, maps, or quizzes currently require workarounds, like using the <amp-iframe> element. The AMP format also explicitly disallows ads to be displayed within a fixed position container, which prevents AMP pages from showing interstitial advertising. In comparison, standard HTML pages allow content creators to decide whether to use advertising formats that may be considered intrusive to users. With these limitations in mind, let’s turn to Progressive Web Apps.

Building a Progressive Web App

Progressive Web Apps are standard web pages enhanced to become more app-like using features like Service Workers, Push Notifications, and Web App Manifests. Using Service Workers, web developers gain access to an API that provides a programmable network proxy within the client-side context that makes it easier to create web applications that can handle network requests offline and failure modes, such as when an HTTP request is made, but a response is never received. This is a common scenario for mobile devices: perhaps the user goes through a tunnel or automatically joins a saturated wifi network.

While building our prototype PWA, we focused on the scenario of reliability and consistency with the goal of removing loading spinners and network timeouts from the user experience.

App Shell Architecture

The first step is creating the app shell, which consists of all assets necessary to render the user interface. These assets are then pre-cached by integrating sw-precache into our build process. Every time we build the application, the service worker is regenerated to account for modified assets.

Here’s an example using Gulp:

JavaScript

gulp.task('service-worker', ['minify'], function(callback) {
  swPrecache.write(buildPrefix + 'service-worker.js', {
    "staticFileGlobs": [
      ... application assets to cache ...
    ]
  }, callback);
});

The next step is loading content into the app shell, which gets pre-cached upon interaction with a given section. The idea is to use the time that the user is scrolling through headlines to load articles in the background, so that when the user taps on a specific article, the content already exists in the service worker cache and network latency effectively becomes invisible to the user.

Service Worker Cache Strategy

For caching, we use sw-toolbox to help define caching behavior, such as maximum cache size or age. For example, images at a given URL are unlikely to change, so these resources can be safely served from cache and fetched only if the resource is unavailable within the cache. On the other hand, resources such as section content use the toolbox.networkFirst handler to ensure the freshest data. Articles aren’t likely to change as quickly as a homepage, so article resources use the toolbox.fastest handler. This requests the article content from the cache and the network in parallel. Usually, this is the cached content, but this handler always makes a network request, so when the network request completes, future cache reads will contain the latest content.

Service Worker Installation from AMP Content

Once we saw how AMP can deliver a consistently fast experience, we looked at how we could use the <amp-install-serviceworker> AMP component to transfer the reader from the Google AMP cache to content on the Washington Post domain.

<amp-install-serviceworker src="https://www.washingtonpost.com/pwa/service-worker.js"
data-iframe-src="https://www.washingtonpost.com/pwa/install-serviceworker.html">
</amp-install-serviceworker>

By including the <amp-install-serviceworker> element in our AMP content, we can bootstrap our service worker before the reader has ever visited our website. If the AMP article is delivered from our domain, it can install the service worker directly using the service worker registration method, but if the article is being served from the AMP CDN, then the element can bootstrap the alternative URL specified by the data-iframe-src attribute.

Should a reader tap on a Washington Post AMP article, the registered service worker ensures that future interactions with the Washington Post domain are fast.

Lessons Learned

In building the PWA, we encountered a number of issues, made a number of concessions, and learned a few things. Here are some of those things.

Caching/Storage Limitations

Service workers provide a developer-friendly cache API, which is currently supported in Chrome, Firefox, and Opera. Microsoft recently announced that Edge support for Service Workers is in development.

Unfortunately, the browsers that don’t currently support service workers also have partial IndexedDB support. Although the Web Storage API is widely supported across browsers, localStorage operations are synchronous, which means it can block the document from loading.

Until the service worker cache API is supported as widely as localStorage, the cross-browser options for client-side caching are still difficult for developers to use.

Pre-Caching Assets

During service worker installation, we pre-cache specific assets so that when the service worker becomes active, we can rely on these resources even when the user is offline. Here’s an example:

JavaScript

self.addEventListener('install', function(e) {  
  e.waitUntil(  
    caches.open(cacheName).then(function(cache) {  
      return cache.addAll(["/", "index.html", "js/app.js"]);
    })  
  );  
});

Once the service worker is installed, we can now rely on “index.html” and “js/app.js”. Note that the paths used are relative to the service worker, so if you want the ability to intercept fetch requests across your entire site, the service worker needs to be installed from the root of your domain.

We also made concessions on what resources should be pre-cached. For example, Twitter content is commonly used, but should we pre-cache all of the CSS and JS used to load Twitter widgets? We made the decision to display tweets as blockquotes if the user is offline, but if the user is online, tweets are automatically upgraded to widgets.

When we looked at the resources we were caching, we quickly realized that our custom fonts made up a large portion of the application bundle size. To keep the pre-cached resources to a minimum, we could have left out some of our fonts. The content would still load and render offline using system fonts, but our fonts can be considered part of our brand. In the end, we decided that it was worth the up-front cost to pre-cache more assets.

Cross-Domain Issues

Service workers are subject to the web’s same-origin model, which means that “It’s Complicated.” For local development, service workers will install from localhost, but once you deploy to your domain, it’s useful to know that service workers installed on “test.example.com” are distinct from “www.example.com”.

When testing the live service worker integration with our AMP documents, we encountered another cross-domain issue. Many users will first encounter AMP articles from Google Search or News, which displays content from the AMP cache. These documents come from the cdn.ampproject.org domain, so it’s not possible to directly install a service worker from washingtonpost.com. A workaround is to load a bootstrap iframe that can be used to do nothing other than install a service worker. Fortunately, the <amp-install-serviceworker> component has an attribute (data-iframe-src) that handles this scenario.

Analytics

Our web site is essentially a single page application, so the first step in adding analytics is to send page views as custom events whenever the user views a section or reads an article. We chose to use Google Analytics, so this meant sending custom user timings for events to break down the load times by app-shell, section, or by individual articles (with or without a service worker).

Once web sites go offline, we need to make sure that analytics still get sent. One way to do this is to add a fetch event listener for analytics requests, queue them in the event of failure, and replay queued requests whenever the service worker starts up.

What’s Next?

While our PWA is in beta, we’re learning how to integrate service workers with our publishing platform, so that we can progressively enhance our existing responsive web site with new features like push notifications and offline functionality. Before we get there, we’re making sure that enhancements to our production experience continue to work cross-browser and cross-platform. If URLs are the web’s super power, then it’s important that content is accessible on any device capable of fetching a URL. Our vision for the near future is to make the entire Washington Post website load extremely fast, work offline, and deliver customized content and push notifications to every platform.

Native Apps vs. PWAs vs. AMPs

Now that you understand the basic functionalities and benefits of native apps, PWAs, and AMPs, let’s compare all three technologies to build more clarity on which one is the most suitable for your business.

  By: Chris Nguyen

References:

https://www.ampproject.org/ https://greenido.wordpress.com/2016/06/07/from-amp-to-pwa/ https://developer.washingtonpost.com/pb/blog/post/2016/07/15/amp-up-with-progressive-web-apps/ https://twitter.com/cramforce