Stop building websites for iPhones
This is the first part of a series where we cover site performance in detail.
RentPath has multiple websites including rent.com, apartmentguide.com and rentals.com that serve a wide variety of renters in different demographics with varying types of devices. We also partner with property managers in Atlanta to offer housing to the homeless, soon-to-be evicted and others in need through our OpenDoors non-profit. If you would like to help start the program in another city, please contact us.
Most of the sites use React, which is known for its great single-page app (SPA) experience but at the cost of initial page load due to a heavy client-side JavaScript. In the first half of 2020, we dove deep into how JavaScript-heavy client-side rendered apps impacted performance with help from Tim Kadlec. In this series, we’ll talk through our experience across a number of topics around browser and site performance.
The Mobile Performance Inequality Gap
I was originally going to spend the first post talking about mobile devices, how they vary in performance widely and some assumptions you may have about connectivity. Instead, you should read Alex Russell’s post The Mobile Performance Inequality Gap, 2021 where he covers several points much better than I would have.
Here are the highlights that we’ll use:
- Most of your mobile traffic is probably 4G so we don’t need to worry about 3G as much. To double-check your numbers here, you can check your Chrome User Experience (CrUX) dashboard under “Connection Distribution”. If you haven’t heard of CrUX, it provides user experience metrics for real-world Chrome browsers aggregated from users with Usage Statistics enabled. If you don’t have a CrUX dashboard, you should create one as it has a lot of useful RUM data from the field. Our sites are reliably showing 95%+ in the 4G range.
- Hardware performance has widened for devices at the high-end vs mid to low markets. 2020 high-end Androids have the same single-core performance of an iPhone 8 from 2017, mid-priced phones are closer to 2014’s iPhone 6 and low-end phones are comparable to 2012’s iPhone 5.
- Browsers and JavaScript engines continue to offload more work from the main thread but the main thread is still where the majority of the execution time is spent. React-based apps are tied to the DOM and only the main thread can touch the DOM therefore React-based apps are tied to single-core performance.
Not everyone can afford the latest iPhone or Galaxy
At RentPath, we strive to challenge our assumptions and biases. It’s not a secret that Product and Engineering are well-paid positions within a company and can earn 6-figure salaries which allows them the luxury of purchasing higher-end mobile phones.
I’ve frequently shared the top-selling phones on Amazon internally as an example that not everyone walks around with the same phone in their pocket.
As of this writing, Amazon’s top-selling unlocked phone is the BLU Studio running the ARM Cortex A55 at 1.6 GHz which has a Geekbench 5 score of under 70 for single-core.
If you compare it to the numbers in the chart above, you’ll note it’d be at the bottom.
Another top-selling cell phone chart list on Amazon shows the “Apple iPhone 8 (renewed)”. Which is roughly 60x faster than the BLUE Studio. Amazing what you can get for 3x the price but for most retail workers earning $12/hr, that’s half your weekly paycheck before taxes.
Another reminder of low-end phones are with free Obama phones, though the program existed well before President Obama took office. These are free cell phones available to welfare recipients and other low-income families that qualify. It comes up frequently as a government spending issue but it’s not funded by the government or taxpayer money. If you’ve ever seen a “universal service fee” on your cell phone bill, that’s to cover the intent of The Telecommunications Act of 1934 to “to make available, so far as possible, to all the people of the United States, a rapid, efficient, Nation-wide, and world-wide wire and radio communication service with adequate facilities at reasonable charges” and extended in the 1996 Telecommunications Act to provide discounted landline service and now cellular service.
In my state, Georgia, you can get 4.5GB data, unlimited texts, 1000 voice minutes and a free Android phone if you qualify. What phone will you get? It varies but, at best, it’ll be similar to the BLU Studio phone that pulled in that whopping score of almost 70.
Now that we’ve established a wide range of phones with a wide range of performance characteristics exist and not everyone’s rocking the latest iPhone but what about my website?
Performance scores for mobile devices, even iPhones
To answer the questions: What does the average device using my site look like? What’s the bottom quartile of users? What should my site accommodate?
We start with your web analytics platform. For us, we have multiple systems but primarily use Google Analytics first. Peeking at Audience > Mobile > Devices, you’ll see something like this screenshot.
Based on just this, you’d think we were mostly serving Apple iPhones but the long tail of Samsung, LG and other Android based devices actually adds up. Fortunately for iPhone users, Apple takes privacy seriously by not relaying much about your phone to websites. Unfortunately for me, I now have to break down this really big number to make sense of it. Let’s start by aligned iPhones to their different resolutions.
The user agent string for an iPhone might be `Mozilla/5.0 (iPhone; CPU iPhone OS 12_2 like Mac OS X)`, which doesn’t say anything about the hardware but you’ll also get the viewport size and device resolution. Using a mapping of device name and resolution, I can determine which iPhone it might be. Some phones overlap in their device resolution, for instance, the iPhone 6, 6S, 7 and 8 all have a 4.7" screen with a viewport size of 375x667.
With the available data though, we can start segmenting the Apple iPhone and iPad categories into subsets of devices with different performance characteristics. The speed column is populated by looking at device speeds as recording on Geekbench 5 single-core scores.
The speed score chosen is the “best case” scenario for that row. For instance, the iPhone 6–8 ranges between 313–924 in scores but we will favor the 924 in our calculations. The hope is that with time, the overall phone speeds are increasing as old phones break and new phones are purchased. Additionally, you can see iPhone penetration is fairly distributed (in this Feb 2020 diagram). David Smith has some great data to compare to.
My goal is to prove we have slow devices so I don’t want to arbitrarily skew the numbers in my favor. Where we can, we’ll round up speeds to favor high-end devices.
For the complete picture, we’ll grab Geekbench speed scores for all our Android devices as well.
Once we have our lookup table, we can CSV dump our Google Analytics data and use a simple formula that combines the “mobile device info” with the “screen resolution” and pull in the speed index/score.
And you’ll end up with:
For this unspecified time frame on a RentPath site, you can see we still have a lot of iPhone devices but now the speed varies considerably from 259 on the iPhone 5 up to the latest iPhone 11’s (this data is older and unspecific for public consumption).
If you were to plot the results, we can see there are a chunk of sessions below 500. If you were to consider anyone with a score of 400 or lower as “slow”, that represents 38% of our sessions. You could easily fiddle with some speed scoring numbers to assume an average or extrapolate the total counts to estimate out 45% of our sessions as “slow”.
That means we’re looking at 38% in the best case to 45% worst case. Closer to the 400 scores are iPhone 5, Samsung Galaxy A20 / S8, Google Nexus 5 / Pixel 2 or similar phones. However, around the 100 score mark are LG Q710AL Stylo 4, LG LM-X220 Aristo 3 and Motorola Moto E6 phones mostly running Snapdragon 425–450 chipset with Cortex-A53 CPUs. That’s the big spike at the 200 for really old chipset phones rounding up due to statistical binning.
How much of this is JavaScript?
JavaScript can take up a lot of time impacting Total Blocking Time (TBT) and First Input Delay (FID). For instance, here’s our main thread:
TBT was in the red for Lighthouse clocking in at 1.5 secs. Overall though it felt fast because the Largest Contentful Paint (LCP) was below 0.5 secs. Safari looks pretty good too:
Oh wait, that was on my super fast Mac. We just spent a bunch of time talking about slow mobile devices. What if we slowed down the CPU and rechecked:
Oh. TBT is 38 secs with the LCP around 1.5 secs. So it didn’t look as slow with LCP but that TBT is really high and the Time to Interactive (TTI) is 48 secs, it sure felt slow when you attempt to scroll the page or interact with it before the page was “ready”.
Well, this page isn’t really a single-page app in nature. I could easily just server-side render all pages and throw everything React out…
Vanilla JavaScript vs React
At this point, the team was challenged to build an experience for both the user and the developer that worked well. In one corner, we already server-side rendered the page so we could slap some vanilla JavaScript on top of the generated HTML to drive what little interactions were needed on some page types, similar to what Netflix did. In the other corner, developers didn’t want to Frankenstein vanilla JavaScript with React components to create additional cognitive overhead for developers.
The bakeoff took just over a month and two POCs were built.
The vanilla JavaScript solution was impressive. JavaScript went from 1.7 MB down to 15 KB. It scored a Lighthouse performance score of 100 on desktop and 95 on mobile. The biggest issue besides developer happiness was that it wouldn’t work on all our page types. Some pages were fairly static and read-only with little interactivity however other pages were highly interactive and this couldn’t help those pages at all.
The other solution scored 99 on desktop and 94 on mobile, very close to the vanilla scores. The team switched from React to Preact, simplified client-side GraphQL data fetching and the store but most importantly, introduced progressive hydration via new Async and Static component wrappers that both could chunk and delay hydration of client-side components.
Building a much faster React site
For an in-depth article on the topic of front-end performance enhancements and tuning, I highly recommend reading the Front-End Performance Checklist on Smashing Magazine. It dives deeper into some of the topics mentioned above including different rendering and hydration techniques.
In an upcoming article, we’ll cover our new Async and Static components for Progressive Hydration and how we took Lighthouse performance scores from 32 to 94 on mobile!
