Lab & Field Data : Evaluating Core Web Vitals

The concept of lab and field data originated in scientific testing to distinguish between experiments conducted in a laboratory and those conducted outside of it.

This distinction between data sets has since been applied to website analysis, allowing us to both predict how a website should perform, and then monitor and adjust to the reality.

Lab Data

Lab Data is collected in a controlled environment and uses predefined device and network specifications. This process is called synthetic monitoring.

Lab Data is empowered by Lighthouse technology that simulates mobile throttling with a lower speed. In most cases, Lighthouse uses one location (e.g. USA) and a slower CPU than the average CPU available for users.

When you measure the web performance with lab tools, they load the website using a predetermined device and connection speed and measure how performant it is at the time the test was run. For that reason Lab Data is useful in reproducing and debugging possible performance issues. While it doesn’t give you insight into your users’ experience, it’s a valuable tool during the development cycle, and can also give you if you don’t have access to a significant sample of users.

Field Data

Field data - also called real user monitoring (RUM), real user metrics, real user measurement, or end-user experience monitoring - reflects the browsing experience of real users who use your website.

It is affected by the connection and device they’re using while browsing. This means that field data is generated by actual Google Chrome everyday users, with different computers/phones (different resources like CPU, RAM and GPU), different internet speeds, and different locations.

For these reasons field data often unearths issues that are hard to capture
in a lab environment.

Google uses the 75th percentile value of all page views to the tested page to produce the score. If at least 75 percent of page views to the tested page meet the "good" threshold, the page is classified as having "good" performance for that metric. So if most of your users come from a population with a high internet speed and powerful devices, it is normal to see field data showing better results than lab lata. On the other hand, if your server is overloaded at the time you are running the test, it is normal to see lab data recording a more negative outlook for your site than field data.

What data does Google’s algorithm use?

As field data is what’s happening on the ground, it's the most accurate way to understand what may be causing poor engagement or ranking outcomes.

As Google’s Martin Splitt said: “Field Data is probably a better indicator for how real users are experiencing your website than Lab Data. Because Lab Data is literally just someone’s server making a request to your thing. And then if that server happens to be quite beefy then you get pretty good-looking numbers, but then the real world isn’t as beefy and nice.”

Field data is important for SEO because it is what Google uses for its page experience ranking factor:

“For search rankings, we use Field Data, as this is what your site’s users have experienced over time. This makes the data more representative for your site, taking into account where your users are located and how they access your website.” John Mueller

We reached out to John Muller on Twitter, who confirmed that Google uses “field data aka CrUX for ranking, not lab tests.”

How we use lab and field data to deliver better experiences. With lab data, we can control for many factors and reliably reproduce results. That makes it a good candidate for debugging capabilities, allowing us to identify, isolate, and fix performance issues. Lab data is a useful tool when you have no field data, but we also recognise its limitations. It might not capture real-world bottlenecks and cannot correlate against real-world page KPIs.

Because lab data is simulated, it does not include user interactions, and therefore cannot help you monitor or improve your First Input Delay (FID) score, one of the three new Core Web Vitals metrics introduced in 2021.

Lab data can prepare your content for the wild, but field data is clearly where the value lies. While lab data is on-demand, field data (via CrUX) is updated every 28 days, which means we have to wait for the most valuable information, a true picture of user experience and the correlation to business key performance indicators.

As publishers, we always want to move as fast as possible, but in SEO, patience is a virtue. When our teams take action to respond to sub-optimal field data, we may need to wait several weeks to see the impact, although it is possible to see the consequences in Search Console within a week.

Field data is a powerful tool, but again, we recognise its limitations. It is a distribution of numbers whose 75th percentile is used to inform Core Web Vitals, an entirely different interpretation to lab data’s laser focus on one set of conditions, and a single output. Field data includes data from markets we may not necessarily be targeting, and significant traffic from emerging markets - where hardware and internet speed are often less advanced - can skew our data.

As we continue to grapple with Google’s new focus on Core Web Vitals, it’s important to understand the role of lab and field data for each metric, and what happens when your data is contradictory.

What happens if lab and field data report different numbers?

It’s not uncommon for lab data to return analysis that is later contradicted by your field data. Lab Data is not a proxy for your site’s actual performance, and as field data is the true picture of user experience, it carries more weight. If your lab data is bad but your field data is good, there is usually no reason to worry, although your lab data may still unearth opportunities for optimisation.

Check out the example from PageSpeedInsights on the article on our website, where we can see that although lab data is reporting a slower FCP than real user data. Lab data should return the same score every time, whereas field data will rely on hundreds of factors e.g. use of fragment identifiers, installed fonts, device screen sizes, cached content. Lab data for FCP also waits until the page is fully loaded, whereas field data measures FCP when a user interacts. For all of these reasons, we avoid direct comparisons of lab and field data. They may be capable of measuring the same metrics, but they are simply different tools that are each well suited to different tasks.

Ultimately, both data sets are a guide, and we seek to leverage all available information to highlight opportunities for improvement, whether it’s eliminating resources that block rendering, reducing unused JavaScripts or CSS, or minifying and deferring all non-critical JS/styles and compressing images. But knowing how the data is gathered is as important as the data itself. The devil is always in the details.

Core Web Vitals, supported by lab and field data, are the latest gift to digital marketers from Google. As we continue our work to meet the evolving standards of search engines and their users, understanding and agility will determine how successful we are in delivering a valuable experience for millions of visitors, and value for our advertising partners.