The Practical Implications of Precision vs. Accuracy in Web Analytics

Many times clients ask why their web analytics data does not match other databases or what they know to be different. This often happens when the client's site has low levels of traffic or form submissions. The expectation is that the web analytics tool -- whether it be Google Analytics, Adobe Analytics, or any other tool -- will show every single visit and submission as an exact match to the internal marketing databases. The reality is that the web analytical solutions cannot be expected to provide such exact measurements and that this is most evident when there are low levels of traffic.

For example, many clients count form submissions and find small differences between the submission in the web analytics and the data received in databases. This is especially visible when the form submissions are very low. The desire is for the web analytics to be an exact measure of every exact measure of performance rather than what it is intended to show - a trend or scale.

The Purpose of Analytics Tools

A key misconception surrounding web analytical tools is that they are designed to capture and report on every interaction. This is not the case. Web analytics tools are meant to show overall trends in aggregate and frequently do not capture exhaustive detail for every individual. Overall trends are less impacted by minor data variances. This is the level of accuracy and precision web analytics tools are centered around. They are not intended to be the tool for recording down to individual interactions. While web analytics tools can be configured for a level of such tracking, individual interactions are better tracked and reported upon with a customer relationship management (CRM) tool.

Accuracy and Precision

Why are web analytics tools not good for micro measurements? The answer lies in understanding the difference between accuracy and precision.

Accuracy means that the data is correct. Precision means that the data is consistent. Data may not be 100% accurate. And precise data may not be accurate at all. The aim is to have high levels of both accuracy and precision.

To have them both you need to be able to use a set of tools that capture, retain, and report on a large pool of data that is recorded in the same matter over time - in other words, you need to have both scale and consistency. This is prohibitive with web analytics tools for a number of reasons:

Tools Change

Web analytics tools use algorithms and other methods to collect, store, and report on data and are routinely updated. This can change the basis for the data as tools attempt to improve accuracy and precision and may add other measures that may not have consistent levels across metrics and dimensions.

Collection May Be Disabled

Web analytics tools are one of the easiest items to be disabled intentionally and unintentionally by a visitor, their browser, ISPs and others along the connection. Besides the complete disabling of tracking, events may occur that impact data collection, such as JavaScript not being allowed to run or failing, cookies being disabled or deleted, or browsers not fully loading or crashing on pages.

Tools Break

Even the best of tools do not always work. Sometimes they only partially capture information and face outages that can impact data collection, precision, and accuracy.

Tools are Implemented Improperly

Every page or item to be tracked, such as emails, PDF files, videos, etc., must contain the same tracking code to ensure all interactions are measured consistently. It is common for this code to be placed on the website, while marketing tools and support tools are overlooked. In addition, these tools need to be configured properly to each unique situation in order to collect all the data desired.

Users May Not Have Consistent Configurations

Each tool must be configured to meet users' needs in a consistent manner. This is particularly difficult in Google Analytics since reports, segments, and many other aspects cannot be set up and managed by an administrator for the users. In addition, a change in the configuration means the data collected and how it is recorded may differ over time. Many users are unaware of these implications when they adjust their tools.

Tools are Not Used Properly

Users may not be up-to-date on how to use web analytics tools, or they might not use the proper or best settings and reports for their needs. They may also improperly interpret data or assume it means something other than intended.

Tools Do Not Connect Sessions or Individuals

Analytics tools may not connect the traffic from an individual if they switch computers, use different devices or browsers, or delete cookies. The impact of this may be reduced through a variety of customizations, but it can never be eliminated.

Tools Do Not Measure Based Upon All Data

Analytics tools routinely ignore or otherwise discard data that is partial, corrupt, or deemed incorrect. This can lead to only part of a visit being measured or a visit not being measured at all. Google Analytics is notorious for reporting based upon a subset of page views when traffic is higher than an allotted portion. This by definition means you cannot have a one-to-one measure of visitor activity since the data is sampled. While statistically accurate, the details related to this sampling must be clearly understood.

Sites May Bypass Analytics

Sites and campaigns may be configured in a manner that bypass tracking. For example, utilizing domain or web page redirects can break the tracking connection.

Tools Do Not Handle Reloads Gracefully

A visitor may reload a page in their browser or cancel pages before they fully display. Both of these are common for receipt and thank you pages. This can throw analytics tools out of whack and record improper conversions.

Tools are Subject to Data Insertions

Every analytics tool has a method to insert external data without authorization. This is useful for items such as email and ad clicks. However, this can impact data as is seen with referral data into Google Analytics data.

Mitigate and Eliminate Faults

Does this mean that web analytics tools are a failure? Absolutely not! The difference is in the expectation versus use case. Web analytics tools are all based upon a statistically acceptable standard of precision and accuracy. This enables analytics tools to be used for macro-level trend reporting. The failure comes into play when attempting to use any single report as a micro-measuring tool with 100% accuracy and 100% precision. There are ways to mitigate but you can never eliminate faults.

Does this mean that GA and other web analytics tools are not useful? No! It simply means we need to understand possible issues and work within limitations. The general movement is towards more accurate and precise collection and reporting but web analytics tools may never offer a fully accurate or precise picture. Depending on your situation, additional tool sets can - and should - be configured to improve reporting to assist further with analysis.