How Much Data Is Missing from Analytics? And Other Analytics Black Holes

Posted by Tom.Capper

If you’ve ever compared two analytics implementations on the same site, or compared your analytics with what your business is reporting in sales, you’ve probably noticed that things don’t always match up. In this post, I’ll explain why data is missing from your web analytics platforms and how large the impact could be. Some of the issues I cover are actually quite easily addressed, and have a decent impact on traffic — there’s never been an easier way to hit your quarterly targets. ;)

I’m going to focus on GA (Google Analytics), as it's the most commonly used provider, but most on-page analytics platforms have the same issues. Platforms that rely on server logs do avoid some issues but are fairly rare, so I won’t cover them in any depth.

Side note: Our test setup (multiple trackers & customized GA)

On Distilled.net, we have a standard Google Analytics property running from an HTML tag in GTM (Google Tag Manager). In addition, for the last two years, I’ve been running three extra concurrent Google Analytics implementations, designed to measure discrepancies between different configurations.

(If you’re just interested in my findings, you can skip this section, but if you want to hear more about the methodology, continue reading. Similarly, don’t worry if you don’t understand some of the detail here — the results are easier to follow.)

Two of these extra implementations — one in Google Tag Manager and one on page — run locally hosted, renamed copies of the Google Analytics JavaScript file (e.g. www.distilled.net/static/js/au3.js, instead of www.google-analytics.com/analytics.js) to make them harder to spot for ad blockers. I also used renamed JavaScript functions (“tcap” and “Buffoon,” rather than the standard “ga”) and renamed trackers (“FredTheUnblockable” and “AlbertTheImmutable”) to avoid having duplicate trackers (which can often cause issues).

This was originally inspired by 2016-era best practice on how to get your Google Analytics setup past ad blockers. I can’t find the original article now, but you can see a very similar one from 2017 here.

Lastly, we have (“DianaTheIndefatigable”), which just has a renamed tracker, but uses the standard code otherwise and is implemented on-page. This is to complete the set of all combinations of modified and unmodified GTM and on-page trackers.

Two of Distilled’s modified on-page trackers, as seen on https://www.distilled.net/

Overall, this table summarizes our setups:

Tracker	Renamed function?	GTM or on-page?	Locally hosted JavaScript file?
Default	No	GTM HTML tag	No
FredTheUnblockable	Yes - “tcap”	GTM HTML tag	Yes
AlbertTheImmutable	Yes - “buffoon”	On page	Yes
DianaTheIndefatigable	No	On page	No

I tested their functionality in various browser/ad-block environments by watching for the pageviews appearing in browser developer tools:

Reason 1: Ad Blockers

Ad blockers, primarily as browser extensions, have been growing in popularity for some time now. Primarily this has been to do with users looking for better performance and UX on ad-laden sites, but in recent years an increased emphasis on privacy has also crept in, hence the possibility of analytics blocking.

Effect of ad blockers

Some ad blockers block web analytics platforms by default, others can be configured to do so. I tested Distilled’s site with Adblock Plus and uBlock Origin, two of the most popular ad-blocking desktop browser addons, but it’s worth noting that ad blockers are increasingly prevalent on smartphones, too.

Here’s how Distilled’s setups fared:

(All numbers shown are from April 2018)

Setup	Vs. Adblock	Vs. Adblock with “EasyPrivacy” enabled	Vs. uBlock Origin
GTM	Pass	Fail	Fail
On page	Pass	Fail	Fail
GTM + renamed script & function	Pass	Fail	Fail
On page + renamed script & function	Pass	Fail	Fail

Seems like those tweaked setups didn’t do much!

Lost data due to ad blockers: ~10%

Ad blocker usage can be in the 15–25% range depending on region, but many of these installs will be default setups of AdBlock Plus, which as we’ve seen above, does not block tracking. Estimates of AdBlock Plus’s market share among ad blockers vary from 50–70%, with more recent reports tending more towards the former. So, if we assume that at most 50% of installed ad blockers block analytics, that leaves your exposure at around 10%.

Reason 2: Browser “do not track”

This is another privacy motivated feature, this time of browsers themselves. You can enable it in the settings of most current browsers. It’s not compulsory for sites or platforms to obey the “do not track” request, but Firefox offers a stronger feature under the same set of options, which I decided to test as well.

Effect of “do not track”

Most browsers now offer the option to send a “Do not track” message. I tested the latest releases of Firefox & Chrome for Windows 10.

Setup	Chrome “do not track”	Firefox “do not track”	Firefox “tracking protection”
GTM	Pass	Pass	Fail
On page	Pass	Pass	Fail
GTM + renamed script & function	Pass	Pass	Fail
On page + renamed script & function	Pass	Pass	Fail

Again, it doesn’t seem that the tweaked setups are doing much work for us here.

Lost data due to “do not track”: <1%

Only Firefox Quantum’s “Tracking Protection,” introduced in February, had any effect on our trackers. Firefox has a 5% market share, but Tracking Protection is not enabled by default. The launch of this feature had no effect on the trend for Firefox traffic on Distilled.net.

Reason 3: Filters

It’s a bit of an obvious one, but filters you’ve set up in your analytics might intentionally or unintentionally reduce your reported traffic levels.

For example, a filter excluding certain niche screen resolutions that you believe to be mostly bots, or internal traffic, will obviously cause your setup to underreport slightly.

Lost data due to filters: ???

Impact is hard to estimate, as setup will obviously vary on a site-by site-basis. I do recommend having a duplicate, unfiltered “master” view in case you realize too late you’ve lost something you didn’t intend to.

Reason 4: GTM vs. on-page vs. misplaced on-page

Google Tag Manager has become an increasingly popular way of implementing analytics in recent years, due to its increased flexibility and the ease of making changes. However, I’ve long noticed that it can tend to underreport vs. on-page setups.

I was also curious about what would happen if you didn’t follow Google’s guidelines in setting up on-page code.

By combining my numbers with numbers from my colleague Dom Woodman’s site (you’re welcome for the link, Dom), which happens to use a Drupal analytics add-on as well as GTM, I was able to see the difference between Google Tag Manager and misplaced on-page code (right at the bottom of the <body> tag) I then weighted this against my own Google Tag Manager data to get an overall picture of all 5 setups.

Effect of GTM and misplaced on-page code

Traffic as a percentage of baseline (standard Google Tag Manager implementation):

	Google Tag Manager	Modified & Google Tag Manager	On-Page Code In <head>	Modified & On-Page Code In <head>	On-Page Code Misplaced In <Body>
Chrome	100.00%	98.75%	100.77%	99.80%	94.75%
Safari	100.00%	99.42%	100.55%	102.08%	82.69%
Firefox	100.00%	99.71%	101.16%	101.45%	90.68%
Internet Explorer	100.00%	80.06%	112.31%	113.37%	77.18%

There are a few main takeaways here:

• On-page code generally reports more traffic than GTM
• Modified code is generally within a margin of error, apart from modified GTM code on Internet Explorer (see note below)
• Misplaced analytics code will cost you up to a third of your traffic vs. properly implemented on-page code, depending on browser (!)
• The customized setups, which are designed to get more traffic by evading ad blockers, are doing nothing of the sort.

It’s worth noting also that the customized implementations actually got less traffic than the standard ones. For the on-page code, this is within the margin of error, but for Google Tag Manager, there’s another reason — because I used unfiltered profiles for the comparison, there’s a lot of bot spam in the main profile, which primarily masquerades as Internet Explorer. Our main profile is by far the most spammed, and also acting as the baseline here, so the difference between on-page code and Google Tag Manager is probably somewhat larger than what I’m reporting.

I also split the data by mobile, out of curiosity:

Traffic as a percentage of baseline (standard Google Tag Manager implementation):

	Google Tag Manager	Modified & Google Tag Manager	On-Page Code In <head>	Modified & On-Page Code In <head>	On-Page Code Misplaced In <Body>
Desktop	100.00%	98.31%	100.97%	100.89%	93.47%
Mobile	100.00%	97.00%	103.78%	100.42%	89.87%
Tablet	100.00%	97.68%	104.20%	102.43%	88.13%

The further takeaway here seems to be that mobile browsers, like Internet Explorer, can struggle with Google Tag Manager.

Lost data due to GTM: 1–5%

Google Tag Manager seems to cost you a varying amount depending on what make-up of browsers and devices use your site. On Distilled.net, the difference is around 1.7%; however, we have an unusually desktop-heavy and tech-savvy audience (not much Internet Explorer!). Depending on vertical, this could easily swell to the 5% range.

Lost data due to misplaced on-page code: ~10%

On Teflsearch.com, the impact of misplaced on-page code was around 7.5%, vs Google Tag Manager. Keeping in mind that Google Tag Manager itself underreports, the total loss could easily be in the 10% range.

Bonus round: Missing data from channels

I’ve focused above on areas where you might be missing data altogether. However, there are also lots of ways in which data can be misrepresented, or detail can be missing. I’ll cover these more briefly, but the main issues are dark traffic and attribution.

Dark traffic

Dark traffic is direct traffic that didn’t really come via direct — which is generally becoming more and more common. Typical causes are:

• Untagged campaigns in email
• Untagged campaigns in apps (especially Facebook, Twitter, etc.)
• Misrepresented organic
• Data sent from botched tracking implementations (which can also appear as self-referrals)

It’s also worth noting the trend towards genuinely direct traffic that would historically have been organic. For example, due to increasingly sophisticated browser autocompletes, cross-device history, and so on, people end up “typing” a URL that they’d have searched for historically.

Attribution

I’ve written about this in more detail here, but in general, a session in Google Analytics (and any other platform) is a fairly arbitrary construct — you might think it’s obvious how a group of hits should be grouped into one or more sessions, but in fact, the process relies on a number of fairly questionable assumptions. In particular, it’s worth noting that Google Analytics generally attributes direct traffic (including dark traffic) to the previous non-direct source, if one exists.

Discussion

I was quite surprised by some of my own findings when researching this post, but I’m sure I didn’t get everything. Can you think of any other ways in which data can end up missing from analytics?

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!

How Do You Set Smart SEO Goals for Your Team/Agency/Project? - Whiteboard Friday

Posted by randfish

Are you sure that your current SEO goals are the best fit for your organization? It's incredibly important that they tie into both your company goals and your marketing goals, as well as provide specific, measurable metrics you can work to improve. In this edition of Whiteboard Friday, Rand outlines how to set the right SEO goals for your team and shares two examples of how different businesses might go about doing just that.

Click on the whiteboard image above to open a high-resolution version in a new tab!

Video Transcription

Howdy, Moz fans, and welcome to another edition of Whiteboard Friday. This week we're chatting about SEO goals, how to set smart ones, how to measure your progress against them, how to amplify those goals to the rest of your organization so that people really buy in to SEO.

This is a big challenge. So many folks that I've talked to in the field have basically said, "I'm not sure exactly how to set goals for our SEO team that are the right ones." I think that there's a particularly pernicious problem once Google took away the keyword-level data for SEO referrals.

So, from paid search, you can see this click was on this keyword and sent traffic to this page and then here's how it performed after that. In organic search, you can no longer do that. You haven't been able to do it for a few years now. Because of that removal, proving the return on investment for SEO has been really challenging. We'll talk in a future Whiteboard Friday about proving ROI. But let's focus here on how you get some smart SEO goals that are actually measurable, trackable, and pertain intelligently to the goals of the business, the organization.

Where to start:

So the first thing, the first problem that I see is that a lot of folks start here, which seems like a reasonable idea, but is actually a terrible idea. Don't start with your SEO goals. When your SEO team gets together or when you get together with your consultants, your agency, don't start with what the SEO goals should be.

• Start with the company goals. This is what our company is trying to accomplish this quarter or this year or this month.
• Marketing goals. Go from there to here's how marketing is going to contribute to those company goals. So if the company has a goal of increasing sales, marketing's job is what? Is marketing's job improving the conversion funnel? Is it getting more traffic to the top of the funnel? Is it bringing back more traffic that's already been to the site but needs to be re-earned? Those marketing goals should be tied directly to the company goals so that anyone and everyone in the organization can clearly see, "Here's why marketing is doing what they're doing."
• SEO goals. Next, here's how SEO contributes to those marketing goals. So if the goal is around, as we mentioned, growing traffic to the top of the funnel, for example, SEO could be very broad in their targeting. If it's bringing people back, you've got to get much more narrow in your keyword targeting.
• Specific metrics to measure and improve. From those SEO goals, you can get the outcome of specific metrics to measure and improve.

Measurable goal metrics

So that list is kind of right here. It's not very long. There are not that many things in the SEO world that we can truly measure directly. So measurable goal metrics might be things like...

1. Rankings. Which we can measure in three ways. We can measure them globally, nationally, or locally. You can choose to set those up.

2. Organic search visits. So this would be just the raw traffic that is sent from organic search.

3. You can also separate that into branded search versus non-branded search. But it's much more challenging than it is with paid, because we don't have the keyword data. Thus, we have to use an implied or inferred model, where essentially we say, "These pages are likely to be receiving branded search traffic, versus these pages that are likely to be receiving non-branded search traffic."

A good example is the homepage of most brands is most likely to get primarily branded search traffic, whereas resource pages, blog pages, content marketing style pages, those are mostly going to get unbranded. So you can weight those appropriately as you see fit.

Tracking your rankings is crucially important, because that way you can see which pages show up for branded queries versus which pages show up for unbranded queries, and then you can build pretty darn good models of branded search versus non-branded search visits based on which landing pages are going to get traffic.

4. SERP ownership. So ideas around your reputation in the search results. So this is essentially looking at the page of search results that comes up for a given query and what results are in there. There might be things you don't like and don't want and things you really do want, and the success and failure can be measured directly through the rankings in the SERP.

5. Search volume. So for folks who are trying to improve their brand's affinity and reputation on the web and trying to grow the quantity of branded search, which is a good metric, you can look at that through things like Google Trends or through a Google AdWords campaign or through something like Moz's Keyword Explorer.

6. Links and link metrics. So you could look at the growth or shrinkage of links over time. You can measure that through things like the number of linking root domains, the total number of links. Authority or spam metrics and how those are distributed.

7. Referral traffic. And last, but not least, most SEO campaigns, especially those that focus on links or improving rankings, are going to also send referral traffic from the links that are built. So you can watch referral traffic and what those referrers are and whether they came from pages where you built links with SEO intent.

So taking all of these metrics, these should be applied to the SEO goals that you choose that match up with your marketing and company goals. I wanted to try and illustrate this, not just explain it, but illustrate it through two examples that are very different in what they're measuring.

Example one

So, first off, Taft Boots, they've been advertising like crazy to me on Instagram. Apparently, I must need new boots.

• Grow online sales. Let's say that their big company goal for 2018 is "grow online sales to core U.S. customers, so the demographics and psychographics they're already reaching, by 30%."
• Increase top of funnel website traffic by 50%. So marketing says, "All right, you know what? There's a bunch of ways to do that, but we think that our best opportunity to do that is to grow top of funnel, because we can see how top of funnel turns into sales over time, and we're going to target a number of 50% growth." This is awesome. This can turn into very measurable, actionable SEO goals.
• Grow organic search visits 70%. We can say, "Okay, we know that search is going to contribute an outsized quantity of this 50% growth. So what we want to do is take search traffic up by 70%. How are we going to do that? We have four different plans.
  • A. We're going to increase our blog content, quality and quantity.
  • B. We're going to create new product pages that are more detailed, that are better optimized, that target good searches.
  • C. We're going to create a new resources section with some big content pieces.
  • D. We're going to improve our link profile and Domain Authority."

Now, you might say, "Wait a minute. Rand, this is a pretty common SEO methodology here." Yes, but many times this is not directly tied to the marketing goals, which is not directly tied to the business goals. If you want to have success as an SEO, you want to convince people to keep investing in you, you want to keep having that job or that consulting gig, you've got to connect these up.

From these, we can then say, "Okay, for each one, how do we measure it?" Well...

• A. Quantity of content and search visits/piece. Blog content can be measured through the quantity of content we produce, the search visits that each of those pieces produce, and what the distribution and averages are.
• B. Rankings and organic traffic. Is a great way to measure product pages and whether we're hitting our goals there.
• C. Link growth, rankings, and traffic. That's a great way to measure the new resources section.
• D. Linking root domains plus the DA distribution and maybe Spam Score distribution. That's a great way to measure whether we're improving our link profile.

All of these, this big-picture goal is going to be measured by the contribution of search visits to essentially non-homepage and non-branded pages that contribute to the conversion funnel. So we have a methodology to create a smart goal and system here.

Example two

Another example, totally different, but let's try it out because I think that many folks have trouble connecting non-e-commerce pages, non-product stuff. So we're going to use Book-It Theatre. They're a theater group here in the Seattle area. They use the area beneath Seattle Center House as their space. They basically will take popular books and literature and convert them into plays. They'll adapt them into screenplays and then put on performances. It's quite good. We've been to a few shows, Geraldine and I have, and we really like them.

So their goal — I'm making this up, I don't actually know if this is their goal — but let's say they want to...

• Attract theater goers from outside the Seattle area. So they're looking to hit tourists and critics, people who are not just locals, because they want to expand their brand.
• Reach audiences in 4 key geographies — LA, Portland, Vancouver, Minneapolis. So they decide, "You know what? Marketing can contribute to this in four key geographies, and that's where we're going to focus a bunch of efforts — PR efforts, outreach efforts, offline media, and SEO. The four key geographies are Los Angeles, Portland, Vancouver, and Minneapolis. We think these are good theater-going towns where we can attract the right audiences."

So what are we going to do as SEOs? Well, as SEOs, we better figure out what's going to match up to this.

• Drive traffic from these regions to Book-It Theatre's pages and to reviews of our show. So it's not just content on our site. We want to drive people to other critics and press that's reviewed us.
  • A. So we're going to create some geo landing pages, maybe some special offers for people from each of these cities.
  • B. We're going to identify third-party reviews and hopefully get critics who will write reviews, and we're going to ID those and try and drive traffic to them.
  • C. We're going to do the same with blog posts and informal critics.
  • D. We're going to build some content pages around the books that we're adapting, hoping to drive traffic, that's interested in those books, from all over the United States to our pages and hopefully to our show.

So there are ways to measure each of these.

• A. Localized rankings in Moz Pro or a bunch of other rank tracking tools. You can set up geo-specific localized rankings. "I want to track rankings in Vancouver, British Columbia. I want to track rankings from Los Angeles, California." Those might look different than the ones you see here in Seattle, Washington.
• B. We can do localized rankings and visits from referrals for the third-party reviews. We won't be able to track the visits that those pages receive, but if they mention Book-It Theatre and link to us, we can see, oh yes, look, the Minneapolis Journal wrote about us and they linked to us, and we can see what the reviews are from there.
• C. We can do localized rankings and visits from referrals for the third-party blog posts.
• D. Local and national ranking, organic visits. For these Book-It content pages, of course, we can track our local and national rankings and the organic visits.

Each of these, and as a whole, the contribution of search visits from non-Seattle regions, so we can remove Seattle or Washington State in our analytics and we can see: How much traffic did we get from there? Was it more than last year? What's it contributing to the ticket sales conversion funnel?

You can see how, if you build these smart goals and you measure them correctly and you align them with what the company and the marketing team is trying to do, you can build something really special. You can get great involvement from the rest of your teams, and you can show the value of SEO even to people who might not believe in it already.

All right, everyone. Look forward to your thoughts and feedback in the comments, and we'll see you again next week for another edition of Whiteboard Friday. Take care.

Video transcription by Speechpad.com

Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!