In the immortal words of
Avinash, web analytics ninja,
'Data quality on the internet absolutely sucks. And there is nothing you can do about it. At least for now.'
There are so many reasons for data quality to be far from optimal and Avinash does a great job of listing them and explaining why they don't really matter as much as technical analysts and some clients believe.
But he also correctly (IOHO) states that you should be happy with 95%.
This means that sometimes data will not tally exactly between what traffic your media company says they drove and the actual traffic you see on your site. It also means that your analyst will sometimes have to re-state results when better data becomes available.
It also means that you will be caused to look goofy in front of the detail-obsessed VP or junior producer. (Why is it that the people at the extreme poles of the food chain get caught up on the most pointless things? Maybe middling managers are too busy to worry about it).
Here's our advice for dealing with all of this:
- Take a chill pill. Stop, breathe, think about how unimportant a +/-10% variation is in the grand scheme of things. Remember that...
- We are looking at trends. If one measurement tool is consistently lower, even by an order of magnitude of 50%, that's fine, as long as it's consistently lower. The most important thing is how the results vary in relation to the same measurement with the same tool last week/quarter/year. Accuracy is much more important than being over-precise.
- Focus on insights. 'How can we trust this recommendation when the data quality is untrustworthy?' Our response is that you're focusing not on the trees, but on the tiny bugs that live in the leaves. Think bigger - what is the overall picture? In most cases, we find that data tends to back up your hunch anyway so you should only be concerned if the data is telling you something completely incredible, as in unbelievable. If it looks and smells right, it probably is. Think big.
 |
| Big picture |
Here's one piece of advice that will help you and your insighteer immensely. If you do have concerns, raise them privately with your analyst, not with everybody and their Mom and their Mom's SVP cc'd on the email. It's probably a simple thing to answer and it may be that you misunderstood something. Every time you raise a concern about what you think is data integrity, you make life harder for everybody....including yourself when you need that critical rush piece of information. Don't get the reputation as the consumer who is so focused on the details that they miss the big points.
 |
| Waste of time - zero insight |
We are definitely not saying that you should not be as rigorous or analytical as you possibly can but if you just follow one piece of advice, it should be this: STAY OUT OF THE RABBIT HOLES. It's so easy to spin and spin on getting that extra 2% or 3% of of precision. Some analysts love that sh!t.
But they're usually the ones who produce those crazy dense 'dashboards' with zero insights.
Think accurate, not necessarily precise to the nth degree.