Different systems used to measure same metrics

Many times there is more than one system used to track the same metrics. Many companies use their legacy internal systems to track revenue, margin, visits to their site, which they do not want to give up and also use the new systems like DART, Atlas to track the same metrics.

There will never be two systems which will track exactly the same. There will always be a discrepancy among the two, but the two big things are to identify what the discrepancy is and why does it exist.

Once identified the systems should track in the same direction - i.e. if one shows 20% increase in revenue the other should indicate the same.

In my experience, I think the best way to identify these issues is to map the data flows for both the systems. Once the data flow is mapped, then identity how much difference exists at each step. Also, it is possible that one system must have a longer cookie window than the other - this is a very common issue I have seen in the Online Marketing arena. So, these things could impact a lot.

One more thing I have noticed is that - sometimes systems are set to track a different level. For example: System A could be tracking at creative level and System B could be tracking at placement level. System A could be capturing only certain transactions (example - only certain product categories) but System B could be tracking everything. System A could be tracking only post-click data but System B could be tracking both post-click and post-impression.

One of the common example is trying to match Google Analytics Site data to DART Spotlight tag data. Yes, both of them are same company now but they track differently and never track the same. Some of the variables could be: session time, tags not firing correctly, tags not firing every time etc etc.

It can be a tedious process to identify the differences but once identified the reporting/measurement process becomes so much easy....

Measurement Plan

Writing up a Measurement Plan is very important especially for the skeptics. As I have learnt over the years it helps explain the "WHY MEASURE" and the "HOW TO MEASURE" of measurement process for a campaign.

This document is often skipped and this causes problem like the various parties involved in the campaign launch are not on the same page. One example is: The agency and the client being on the different page in terms of campaign objectives, KPIs etc

This document ensures that all the stakeholders of the campaign (agency, brand manager, product manager, marketing team, advertising team) understand that campaign objectives, consumer flow, KPIs, reporting needs, dashboard structure and the reporting frequencies.

What constitutes the plan:

1. Campaign Overview
2. Consumer Flow
3. Tracking mechanisms like DART tags, Webtrends tags, Google Analytics, Omniture Tags, Hitbox tags, email vendor tracking etc
4. Diagnostic Metrics - example: CTR, Interaction Rate, Email open rate, Email click rate, Phone answer rate
5. Success Metrics - example: ROAS, ROI, Conversion Rate
6. Dashboard - showing the key metrics help to understand the performance of the campaign. Data representation is an very important part the process. I will be providing some illustrations soon.
7. Reporting Frequencies

Happy Measuring....