The case of "WASTED" impressions

In my earlier posts, I have talked about optimizing the media in various forms like - Frequency, Creative Optimizer. There are more ways to optimize - Geographically, day parting, week of the day etc.

But the big question is WHY OPTIMIZE? A media buyer would say - I was given a budget of $10 MM and I bought the most effective media. Well, the buyer bought the media which he/she thought would be most effective but what happened in reality?

If the media is reaching out the consumers at a frequency of 15 and optimal is 10, then 5 impressions per unique user are wasted. Now, in terms of $ if the average cost per 5 impressions is $.01 (assuming $2 CPM) then we are wasting $.01 per consumer. If the campaign is reaching 50 MM unique individuals, then the loss is $500,000.

Yes, $500,000 in wasted impressions - money used to reach out to consumers who have already seen your ad 10 times.

So, as per my previous posts there are many advertisers who if not paying attention to frequency are wasting impressions and increasing their cost per acquisition. Personally, I believe this was/is the case with Vonage, Netflix, Zecco, Sarah Marshall (the movie which is about to be released).

In the online world, cookies should be used to identify if this is an existing customer or not and then try to up-sell or cross-sell their products.

Similar math can be done to estimate the wasted impressions based on day parting (what time of the day does your market segment most active), day of the week, Geographically etc.

Another example is engagement, move impressions out of placements which have a lower consumer engagement (click-through for online, phone call rate for DRTV etc). Move the impression to places where there is higher engagement. Make the most out of your marketing $, it is money which could be used for a new product launch or your next big campaign like holiday campaign in Q4.

It is very important to do a mix-media modeling if you are using various channels (Out of home - Cabs, Billboards, buses, trains, Online, Search, TV) to find out the overlap of consumers within each of these channels. This could be hard but hitting the same consumer again and again from all channels - NOT A VERY GOOD IDEA :). In simple terms - all the media buyers for various channels should talk before planning, estimate reach and frequency, if possible research the day parts and then launch the campaign.

Reach and Frequency

I often report numbers showing 22+ frequency and always wonder if it really true? Why are we showing the same Ad to the same person 22 times. Something is wrong with the data... but, if it is for every campaign it must be true. Probably, that is how it is...OK.


I was driving in downtown Chicago (taking my wife to the Yoga class), and guess what - I start seeing Zecoo trading's ads. First ad - I said "Hey, the brokerage account that I use for trading, cool!", the next minute I see 3 cabs with the same ad for Zecco trading, I thought great! they are really try to acquire new customers. Believe it or not, in my round trip of less than 2 miles, I saw 8 Zecco trading ads. At the end of the drive, I was like OKAY! I get it, Zecco trading is offering 10 free trades and I use them every month. Forgot to mention... I see their online banner ads during the day at work.


The above is a classic example of over saturating the market with ads. Advertisers need to realize that there are only "n" number of consumers out there in the market. By spending more money, we cannot increase our market size. The more money the advertisers spend, their expectations for returns also go up. Is that justified? No! By showing more ads, you are reaching the same consumer audience and you cannot force them to start using your service or buy your product.


In my experience in the online world, 4-6 Ads bi weekly per user should be the ideal frequency. If a consumer does not convert due to exposure 4-5 times, chances that he will convert with 22 views are also very slim :)

Different systems used to measure same metrics

Many times there is more than one system used to track the same metrics. Many companies use their legacy internal systems to track revenue, margin, visits to their site, which they do not want to give up and also use the new systems like DART, Atlas to track the same metrics.

There will never be two systems which will track exactly the same. There will always be a discrepancy among the two, but the two big things are to identify what the discrepancy is and why does it exist.

Once identified the systems should track in the same direction - i.e. if one shows 20% increase in revenue the other should indicate the same.

In my experience, I think the best way to identify these issues is to map the data flows for both the systems. Once the data flow is mapped, then identity how much difference exists at each step. Also, it is possible that one system must have a longer cookie window than the other - this is a very common issue I have seen in the Online Marketing arena. So, these things could impact a lot.

One more thing I have noticed is that - sometimes systems are set to track a different level. For example: System A could be tracking at creative level and System B could be tracking at placement level. System A could be capturing only certain transactions (example - only certain product categories) but System B could be tracking everything. System A could be tracking only post-click data but System B could be tracking both post-click and post-impression.

One of the common example is trying to match Google Analytics Site data to DART Spotlight tag data. Yes, both of them are same company now but they track differently and never track the same. Some of the variables could be: session time, tags not firing correctly, tags not firing every time etc etc.

It can be a tedious process to identify the differences but once identified the reporting/measurement process becomes so much easy....

Sort Sort Sort....

Sorting is such a simple thing, but it helps so much in interpreting the data and making business sense and insights.


Below, I take an example of only 5 data points but with 15-20 data points all over the place, it is hard to understand the data and derive insights.


In the first chart, it is easy to make out that impressions and CTR is not a 1:1 co-relation but in the lower chart where the Impressions are sorted with highest to lowest, you can see a clear inverse relation - the higher the impression the lower the CTR. Thus, if a media planner were to look at this data, it would come to mind to move the impressions from the lower CTR sites to the higher ones where the CTR is much higher.

Creative Optimizer

If we think about large advertisers - advertising over a billion impressions per month, optimization would be very hard or next to impossible. Possibly, there would be over 1,000 live placements and if there is more than 1 creative...GOOD LUCK with that.

But thanks to DART's DoubleClick Creative Optimizer you can set the the metric you want to optimize on - Click-through rate, conversion rate, revenue etc. What the optimizer does is that it dynamically moves the impressions to the creative which is delivering higher click-through rate, conversion rate or revenue...and the great thing about it is that it works at placement level. So, if on one particular placement creative A is performing better - it will get more impressions and if on another placement creative B is performing better - it will get higher impressions. One more good thing about this is that you can set a minimal threshold, example serve a minimum of 10% of total impression for each creative.

However, there are some drawbacks with every good system, one of the biggest that I have noticed is that you cannot start with a set creative split. It always starts with an even split. So, if you have 2 creatives it starts with a 50/50 split.

But hey, optimizing at creative and placement level and that too AUTOMATED, I will take it any day.

A/B Testing - Display Ad/Site/Newsletter...

Most of the analytics peeps keep talking in their jargon, one of the favorite is A/B Testing. The marketing folks most times start wondering what are they talking about.

A/B testing is essentially testing creatives - for the campaign. This could be applied to almost any form of advertising - Display, Site, Newsletters. The basic principle is keep an even split among all the creatives and try to measure the impact on the action - click through rate or conversion event. Some people think it is for only 2 creatives but one can use it for multiple 2,3, 4,5....

One important thing is that the sample size - it should be big enough to provide statistically significant results (t-test).

The first time I did a A/B test was a HomePage design for one of my clients. We were trying to measure which one of the two HomePage designs drove higher consumers to better use the entire website. Typically, on a Homepage you want information for existing customers, knowledge center and an acquisition section (to attract new customers). The banking websites do a pretty good job of this.

Since, then I have tested multiple forms of creative (display), emails (different forms of emails newsletters and then measuring the call to action rate), websites.

Dashboards...

I am sure people in marketing have been hearing this term -"Dashboard" and a lot of time people wonder what is it?

As I see - A dashboard is a graphical representation of the key success and diagnostic metrics used to measure the performance of the campaigns.

I was browsing the internet and found this great site which talks about Dashboards - http://chiefmarketer.com/crm_loop/roi/marketing_dashboard/

Personally, I have built a lot of them for various types of campaings - Direct Marketing, Branding, Channel Performance (email vs. phone vs. display vs. DRTV), Site Performance, Product Performance (product A vs. product B)

As an example the table below shows all the key metrics week over week but it is like staring at 100s of numbers and it is hard to make business sense out of these. What exactly is going on...

Now, looking at the dashboard below (same data as the above table) helps understand all the key metrics in a clear and consise way thus helping make better business decisions and also come up with questions - why did a particular metric drop or rise.

Data Trending

I just realized a few weeks ago the importance of trending data. I had always set up dashboards to show trends, so it was always second nature to me - looking at trends.

Just imagine looking at a dashboard which does not show any week over week trends. Just a weekly refresh showing the previous week's results.

It would be obvious to me why the final recipients of the dashboard would not be happy, what is missing? I would not be able to get the answers to questions like - how was this metric 3 weeks ago, make me realized a ah! Week over Week trending is missing.

Just imagine this data shown in 5 separate charts, how will you derive any business insights out of this? But looking at the chart above you can clearly see that there was a clear upward trend until week 3. WHY? Probably seasonality or launch of campaign ...Why sudden drop in week 4 and then again an upward trend in week 5. Was week 4 just an anomaly or was there really a reason for this drop?

All these questions are sparked looking at week over week trends.

Data Representation/Visualization

Data can be seen in many forms but how easy it is to interpret to make business decisions is the key. A CEO or CMO cannot be presented to look at 100s or 1,000s of rows of data in excel or a SAS output which even analysts find it hard to read.

Not even CEO or CMO, presenting data (1,000s of rows) to someone who is not playing with the data on a day to day basis is not fair.

One would then argue oh! I will summarize the data and bring it down to a table. My thought would be great, but there is something called Data Visualization. Let's take an example, below is a table showing spend and revenue. The ROAS (Return on Advertising Spend) in the table is defined as Revenue/Spend.

This shows the data but it is not very easy to visualize in a table. Now, look at the charts below.

This chart shows the % spend by site and the % revenue generated due to advertising at each of those sites. It is very easy to visualize that MSN was the most efficient site in terms of generating revenue. MySpace had the lowest efficiency.

The chart below is another way of showing the spend efficiency by site.
Aren't the two charts a better way to look at the data?

Diagnostic and Success Metrics

It took me a while to figure out what is the difference in these two metrics and one day I was explained in a very nice way, which was embedded in my mind.

Success Metric - A metric which the CEO or CMO would care about. Example: Total new clients or revenue
Diagnostic Metrics - A metric which is used to explain or diagnose the success metric if it over achieved or under achieved. Example: Click-through rate, Interaction rate, Landing page visits

I feel it is very important to put them in the measurement plan and also realize if it is feasible to calculate them. The reason I talk about feasibility is that many time due to technological limitations, it is not possible to get all the data required to calculate the metrics.

For example our friends at Google do not accept third party vendors (example: DART) to serve display ads (banner) and thus it is hard to get the number of impressions and clicks or any other post-impression, post-click activity for the banners served through the Google ad network. Yes, I do know that Google just bought DART - and trust me until this day, they do not accept third party tags.

I have been in various situation where earlier it seems feasible to get the data but when the campaign really launches, it becomes hard or impossible to get the data. OR sometimes the correct tags are not put in place which enable to get the data in the correct form.

Consumer Flow

The image above shows the consumer flow or consumer experience to sign-up for a Newsletter. It is very important to capture each and every step of the process to be able to understand - "At what stage or step do we have the highest drop-off" But looking at the above flow, do you think one will be able to get a complete understanding? Some people might say - Yes, of course why not. What do you think?

Now lets lets take another look:

This flow is showing each and every step that a consumer will go through before being able to sign-up for the newsletter. One can measure a drop-off rate at each of the above steps and then be able to put a cork on the leak, helping increase the "sign-up rate."

The above example is a very simple flow - at times there are about 10-20 steps in the complete process. I have had the privilege of looking at a flow of over 100 steps as well, it was was a check out process for a leading e-commerce company.

As with every step of measurement, a many people do not believe in mapping a complete consumer flow but trust me it is one of the most important steps.

Quick Tip: Use Microsoft Visio to draw consumer flow in minutes

Measurement Plan

Writing up a Measurement Plan is very important especially for the skeptics. As I have learnt over the years it helps explain the "WHY MEASURE" and the "HOW TO MEASURE" of measurement process for a campaign.

This document is often skipped and this causes problem like the various parties involved in the campaign launch are not on the same page. One example is: The agency and the client being on the different page in terms of campaign objectives, KPIs etc

This document ensures that all the stakeholders of the campaign (agency, brand manager, product manager, marketing team, advertising team) understand that campaign objectives, consumer flow, KPIs, reporting needs, dashboard structure and the reporting frequencies.

What constitutes the plan:

1. Campaign Overview
2. Consumer Flow
3. Tracking mechanisms like DART tags, Webtrends tags, Google Analytics, Omniture Tags, Hitbox tags, email vendor tracking etc
4. Diagnostic Metrics - example: CTR, Interaction Rate, Email open rate, Email click rate, Phone answer rate
5. Success Metrics - example: ROAS, ROI, Conversion Rate
6. Dashboard - showing the key metrics help to understand the performance of the campaign. Data representation is an very important part the process. I will be providing some illustrations soon.
7. Reporting Frequencies

Happy Measuring....

Measurement Process

Yes, like for everything else there is a process which helps in measuring accurately. Just as a top line, the following steps are very important:

1. Define the campaign objectives
2. Define the key performance indicators i.e. success metrics
3. Anticipate or forecast the results, example acquire 100,000 new clients or generate $1,000,000 in revenue or generate a ROI of $5
4. For online marketing - make sure the tracking is in place to get the accurate data
5. Analyze the data and Report Results

And the circle of Measurement continues….

Why & How Measure?

The first thought to everyone's mind is why do we need to measure? Well, so I guess a very simple answer is that - you measure so that you know what you got out of your marketing campaign or marketing dollars. If you are a brand manager, you need to justify that spend. If one spends $1 MM what do you get out of it?

You cannot quantify everything in terms of ROI i.e. how much revenue did we generate by spending $X. There are many other ways to measure marketing:

1. Brand - How many more people know about my brand i.e. did the awareness of my brand increase? One can measure this by conducting surveys for example using Dynamic Logic.
2. Increasing visits to your website is an indicator that your marketing dollars are paying off.
3. Increased unique visitors, time spend on the website are some other metrics which can be used

Many a times as an advertiser, one needs to justify why does he/she needs that $10MM budget. OR what we also call as Business case.

Sometimes one needs justify in terms on "$" value that spending $10MM would help the business's bottom line by $20MM. Based on each business model, there are different ways how this can be done.

The simplest and easiest way is to use a ROI model i.e. Revenue/Spend to get the amount of revenue generated per $1 spent in marketing.

If ROI is not the measure, which is actually not possible in a lot of cases where there is no e-commerce platform, there need to be other ways to create business models. For example: If there is a action that an advertiser wants the consumers to take - it would be a good idea to attribute a "$" value to that action. This action could be "find a store" or "contact us". So, if you know that for every find a store, a consumer would make a $100 purchase, you can attribute revenue to each "find a store" and thus be able to work on your business case.

Who am I ?

Hi - I thought the first post should be about me, who am I - what do I do, where do I come from and why am I writing this blog.

I currently work at one of the Ad Agencies in Chicago, IL, USA.

Professionally, I have two Masters Degrees - one in Electrical and Computer Engineering and the other in Management Information Systems from the University of Illinois at Chicago (UIC). While, at UIC I worked for an online grocery store providing hard to find food items from all over the world (www.ethnicgrocer.com) where I managed the complete operations (ordering goods, forecasting, customer service, analytics, and logistics). I did understand the basics of running a business at that shop, and still remember every day I worked there. My gurus Parry Singh and Subash Bedi did do an awesome job of training me.

After graduating from UIC, I started working at TNS NFO Research where I was involved in forecasting the sales new products i.e. tell the clients like P&G, Kellogg Snacks how many units of their products would sell if they were to launch. It was fun working on products like Tide, Keebler cookies...

In July 2005, I did took the big step into the exiting world of Online Advertising - working for an full service agency in Chicago. Learnt a lot there while working in their measurement team - "measured" email campaigns, website performance, online banners, search, DRTV. It was fun!!!

In Feburary 2007, I moved over to another agency in Chicago and since then have measured Billions and Billions of ad impressions. I was tasked to set up the analytics platform for one of the largest Online Advertisers in the country. Started with creating weekly dashboards, led the client to know more about their seasonality curves - conversion rates, click rates. Worked with Datafeed, search, display, DRTV.

Net net - I do have about 5+ years of consumer analytics, across various industries, clients, channels. Measuring marketing efforts all over.....