Saturday, April 26, 2008
But the big question is WHY OPTIMIZE? A media buyer would say - I was given a budget of $10 MM and I bought the most effective media. Well, the buyer bought the media which he/she thought would be most effective but what happened in reality?
If the media is reaching out the consumers at a frequency of 15 and optimal is 10, then 5 impressions per unique user are wasted. Now, in terms of $ if the average cost per 5 impressions is $.01 (assuming $2 CPM) then we are wasting $.01 per consumer. If the campaign is reaching 50 MM unique individuals, then the loss is $500,000.
Yes, $500,000 in wasted impressions - money used to reach out to consumers who have already seen your ad 10 times.
So, as per my previous posts there are many advertisers who if not paying attention to frequency are wasting impressions and increasing their cost per acquisition. Personally, I believe this was/is the case with Vonage, Netflix, Zecco, Sarah Marshall (the movie which is about to be released).
In the online world, cookies should be used to identify if this is an existing customer or not and then try to up-sell or cross-sell their products.
Similar math can be done to estimate the wasted impressions based on day parting (what time of the day does your market segment most active), day of the week, Geographically etc.
Another example is engagement, move impressions out of placements which have a lower consumer engagement (click-through for online, phone call rate for DRTV etc). Move the impression to places where there is higher engagement. Make the most out of your marketing $, it is money which could be used for a new product launch or your next big campaign like holiday campaign in Q4.
It is very important to do a mix-media modeling if you are using various channels (Out of home - Cabs, Billboards, buses, trains, Online, Search, TV) to find out the overlap of consumers within each of these channels. This could be hard but hitting the same consumer again and again from all channels - NOT A VERY GOOD IDEA :). In simple terms - all the media buyers for various channels should talk before planning, estimate reach and frequency, if possible research the day parts and then launch the campaign.
Thursday, April 24, 2008
I was driving in downtown Chicago (taking my wife to the Yoga class), and guess what - I start seeing Zecoo trading's ads. First ad - I said "Hey, the brokerage account that I use for trading, cool!", the next minute I see 3 cabs with the same ad for Zecco trading, I thought great! they are really try to acquire new customers. Believe it or not, in my round trip of less than 2 miles, I saw 8 Zecco trading ads. At the end of the drive, I was like OKAY! I get it, Zecco trading is offering 10 free trades and I use them every month. Forgot to mention... I see their online banner ads during the day at work.
The above is a classic example of over saturating the market with ads. Advertisers need to realize that there are only "n" number of consumers out there in the market. By spending more money, we cannot increase our market size. The more money the advertisers spend, their expectations for returns also go up. Is that justified? No! By showing more ads, you are reaching the same consumer audience and you cannot force them to start using your service or buy your product.
In my experience in the online world, 4-6 Ads bi weekly per user should be the ideal frequency. If a consumer does not convert due to exposure 4-5 times, chances that he will convert with 22 views are also very slim :)
Wednesday, April 23, 2008
There will never be two systems which will track exactly the same. There will always be a discrepancy among the two, but the two big things are to identify what the discrepancy is and why does it exist.
Once identified the systems should track in the same direction - i.e. if one shows 20% increase in revenue the other should indicate the same.
In my experience, I think the best way to identify these issues is to map the data flows for both the systems. Once the data flow is mapped, then identity how much difference exists at each step. Also, it is possible that one system must have a longer cookie window than the other - this is a very common issue I have seen in the Online Marketing arena. So, these things could impact a lot.
One more thing I have noticed is that - sometimes systems are set to track a different level. For example: System A could be tracking at creative level and System B could be tracking at placement level. System A could be capturing only certain transactions (example - only certain product categories) but System B could be tracking everything. System A could be tracking only post-click data but System B could be tracking both post-click and post-impression.
One of the common example is trying to match Google Analytics Site data to DART Spotlight tag data. Yes, both of them are same company now but they track differently and never track the same. Some of the variables could be: session time, tags not firing correctly, tags not firing every time etc etc.
It can be a tedious process to identify the differences but once identified the reporting/measurement process becomes so much easy....
Tuesday, April 22, 2008
Below, I take an example of only 5 data points but with 15-20 data points all over the place, it is hard to understand the data and derive insights.
In the first chart, it is easy to make out that impressions and CTR is not a 1:1 co-relation but in the lower chart where the Impressions are sorted with highest to lowest, you can see a clear inverse relation - the higher the impression the lower the CTR. Thus, if a media planner were to look at this data, it would come to mind to move the impressions from the lower CTR sites to the higher ones where the CTR is much higher.
But thanks to DART's DoubleClick Creative Optimizer you can set the the metric you want to optimize on - Click-through rate, conversion rate, revenue etc. What the optimizer does is that it dynamically moves the impressions to the creative which is delivering higher click-through rate, conversion rate or revenue...and the great thing about it is that it works at placement level. So, if on one particular placement creative A is performing better - it will get more impressions and if on another placement creative B is performing better - it will get higher impressions. One more good thing about this is that you can set a minimal threshold, example serve a minimum of 10% of total impression for each creative.
However, there are some drawbacks with every good system, one of the biggest that I have noticed is that you cannot start with a set creative split. It always starts with an even split. So, if you have 2 creatives it starts with a 50/50 split.
But hey, optimizing at creative and placement level and that too AUTOMATED, I will take it any day.
Monday, April 21, 2008
A/B testing is essentially testing creatives - for the campaign. This could be applied to almost any form of advertising - Display, Site, Newsletters. The basic principle is keep an even split among all the creatives and try to measure the impact on the action - click through rate or conversion event. Some people think it is for only 2 creatives but one can use it for multiple 2,3, 4,5....
One important thing is that the sample size - it should be big enough to provide statistically significant results (t-test).
The first time I did a A/B test was a HomePage design for one of my clients. We were trying to measure which one of the two HomePage designs drove higher consumers to better use the entire website. Typically, on a Homepage you want information for existing customers, knowledge center and an acquisition section (to attract new customers). The banking websites do a pretty good job of this.
Since, then I have tested multiple forms of creative (display), emails (different forms of emails newsletters and then measuring the call to action rate), websites.
As I see - A dashboard is a graphical representation of the key success and diagnostic metrics used to measure the performance of the campaigns.
I was browsing the internet and found this great site which talks about Dashboards - http://chiefmarketer.com/crm_loop/roi/marketing_dashboard/
Personally, I have built a lot of them for various types of campaings - Direct Marketing, Branding, Channel Performance (email vs. phone vs. display vs. DRTV), Site Performance, Product Performance (product A vs. product B)
As an example the table below shows all the key metrics week over week but it is like staring at 100s of numbers and it is hard to make business sense out of these. What exactly is going on...
Friday, April 18, 2008
Just imagine looking at a dashboard which does not show any week over week trends. Just a weekly refresh showing the previous week's results.
It would be obvious to me why the final recipients of the dashboard would not be happy, what is missing? I would not be able to get the answers to questions like - how was this metric 3 weeks ago, make me realized a ah! Week over Week trending is missing.
Just imagine this data shown in 5 separate charts, how will you derive any business insights out of this? But looking at the chart above you can clearly see that there was a clear upward trend until week 3. WHY? Probably seasonality or launch of campaign ...Why sudden drop in week 4 and then again an upward trend in week 5. Was week 4 just an anomaly or was there really a reason for this drop?
All these questions are sparked looking at week over week trends.
Thursday, April 17, 2008
Not even CEO or CMO, presenting data (1,000s of rows) to someone who is not playing with the data on a day to day basis is not fair.
One would then argue oh! I will summarize the data and bring it down to a table. My thought would be great, but there is something called Data Visualization. Let's take an example, below is a table showing spend and revenue. The ROAS (Return on Advertising Spend) in the table is defined as Revenue/Spend.
This chart shows the % spend by site and the % revenue generated due to advertising at each of those sites. It is very easy to visualize that MSN was the most efficient site in terms of generating revenue. MySpace had the lowest efficiency.The chart below is another way of showing the spend efficiency by site.
Aren't the two charts a better way to look at the data?
Wednesday, April 16, 2008
Success Metric - A metric which the CEO or CMO would care about. Example: Total new clients or revenue
Diagnostic Metrics - A metric which is used to explain or diagnose the success metric if it over achieved or under achieved. Example: Click-through rate, Interaction rate, Landing page visits
I feel it is very important to put them in the measurement plan and also realize if it is feasible to calculate them. The reason I talk about feasibility is that many time due to technological limitations, it is not possible to get all the data required to calculate the metrics.
For example our friends at Google do not accept third party vendors (example: DART) to serve display ads (banner) and thus it is hard to get the number of impressions and clicks or any other post-impression, post-click activity for the banners served through the Google ad network. Yes, I do know that Google just bought DART - and trust me until this day, they do not accept third party tags.
I have been in various situation where earlier it seems feasible to get the data but when the campaign really launches, it becomes hard or impossible to get the data. OR sometimes the correct tags are not put in place which enable to get the data in the correct form.
Tuesday, April 15, 2008
Now lets lets take another look:
This flow is showing each and every step that a consumer will go through before being able to sign-up for the newsletter. One can measure a drop-off rate at each of the above steps and then be able to put a cork on the leak, helping increase the "sign-up rate."
The above example is a very simple flow - at times there are about 10-20 steps in the complete process. I have had the privilege of looking at a flow of over 100 steps as well, it was was a check out process for a leading e-commerce company.
As with every step of measurement, a many people do not believe in mapping a complete consumer flow but trust me it is one of the most important steps.
Quick Tip: Use Microsoft Visio to draw consumer flow in minutes
This document is often skipped and this causes problem like the various parties involved in the campaign launch are not on the same page. One example is: The agency and the client being on the different page in terms of campaign objectives, KPIs etc
This document ensures that all the stakeholders of the campaign (agency, brand manager, product manager, marketing team, advertising team) understand that campaign objectives, consumer flow, KPIs, reporting needs, dashboard structure and the reporting frequencies.
What constitutes the plan:
- Campaign Overview
- Consumer Flow
- Tracking mechanisms like DART tags, Webtrends tags, Google Analytics, Omniture Tags, Hitbox tags, email vendor tracking etc
- Diagnostic Metrics - example: CTR, Interaction Rate, Email open rate, Email click rate, Phone answer rate
- Success Metrics - example: ROAS, ROI, Conversion Rate
- Dashboard - showing the key metrics help to understand the performance of the campaign. Data representation is an very important part the process. I will be providing some illustrations soon.
- Reporting Frequencies
Monday, April 14, 2008
- Define the campaign objectives
- Define the key performance indicators i.e. success metrics
- Anticipate or forecast the results, example acquire 100,000 new clients or generate $1,000,000 in revenue or generate a ROI of $5
- For online marketing - make sure the tracking is in place to get the accurate data
- Analyze the data and Report Results
And the circle of Measurement continues….
You cannot quantify everything in terms of ROI i.e. how much revenue did we generate by spending $X. There are many other ways to measure marketing:
- Brand - How many more people know about my brand i.e. did the awareness of my brand increase? One can measure this by conducting surveys for example using Dynamic Logic.
- Increasing visits to your website is an indicator that your marketing dollars are paying off.
- Increased unique visitors, time spend on the website are some other metrics which can be used
Many a times as an advertiser, one needs to justify why does he/she needs that $10MM budget. OR what we also call as Business case.
Sometimes one needs justify in terms on "$" value that spending $10MM would help the business's bottom line by $20MM. Based on each business model, there are different ways how this can be done.
The simplest and easiest way is to use a ROI model i.e. Revenue/Spend to get the amount of revenue generated per $1 spent in marketing.
If ROI is not the measure, which is actually not possible in a lot of cases where there is no e-commerce platform, there need to be other ways to create business models. For example: If there is a action that an advertiser wants the consumers to take - it would be a good idea to attribute a "$" value to that action. This action could be "find a store" or "contact us". So, if you know that for every find a store, a consumer would make a $100 purchase, you can attribute revenue to each "find a store" and thus be able to work on your business case.
I currently work at one of the Ad Agencies in Chicago, IL, USA.
Professionally, I have two Masters Degrees - one in Electrical and Computer Engineering and the other in Management Information Systems from the University of Illinois at Chicago (UIC). While, at UIC I worked for an online grocery store providing hard to find food items from all over the world (www.ethnicgrocer.com) where I managed the complete operations (ordering goods, forecasting, customer service, analytics, and logistics). I did understand the basics of running a business at that shop, and still remember every day I worked there. My gurus Parry Singh and Subash Bedi did do an awesome job of training me.
After graduating from UIC, I started working at TNS NFO Research where I was involved in forecasting the sales new products i.e. tell the clients like P&G, Kellogg Snacks how many units of their products would sell if they were to launch. It was fun working on products like Tide, Keebler cookies...
In July 2005, I did took the big step into the exiting world of Online Advertising - working for an full service agency in Chicago. Learnt a lot there while working in their measurement team - "measured" email campaigns, website performance, online banners, search, DRTV. It was fun!!!
In Feburary 2007, I moved over to another agency in Chicago and since then have measured Billions and Billions of ad impressions. I was tasked to set up the analytics platform for one of the largest Online Advertisers in the country. Started with creating weekly dashboards, led the client to know more about their seasonality curves - conversion rates, click rates. Worked with Datafeed, search, display, DRTV.
Net net - I do have about 5+ years of consumer analytics, across various industries, clients, channels. Measuring marketing efforts all over.....