Intro to Incrementality Series: Part III

by | Jun 27, 2022 | CTV/OTT

Recap

In Part 1 of our Intro to Incrementality series, we went through the basics of incrementality analysis. We talked about our two groups, test (or exposed) and control (or holdout), and how this type of analysis measures the lift caused by a specific variable of one over the other, thereby attaining a true measure of that variable’s impact. In Part 2, we homed in on that control (or holdout) group and explained exactly how to go about creating that group through ghost bidding in a way that avoids skewing the incrementality analysis. So, we’ve covered the basics and we’ve made sure that by the time we get to our incrementality results, we can trust that there’s no skew or bias.

In this last installment, we will discuss the final, and perhaps most important, part of this process: putting this analysis to work in real time. To get to the heart of it, let’s look back on our initial basketball example:

An NBA basketball team has two players who both make 40% of their foul shots. The team wants to improve that percentage, so it decides to hire a shooting coach. It designs a test to evaluate that coach, and assigns him to only one of the two players. Both players are told to do everything else they had been doing, exactly as they had been doing it. For instance, they’re told to spend the same amount of time in the gym, keep asimilar diet, and maintain their same weight. After a year, the player who worked with the shooting coach makes 80% of his free throws the following season, while the player not assigned the shooting coach makes 50% of his free throws.

While this example has been useful in demonstrating the basics of incrementality, the test (the player assigned the coach), the control (the player not assigned the coach), and the new variable (the coach), it neglects some crucial real-world implications.

For starters, if something appears to be working, it doesn’t necessarily benefit the team to wait a whole season to evaluate exactly how well it’s working. If that coach got assigned to the control player mid-way through the season, the results might differ, but it’s quite likely the team could have boosted two players’ free throw percentage instead of one. It’s also worth taking a look at exactly what the coach is doing. Is it just the extra repetitions demanded by the coach that are causing the improvement? Is it an adjustment in form? Is it some mental component or confidence boost? Finally, and most importantly, this coach represents an investment. And the team is paying. The team has to determine if that investment is justified by the incremental improvement. If it is, it has to then determine whether should it increase that investment, and how. Our example simplifies a problem that, in basketball or marketing, is quite messy. The point of this final installment is to discuss cleaning up that mess.

 

Putting It All Together

1. In marketing, incrementality analysis should be ongoing and in real time. Marketers don’t have an unlimited budget or the luxury of conducting tests in a vacuum. It’s important to note that what incrementality results look like after two weeks might be different than what they look like after two months, but that doesn’t invalidate the two-week results. It’s a continual process of data collection and analysis that should inform decision making. What decision making? We’ll get there shortly.

2. Incrementality analysis is often conducted at the media type level. In our former marketing example, we discussed determining the incremental impact of adding a CTV campaign to a larger marketing mix. The reality is that the CTV campaign very likely consisted of several streaming services, maybe Sling, Hulu, and Pluto, and several creatives, maybe a:30 second creative and two :15 second creatives, across more than one audience, maybe an intent-based audience and a demographic-based audience. When we conduct this type of analysis, it’s important to get more granular than just the overall media type to unearth additional valuable insights.

3. This will come as no surprise to anyone, but paid media costs money. We cannot, and should not, treat this analysis as independent of cost.

 

CTV Test Campaign Example

How do we put this analysis to work in real time, granularly, and factoring in cost? We apply it to campaign optimization. Here’s another marketing example:

A brand decides to add a $1k CTV test to their marketing mix that previously consisted only of search and social media campaigns. The brand’s goal is to optimize toward the lowest cost-per-checkout (CPC) possible for its CTV campaigns. The brand has only one creative and is testing only one intent-based audience, but it doesn’t want to put all its eggs in one basket, so it decides to test three publishers, Sling, Hulu, and Pluto TV.

Most performance CTV vendors don’t report incremental conversions, so the brand observes the following checkouts and cost-per-checkout across the variables in the CTV campaign. 

 

 

 

 

Any brand that sees those results would think: “well, it looks like Sling is the best, we should put more budget there and less budget in Hulu and Pluto TV”—and, in a vacuum, the brand would be absolutely correct. But media doesn’t work in silos, it works across silos, and the brand is also running search and social, plus, it’s got all this organic demand it worked so hard to build up.

The brand, knowing this, decides to add incrementality analysis as an additional data point, and it finds that Sling’s incrementality percentage is 10%, Hulu’s is 80%, and Pluto TV’s is 50%. In other words, it finds that 90% of conversions recorded from Sling would have happened despite those Sling exposures, 20% of conversions recorded from Hulu would have happened despite those Hulu exposures, and 50% of those conversions recorded from Pluto TV would have happened despite those Pluto exposures. 

 

This is worrisome and tricky when it comes to future budget allocation. What the brand is seeing is a commonplace occurrence in the marketing world: conversions reported by platforms are duplicative because each platform the brand operates in works only with the media it runs. So the brand’s CTV vendor takes credit for its social media conversions, the brand’s search vendor takes credit for its CTV conversions, and so on. 

But the brand has a secret weapon: incrementality-informed optimization. Instead of using only CPA metrics, the brand can, very simply, apply the incrementality analysis to the cost-per-checkout and the result is a new metric: cost-per-incremental-checkout (iCPC). By multiplying the number of checkouts and the incrementality percentage, the brand unearths the below results:

 

The takeaway? Without incrementality analysis applied to the brand’s performance numbers, it would have been optimizing its CTV campaign in a way that was actually counter to its bottom line, toward conversions that would have happened anyway. By adding this additional analysis, it can actually see it would be best served spending more on Hulu, and that the top performer from only a CPC standpoint (Sling) actually finishes well behind the top performer from an iCPC standpoint (Hulu).

 

In Conclusion

As brands get smarter with their budget allocation across and within media types, incrementality analysis becomes a crucial stepping stone on the path to profitability and cross-channel ROAS. 

Flip, our performance CTV platform, not only offers incrementality analysis but also allows brands the option to leverage incrementality-informed optimization strategies, ensuring that their CTV dollars are getting put to work efficiently within a cross-channel media mix. To learn more speak to a member of our team today.

Mike Seiman

Mike Seiman, CEO & Chairman, is the founder of Digital Remedy, a digital media solutions company leading the tech enabled marketing space he co-founded while still a college student at Hofstra University in the early 2000s. The company has grown quickly and is now a major player within the crowded digital advertising landscape. The rapid growth of Digital Remedy, formerly CPXi led to its inclusion on Inc. Magazine’s list of fastest growing privately held advertising/marketing companies in 2008, 2009, 2010 and 2014. Mike was selected as a semi-finalist in Ernst & Young’s Entrepreneur of the Year initiative in 2010 and 2013 and as a finalist in 2009 and 2014. In his free time, Mike serves on the Board of Trustees of his alma mater, Hofstra University. He also focuses on numerous philanthropic initiatives including sitting on the boards of the H.E.S. (Hebrew Educational Society non-profit community center) and Children International, where he spearheaded the development of community centers in both Guayaquil, Ecuador in 2010 and Barranquilla, Colombia in 2014.

David Zapletal

Before graduating in 2005 with degrees in Retail and Consumer Science (with an emphasis in eCommerce) and a Minor in Public Business Administration at the University of Arizona, David Zapletal had already successfully grown a start up ad network from serving an initial 1 million impressions per day to over 10 million impressions per day. It was his deep understanding of internet advertising during the industry’s beginning stages that led him to another start up at the time, CPXi. More than 8 years that have passed and Zapletal currently serves as Chief Operating Officer for Digital Remedy. In that role he continues to help grow and implement optimization tactics across various ad serving platforms, oversee daily operations of the account management and trafficking groups and maximizes ROI for Direct Response advertisers as well as for Publishers. Outside of Digital Remedy, Zapletal commits his efforts to an organization called Camp Dream Street, a camping program for children with disabilities, where he serves on the Board of Directors.

Jeff Reitzen

Jeff has worked in multiple facets of the online industry, from sales to operations as well as consumer engagement, content analytics and most recently in data optimization. His career began as a wedding, bar mitzvah, and Sweet 16 DJ where he learned the delicate balance of crowd energy management. Quickly, this skill made him incredibly successful in managing online sales for Geico. He joined CPXi at its startup stage as employee number 4 and has been a key driver of continued growth. His unique knowledge of what converts in the digital ad space, the application of data, and how to optimize platforms for efficiency continues to be invaluable to Digital Remedy clients. Today, Jeff is responsible for innovating and optimizing all Digital Remedy offerings including platforms, systems, tools, and internal processes—ensuring the organization remains on the precipice of the marketplace.

Mike Juhas

Mike Juhas, has over 13+ years of experience in ad tech client services, working with brands, agencies, and publishers ranging from top 10 advertisers to small regional organizations, to rep firms, holding companies and independent shops. An integral member of the Digital Remedy team, Juhas leads all client relationships, including facilitating onboarding and integration, establishing relationship protocols, overseeing Quarterly Business Reviews and status meetings, navigating financial coordination, and overseeing 24/7 team support. His specialties include consultative services, planning strategy, and account management disciplines. Juhas lives in New Canaan, CT with his wife and two daughters, and their dog, Perry–the unofficial company mascot.

TJ Sullivan

TJ Sullivan has over 20 years of media sales and leadership experience. His knowledge of the digital media landscape, ability to develop strategic solutions that solve brand challenges, and talent for motivating sales teams, have made him a vital member of several media and ad tech organizations. Before joining Digital Remedy, Sullivan was VP, Connections at iHeart Media, a cross-divisional group that enabled national advertisers to seamlessly work with multiple iHeart business units; CRO of Reelcontent, a video distribution company for brands; SVP of sales at AdoTube, a video ad network; and was Co-Founder and SVP of video measurement company, OpenSlate—for which he is still an advisor.

Sullivan had served as the President of 212, New York’s Interactive Advertising Club, and currently advises many early stage start-ups in the programmatic and video space. Notably, RUN (sold to Publicis in 2014), Futures Media, Transmit.Live, and Kubient.

Outside of his work in the media industry, Sullivan sits on the Board of St. Elizabeth School in Wyckoff, New Jersey and the Advancement Committee of St. Peter’s Prep in Jersey City, New Jersey. He resides in New Jersey with his wife and four children.

IP Zone: Leverage the relationship between users and the location of their IP addresses in a cookie-free, safe and scalable way.

Behavioral: Combine first party with network analytics and enormous scale to define custom audience channels that are optimized to sites with high brand engagement.

Demographic: Reach the right audience based on demographic characteristics such as age, gender and income.

Mobile Device: Target users by mobile device type (Smartphone, tablet, etc.), carrier and operating system.

Geo-Location: Target your audience residential cluster, proximity to retail locations, campaign level DMAs and more.

Contextual: Place a relevant ad in front of a user who is reading content that contains specific terms.

Buying Power and Quality

Let’s be honest. Your ad ops are limited by your access to only a handful of channels and DSPs. You want more reach and better prices, but don’t want to sacrifice quality in order to achieve those goals. Digital Remedy has access to a vast multitude of channels based on relationships we’ve cultivated over nearly 20 years in business. That means we get the best prices, have personal relationships, and don’t get sent to voicemail when we call.

It also means that we can execute omni-channel ad ops for better prices than an internal team, while ensuring the quality is up to industry standards. On top of access to all sorts of specific audiences we’re able to to leverage first- and third-party data across all your campaigns and pivot across platforms based on results. Any campaign, any budget, any platform, any audience. Digital Remedy backed by AdReady is restriction free ad ops….and what could be better than that?

Resources, Time, and Overhead

When was the last time you enjoyed balancing budgets, reading resumes, dealing with aggressive sales teams, or wasting years of time for small gains in performance. By partnering with Digital Remedy you get the full support of $30m in OPEX including marketing teams, sales teams, and a dedicated 24/7 ad operations team. No hiring new employees for media optimization, business development, or account management. No months of training and on-boarding. No long meetings crafting sales materials. Just your team focused on making deals, and our teams and tech focused on supporting and executing those deals in a tech-enabled, digital ecosystem designed to get the most out of any KPI.

Reporting and Support

How nice would it be to have all of your data and insights in one location. We don’t mean an excel sheet sent out once a week with complicated charts, or an XML file with pages and pages and pages of tables (what is this, 2003?). We mean a fully customizable dashboard reporting in real time, or as real time as possible. You get to decide the how, what, and when of the reporting you’re seeing. And that’s ALL of the how, what, and when’s. If you want to see breakouts of all of the individual campaigns in your system, done. If you want broad scope comparisons of all of the campaigns in the last year, done. If you want to see CTR’s for specific audiences and compare them to CPM for your best performing advertiser but limit the scope to campaigns greater than 30k, done. All of this is at your fingertips with the AdReady Dashboard, all in one place.

Jessica Cortapasso

With more than a decade of experience in human capital management, Jessica Cortapasso serves as SVP, People at Digital Remedy. After graduating from Muhlenberg College, she quickly recognized her passion for people and entered the workforce in Human Resources where she gained expertise in employee relations, designing strategic benefit plans, and the development, implementation, and curation of corporate engagement initiatives for big-name brands and small companies alike. Becoming a member of the Digital Remedy family in 2013 while simultaneously acquiring her Masters Degree in Human Resources Management and Development from New York University, Jessica has steered company culture through significant events ranging from acquisitions and a rebranding, to the development and application of our Core Values that shape our daily business practices. Cortapasso resides in Brooklyn, plays competitive volleyball, and loves spending time with her nieces.

Erez Feld

Responsible for the financial and legal practices of Digital Remedy, Erez brings 22 years of experience in precision financial analysis, growth management practices, strategic acquisition, and investment leadership. A graduate of Hofstra University, Erez began his career modeling for corporate finance, and expanded his accounting prowess in the real estate sector. Erez joined Digital Remedy in 2008 as a senior accountant, and helped to create and build an accounting department that could support the rapid growth of the company and aligned with those needs. Over the past 12 years he has evolved through various positions at the company within the finance discipline, supervising and mentoring additional finance personnel, while growing under the tutelage of Michael Fleischman, former CFO of Digital Remedy. Today, he leads the Finance Department by supporting high-level projects such as acquisitions and restructuring, and is responsible for overseeing all financial assets, establishing financial procedures, controls, and reporting systems.

Michael Fleischman

After a successful career as an accomplished Fortune 500 financial professional leading Corporate Finance and Strategic Planning at Cablevision Systems Corporation and its programming subsidiary Rainbow Media Holdings, Michael currently plays a role in the overall management of Digital Remedy including direct responsibility for all financial-related activities including accounting, financial planning, M&A, legal, insurance, real estate and banking relationships. Michael brings more than 25 years of media experience at Cablevision and Rainbow Media and during his career was instrumental in the launching and managing of a number of cable television networks including 10 Regional Sports Networks across the US, American Movie Classics, Bravo, and the Independent film channel as well as the structuring of corporate partnerships with companies including Liberty Media, NBC, Fox/NewsCorp and MGM. Additionally, he was the finance lead on a number of professional sports team acquisitions including Madison Square Garden, the successful IPO of Cablevision and a tracking stock at Rainbow Media.Michael was involved in the creation and launch of Rainbow Advertising Sales which was one of the Cable Industry’s first Local Advertising Sales Divisions.

Tony Pascal

With over two decades of experience in the design, product, and technology space, Tony joined Digital Remedy as a graphic designer in 2007. His responsibilities quickly expanded, landing him in leadership roles across multiple disciplines including creative direction, analytics, monetization optimization, and management of platform development. He continued to grow with the organization over the last fourteen years, overseeing all design, development, and execution of Digital Remedy products and platforms. In his current role as SVP, Product & Technology, Tony leads product and technology development for the company and acts as the go-to liaison across teams, ensuring alignment on all aspects of internal and client-facing technology initiatives.

Prior to Digital Remedy, Tony built and ran his own direct response company from 2002-2007 after graduating from New York Institute of Technology, where he learned the fundamentals of digital advertising and optimization strategies that still remain relevant today.

In a previous life, Tony was a ski instructor and still remains an avid skier today. When he is not leading product development, he can be found working on old cars, rock climbing and hitting the slopes.

Gayle Meyers

Gayle Meyers is an entrepreneur, venture partner, investor, and operating resource in the digital media and marketing industry, with over two decades of executive leadership experience. After launching a management consulting firm, Growthing which is focused on optimizing growth strategies for executive leaders and their organizations, Meyers has frequently been tapped for high-profile consulting and advisory positions to help marketing technology companies enhance their in-market presence.

“Gayle is widely recognized as a leading strategist with years of expertise in the ad tech space,” said Mike Seiman, Chairman and CEO of Digital Remedy. “Her career in discovering and integrating game-changing technologies in the marketing industry will serve as an invaluable resource as we continue to enhance our product suite in the months ahead.”

With expertise spanning multiple disciplines, Meyers frequently serves as a keynote speaker at industry conferences for companies such as Google, Verizon, Omnicom, LiveRamp, LinkedIn, Twitter, and Oracle. A list of her notable past clients who have benefitted from her unique insights to increase shareholder value includes Tinuiti (acquired by New Mountain Capital), Adometry (acquired by Google), MediaForge (acquired by Rakuten), Integral Ad Science (acquired by Vista Equity Partners), and Tapad (acquired by Telenor).

Matt Sotebeer

With 14+ years of experience in ad tech and emerging technologies, Matt Sotebeer brings an uniquely innovative approach in his role as Chief Strategy Officer at Digital Remedy. As CSO, Matt focuses on the intersection between the product, product marketing, tech, data, and sales teams, while fostering productive cross-functional company-wide relationships to inform and influence sales, educate clients, and optimize company performance.

Matt has an extensive knowledge in the integration of data science, creative, and media solutions to drive sustainable growth for companies, with a focus on designing customized, scalable solutions leveraging machine learning alongside human intelligence. Throughout the tenure of his career, he has successfully managed global teams aligned to common goals, encouraged collaborative problem solving, and supported talent growth for entrepreneurial companies including MiQ, Rocket Fuel, and Audience Science.

Jeremy Haft

A proven strategic, revenue, and team leader with over 20 years of experience managing and scaling revenue in the competitive ad tech landscape, Jeremy serves as Chief Revenue Officer at Digital Remedy.

Before joining the team in October 2022, he served as CRO at Channel Factory, where he reorganized the revenue team for sustainable growth and increased the sales team by 3x to drive predictable and more accountable revenue. Prior to that, he served in a decade of leadership positions. Most notably, as SVP of Sales at Amobee and as VP of North America Sales at Viant/Adelphic. At both organizations, Jeremy successfully built and scaled platform and business solutions from their infancy to achieve the desired corporate goals.

Jeremy graduated from The University of Vermont and currently resides in New Jersey with his wife and two children. In his spare time, he enjoys traveling to tropical locations, dining out, cooking feasts with friends, and any new fitness trend he can get his hands on.

Can we use your data to tailor ads for you? Our partners will collect data and use cookies for legitimate business purposes, ad personalization and measurement. We also collect anonymized personal data for legally necessary and legitimate business purposes. Read our PRIVACY POLICY.