TOP PUBLISHERS USE PUBNATION TO MANAGE AD QUALITY
read whyRequest A Demo
Thank you! Your submission has been received!
Something went wrong while submitting the form
Price FloorsOptimizationAd Exchanges
June 22, 2016

Price Floor Optimization Using Continuous Experimentation

In this blog post, we share our latest strategy for optimizing price floors. Our prior articles have showcased work we’ve done on price floor theory, busted auctions in ad exchanges, and the result...

Chris Cummings

In this blog post, we share our latest strategy for optimizing price floors. Our prior articles have showcased work we’ve done on price floor theory, busted auctions in ad exchanges, and the results of a simple A/B test that raised AdX revenue by 10%. The culmination of this work is a new price floor optimization strategy we call “Continuous Experimentation”, which hones in on the optimal price floors, while adapting to shifts in the market. The strategy also quantifies the value of the price floors over time. Using this strategy on our site, we’ve raised AdX revenue an average of 19%.

The Shifting Price Floor Conundrum

Most efforts to optimize price floors for particular segments of traffic start with the wrong question: “What is the optimal price floor?” Wrapped into that question is the assumption that there is one, optimal floor. While in reality, the best price floor optimization strategies recognize that setting price floors is a continuous process. Publishers know intuitively that the optimal price floor for December is likely to be a bust in January. But how do we know when the optimal price floor shifts? In our price floor optimization strategy, we set out to create an approach that could evolve with changes in the market.

The Goldilocks A/B Test

We are big believers in the power of a/b testing in ad ops. For our price floor experimentation, however, we realized that a simple a/b test--one price floor versus another price floor--would be inadequate. We’d never know for sure if the winning price floor was the optimal price floor because there would always be a chance that we should have tested a price slightly higher or slightly lower. To home in on the optimal floor--not to high, not too low, but just right--we decided that we needed to test a range of price floors, simultaneously. We created an experimental design that looked like this:

Price Floor Experimental Design

If a higher floor experimental group won, in the next round of tests, we’d shift the default floor up. If a lower floor experimental group won, we’d shift the default floor down. By leaving an experimental group without a price floor, we could ensure that our price floors continued to add value, and quantify the impact of the price floor at any point in time.

Continuous Experimentation

The big shift in our thinking occurred when we decided that we’d continuously run this experiment. That way if the market shifted up or down, we’d always be able to identify the shift and respond to it.

Identifying the optimal interval to update the price floors is a tricky question. In an ideal world, the price floors would be updated on very tight intervals, perhaps hourly, automatically. But with a shorter timeframe, less data is available, so the results may be spurious. Updating the floors also requires manpower, as most exchanges lack solid APIs for adjusting price floors. For now, we chose to update the default floor weekly.

Analyzing the Results

We are currently using this approach to optimize our AdX price floors. We’ve set up an automatic report to send us a CSV file each week with the data for each price bucket in the test. The report produces the following dimensions:

Experiment Results Report

We then paste the data into a spreadsheet that calculates the optimal price floor given assumptions about the recovery rate. We assume we have a recovery rate of 30-60%.

Price Floor Results Analysis

You can see in the image above that at a 50% recovery rate, the default price floors are leading to a roughly 19% improvement in revenue for Desktop ATF (Above-the-fold) inventory. In this week’s test, however, if we moved the price floor up 25%, we could see another 3% gain in revenue. So that’s what we’d do.

Segmenting the Price Floors

Because there is some overhead in setting up, monitoring, and adjusting the price floors in this approach, before establishing the continuous experimentation structure, we ran analyses to determine what segments we’d test. In our case, we found that testing Desktop ATF, Desktop BTF, and Mobile units focused on English speaking countries (United States, Canada, United Kingdom, and Australia) would allow us to optimize a broad swath of our revenue, while keeping distinct floors for ad units that diverged notably in performance. We’d expect these segments to vary from site to site, depending on where the site is earning the bulk of its revenue and how divergent the CPMs are for different segments.

Where to Go From Here

We’ve been impressed with the lift we’ve seen by using continuous experimentation with price floors. We are looking forward to adopting a similar strategy at other exchanges. AdX has provided great flexibility in allowing us to set up and report on granular price floors. But AdX is a minority of our revenue. We’d like to see this type of flexibility available at all our partners. Even better, we’d like to see exchanges themselves provide automated price floor adjustments that optimize revenue, or at least open up APIs that would allow 3rd parties to programmatically analyze and adjust price floors.

Longer term, the price floor optimization problem appears to be a strong candidate for an impression-by-impression prediction algorithm. On the buy side, nearly all major DSPs have some form of machine learning that helps optimize ad placement. On the supply side, however, an equivalently essential function has yet to be filled, which is using similar algorithms to optimize the price of ad inventory on an impression-by-impression basis.

If you have any questions or comments, please feel free to share them in the comments.

Follow PubNation Blog

A/B Testing Advertising: A Playbook for Publishers

Six months ago we launched an A/B testing framework for our site, SpanishDict.com, that has allowed us to make ad ops decisions smarter, faster, and with measurable improvements to revenue. Our fir...

Header Bidder Optimization: When Ads Compete, You Win

In the past two months, we've radically overhauled our programmatic advertising stack on SpanishDict.com, shifting our focus to header bidding advertisers--advertisers that submit the price they wi...