TOP PUBLISHERS USE PUBNATION TO MANAGE AD QUALITY
read whyRequest A Demo
Thank you! Your submission has been received!
Something went wrong while submitting the form
OptimizationHeader Bidding
June 22, 2016

Header Bidder Optimization: When Ads Compete, You Win

In the past two months, we've radically overhauled our programmatic advertising stack on SpanishDict.com, shifting our focus to header bidding advertisers--advertisers that submit the price they wi...

Chris Cummings

In the past two months, we've radically overhauled our programmatic advertising stack on SpanishDict.com, shifting our focus to header bidding advertisers--advertisers that submit the price they will pay for each impression into an auction that occurs in the header of a webpage. We’ve seen sustained CPMs at levels far above what we’ve received from ad networks and exchanges. In this article, we share the insights and lessons learned on how to optimize programmatic advertising using header bidders.

Fragmentation: The Fundamental Problem with Programmatic Advertising

Before delving into strategies for optimizing programmatic advertising, it’s worth pausing to ask why we need to optimize advertising in the first place. No one optimizes the price of oil, gold, or stocks. In these markets, the price is determined by what buyers are willing to pay. These markets are transparent, open, and globally connected. Why, then, must we optimize the prices of programmatic advertising inventory, which is almost by definition commoditized advertising? The reason is because of fragmentation. We don’t have a unified market for buyers and sellers, where buyers can compete to purchase advertising inventory using the same information. The investment bank Luma Partners compiled a landscape of the display advertising ecosystem which illustrates the fragmentation:

We can see above that supply, demand, data, analytics, and other services are fragmented, very fragmented. In the past, many ad ops professionals have set up “waterfall” structures to try to tap into multiple demand sources and find the highest paying advertisers for each impression. The fundamental problem with the waterfall structure, however, is that the networks never competed against each other. They only competed against a price floor, which acted as a flawed representation of what other partners would have paid for that impression.

In an alternate universe, the industry might look something like this:

In this world, all advertisers compete with each other to set the price for each slice of ad inventory. Header bidding brings us closer to this alternate universe. The unified ad exchange, surprisingly, is the web browser.

An Economist's Dream: An Auction for Every Impression

The promise of header bidding is that it can unite various ad exchanges and networks into one unified auction where there is competition for every impression, on every ad unit, on every page. The unified competition should result in the highest price that the market will sustain for the ad inventory.

A header bidder driven programmatic strategy changes ad ops in fundamental ways. In a scenario where every advertiser bids in the header, it’s no longer necessary to create a waterfall. The highest paying advertiser gets the impression the first time. That means passbacks become irrelevant, and setting price floors becomes obsolete. In the future, even negotiating revenue share and fee agreements with ad networks could become antiquated. If the advertiser’s bid doesn’t beat the alternatives, the advertiser won’t show any ads. Conversely, as long as the advertiser can beat the alternatives, it can take as big a share of revenue as it wants. The price for each piece of ad inventory should reflect the market’s highest willingness to pay. The unified auction is the ultimate optimization.

The Who’s Who of Header Bidders

Header bidding is a relatively new approach to serving ads, but it is spreading rapidly throughout the industry. Here are the advertisers we’ve identified that have header bidding capabilities:

AdX and AdSense are noticeably absent from the list, as they don’t have pure play header bidding integrations. They can, however, participate in the unified auction if you use DFP as your ad server. To achieve this, you must allow for AdSense and AdX to compete with the other header bidding partners that are setup in the ad server.

We started with the header bidding partners we thought would lead to the highest performance. What we’ve found, however, is that it is very difficult to predict beforehand how a header bidder will perform. We now try each of them, measure performance, and only continue to work with the ones that perform well.

The Header Bidder Implementation Slog

Each header bidder implementation is different, and each is a pain in its own way. Activating a header bidder partner requires you to include javascript in the page header to request the bids. The various partners implementation guides often assume that they are the only game in town. We've found that we have to do a lot of work in order to allow multiple header bidders’ javascript to load correctly at the top of the page. We require that all header bidders load asynchronously, and we allow up to 500ms for the auction to run. We experimented with allowing the auction to take 1000ms, but saw a small decline in overall yield. The header bidder logic looks something like this:

The second key step in setting up the header bidders requires creating the order and line items in DFP to serve the ad for the winning bidder. Our advertising structure uses an approach that allows for A/B testing ads and fine-grained reports by geography, ad unit, and mobile vs. desktop. To accommodate the line-items for each price bucket that header bidders require, we had to create thousands of new line items. The DFP interface can be frustratingly slow, so doing this manually would have taken months. Instead, we wrote a line item generator script that uses the DFP api to create the line items and set the appropriate targeting and creatives programmatically. This was key to being able to quickly and reliably set up so many partners. If you are interested in getting a copy of the line item generator script, let us know in the comments. UPDATE: The line-item generator script is now available on Github.

A few lessons we’ve learned for when first establishing a relationship with a header bidding partner:

  • Require "net bids" - specify in your contract that the bid the advertiser sends to the browser be a net bid. If the bid is a gross bid, before fees and revenue share arrangements, it doesn’t represent what the advertiser is actually willing to pay. Allowing gross bids will lead to instances where the advertiser won the auction, but couldn’t actually pay more than the next guy on a net basis. One huge benefit of accurate bidding is that DFP can be used for reporting by partner, allowing for immediate insight into revenue and CPM for each header bidder partner, right from within DFP.
  • Aim for the smallest increments possible - the best possible increments, or price buckets, you can establish with the advertiser are 1 cent increments--an exact representation of what they are willing to pay. The smaller the increment the less money will be left on the table because of rounding. If price buckets are higher than 1 cent, make sure that the advertiser is always rounding down before placing their bid. If they are willing to pay $1.56, and are using 10 cent price buckets, the bid should come in for $1.50, rather than $1.60.

We devoted 1-2 months of an engineer’s time to set up the new header bidding approach. Efforts are afoot to make this process easier. Tools like PreBid.js andTechnorati hold promise for simplifying the integration of multiple header bidders. During our setup, however, particularly as we tried PreBid, we found that the advertising partners did not easily integrate with these solutions. It’s likely that in the coming year, there will be more solutions on the market to make integration easier.

Mitigating The Risks of Header Bidders

Many of the risks with header bidders are similar to those posed by ad exchanges run through an ad server. Header bidders, however, have the unique quality of loading on every page, and loading in the header of the page, rather than in an ad slot. This introduces a couple risks that need to be managed:

  • Site Speed - All header bidders should be implemented using async javascript requests, so as not to block the loading of the page content while waiting for the bid response. Even still, it is important to monitor the code that the header bidders load on the page to ensure it doesn’t make numerous subsequent network requests that can slow down the page. Our engineering team conducts audits on the impact of adding a header bidder script before setting it live.
  • Advertising Quality - Many header bidders use ad exchanges to fill the ad inventory, and ad exchanges face a tough challenge in policing ad quality.PubNation helps us manage the quality of the advertisements that come through the header bidders.
  • Data Leakage - Header bidders can load additional tracking pixels, even if they don’t win the bid. It’s important to analyze how a header bidding script works and understand what data the advertiser is gathering in the event they don’t win the bid.

In our case, none of these risks outweighed the upside of using header bidders, but they need to be managed.

Header Bidders on Mobile Web

As users continue to increase their usage of mobile devices, it’s worth considering how header bidding will affect the mobile monetization. The advantages of the unified auction apply in the same way to mobile units that they do to desktop units. On mobile, however, the number of network requests made in the header becomes an even bigger concern. Mobile browsers often have stricter limits in the number of simultaneous requests they can make, so numerous header bidders could slow down the loading of other parts of the page. Mobile devices also typically have slower internet connections, meaning the time to execute the auction could take longer.

Perhaps because of these limitations, header bidding technologies have been slower to come to mobile web units like the 320x50. Our strategy for mobile header bidding has been to keep the number of header bidders we work with on mobile constrained--somewhere between 2 and 5, and to monitor the impact on site speed and load times carefully.

Auditing Header Bidder Performance

Header bidders introduce an important new reporting requirement to ensure that the bidders are paying the full amount of what they bid. We do regular audits where we run a report in DFP that shows the expected total revenue for each partner, and then we compare that to the report in the advertiser’s interface. We’ve found in the past advertisers that have a significant discrepancy between what they bid and what they pay. This is a problem. If the advertisers are bidding more than they pay, the impressions may not in fact be going to the highest bidder, leaving revenue on the table.

The clearest fix for this problem to to require the advertiser to update their bids to reflect what they pay on net terms. If for some reason that is not possible, the next best option is to adjust the line item rate in ad server down by the maximum discrepancy. For example, if an advertiser bids a $2.00 CPM, but only ends up paying $1.60, adjust the ad server rate for all of their bids down by 20%, so that the auction logic in the ad server still works correctly.

Layering in Ad Networks and Exchanges

We aim to work with all new partners on a header bidder basis. But we still have network and exchange relationships that add value to our ad stack--although much less than they did in the past. Our prefered way of working with networks is to negotiate fixed CPM deals for a maximum amount of impressions. This approach, when added to the ad server, allows the networks to compete in the same auction as the header bidders. The fixed CPM, fixed impression deals with networks change the way that we are able to work with networks. We can work with networks that need a small volume of impressions at a high CPM, and networks that need a huge volume of impressions at a low CPM.

The key metrics that we look to when including ad networks is no longer CPM. Instead, we look to find partners where we can reliably send them traffic that satisfies their needs, while including them in the ad server at a rate that is lower than the negotiated CPM. This is a key point. If we strike a deal with a network for a $1 CPM, and include them in the ad server with a rate of $1, the incremental value of working with the ad network would only be the tiny increase of their CPM vs. next highest header bidder CPM, which is likely only a few pennies. To create value with our network relationships, we look to include them in the ad server with a rate as low as possible, while ensuring they receive valuable traffic. The difference between what they are willing to pay and what we include as the rate represents the improvement from working with the ad network as compared to what we would receive from the header bidders. We view this as an iterative process, where we constantly must refine the relationship between the the negotiated CPM and the rate, based on meeting the ad network’s needs and checking to ensure we are serving the maximum impressions allowed under the deal. The process looks like this:

Speculation: How Header Bidding Could Change the Industry

The advent of header bidding has created a relatively efficient auction for our programmatic ad inventory, increasing ad inventory revenue and decreasing the management time required for optimizing ad performance. These advantages are likely to cause more and more publishers to switch to a header-bidding-first strategy. The current implementation requirements for header bidding are slowing adoption, but the advent of 3rd party tools to streamline implementation will likely overcome these hurdles. If header bidding continues to grow quickly, the traditional ad network model may likely be squeezed out of the industry entirely, as they are unable to bid in the auction for the best inventory, and they have less data on which to bid because they are not in the header on the page. The second likely outcome is that publishers will work with a limited number of header bidders on the page--reluctant to load so many 3rd party javascript tags in the header of their page--and start hosting private exchanges that allow for server-side integrations for new bidders. On the whole, header bidding is creating a far more efficient market for buying and selling advertising inventory. So far, it has proven to be a great improvement in our programmatic ad stack.

Follow PubNation Blog

Publishers Fight Back: How The Top 50 Websites Combat Adblock

On our site, SpanishDict.com, adblock has become a major problem. We lose over 15% of our revenue each year because of adblockers--and the impact is growing larger every month. We are now embarking on

A Simple A/B Test that Raised AdX Revenue by 10%

A simple price floor A/B test in AdX that resulted in a revenue increase of 10%. We’ll also show you the details of how we ran the test, so you can give it a try for yourself to see if it boosts your