RTB Part III — Cookies & User Data
February 22nd, 2010
Cookie Matching.
A number of people have asked about how cookie matching works in the RTB world. Everybody today relies on cookies to store information about a user — which ads he’s seen, what sites he’s been to, which behavioral segment he’s in — all the good stuff. Without cookies, we can’t frequency cap, we can’t remarket, we can’t track, basically, advertising becomes pretty useless!
Now the way browsers work, cookies are tied to a domain. Say an ad-exchange has ad tags on a page under ‘ad.exchange.com’. When a browser requests an ad from the exchange, it only passes along the cookie data that’s stored inside that domain name. This means that the exchange has zero insight into whatever data the bidder might have collected under ad.bidder.com.
So why is this a problem? Well, if the exchange is going to do a server-side bid request to the bidder, it’s pretty hard for the bidder to make a decision about what ad to serve if it has no access to either frequency or behavioral data!
An Example. Bidder is running a behavioral campaign for marketer “marketer.com”. Bidder has a pixel on the marketer’s site, it’s internal ID for the pixel is 27. Each time this pixel fires the bidder ads a cookie to it’s domain ‘ad.bidder.com’, ’segment=27′. The bidder now wants to buy as much inventory as possible for this remarketing segment across exchanges. Bidder has a user-id, ‘ABC’ for this cookie.
Exchange, ‘exchange.com’ wants the bidder to be able to buy as much inventory as possible. Hence the exchange wants the bidder to be able to use all remarketing segments to buy. Problem is (as mentioned above), they’re using separate cookie domains. So what needs to happen is that the exchange and the bidder need to synchronize their domains so that when the bidder receives a real time bid request he can look-up the information associated with that user.
Fundamentally the logic for synchronizing IDs is pretty simple. The exchange drops a pixel on a page somewhere which points to a bidder’s domain and a simple usersync call. When the call hits the bidder, it reads in it’s cookie ID, redirects off to the exchange passing it’s ID in the querystring so that the exchange can read and store it into it’s cookie space.
The above diagram shows this process. First a pixel is dropped that hits the bidder domain. The bidder receives this call and reads in it’s cookie ID of ‘ABC’. Bidder then redirects the user to get a request from the exchange under ad.exchange.com, passing in the id ‘ABC’ in the querystring. The exchange now receives a call, reads in from the cookie that his ID is 123, and can now map that his user 123 maps to bidder’s user ABC.
Now user IDs are synchronized, the exchange and the bidder can talk about a single user in the same manner. Note that in the above example I noted that the exchange is storing the ID for the bidder. Admeld, PubMatic & the like all work like this. Google does it slightly differently and instead requires the bidder to store the mapping on his end — effectively just the reverse of the above process.
Related Posts:
- Battle over the Cookie
- What’s really in my cookie cache?
- The uberall definition of spyware
- How do behavioral networks work?
- Selling behavioral data in a multi-platform ad-industry
February 24th, 2010 at 3:25 am
Great explanation Mike! Thanks for posting this.
Mukul.
February 24th, 2010 at 3:52 pm
Is this also how the data sellers (i.e. Bluekai, et al) do it?
February 24th, 2010 at 3:55 pm
Wait! Let me reask the question.
I go out and buy a bunch of Bluekai cookies, corresponding to a specific segment (travel intenders, say). Then I go to an exchange and tell the exchange I only want users who correspond to that segment.
How does the exchange do this? Are they already synched with Bluekai?
February 24th, 2010 at 10:37 pm
Excellent Mike!
March 1st, 2010 at 12:11 pm
Mike, can you talk about the pros / cons of the 2 different approaches? i.e Privacy conern, ease of manipulation without having to re-map, etc.
March 4th, 2010 at 11:04 am
Eran — Great question.
From a coverage & tech perspective having the seller store the UID is far more efficient. If the seller drops a sync pixel on every ad call it means that there’s near 100% sync between the two parties. Additionally, sending the buyers UID to him means that he only has to do *one* read on the database instead of two. This might sound small, but at 100k RPS, adding 100k lookups actually has an impact on performance.
From a privacy perspective… I’m not sure… call your legal council? I feel like either way there’s a privacy problem in that lots of companies are conglomerating large cookie pools.
Jerry –
The question I have to ask you back is — what technology platform are you using? If you are using a buying platform that is hooked into all the exchanges then in theory all you do is have bluekai stamp segments on the platform and then they are automatically available for buying across all exchanges. If you have no technology platform them you’d have to have bluekai stamp each individual exchange or run your own pixel-server and piggyback exchange pixels to broadcast this data to each individual exchange.
The data providers themselves are also integrating with the exchanges to make buying their data easier — but that’s a whole separate topic.
March 21st, 2010 at 2:40 pm
[...] A number of people have asked about how cookie matching works in the RTB world. Everybody today reli… [...]
April 17th, 2010 at 2:19 pm
Great explanations !
But, is there a speed problem here? If many bidders are competing for the same ad, and requiring to sync cookies with the exchange, there will be a lot of requests between servers and browser before the ad is actually served
April 17th, 2010 at 2:22 pm
Hi Matt,
No there’s not because the cookie syncing process actually happens *before* the actual RTB bid requests. There is a problem of too many pixels on too many web pages, but that’s an entirely different one!
-Mike
April 18th, 2010 at 4:58 am
Thanks Mike for this quick answer!
One more question : how is it done?
is there 2 scripts in the page : first one to sync cookies, second one to load the ad?
April 18th, 2010 at 9:43 am
Hi Matt,
They’re actually two completely separate processes. You’ll see these sync pixels tacked on to a large number of ads you see on the internet today but are disconnected from the actual real-time process.
-Mike
May 26th, 2010 at 6:18 am
Hi Mike,
This information is very useful for me.
-Nandhini