Notice: This blog is no longer updated. You may find a broken link or two

You can follow my new adventures @mikeonwine

Networks: Friend or Foe?

May 2nd, 2008

There’s a great post from Christian Kreibich dissecting Adsense’s JavaScript. Near the very bottom there’s a little hidden nugget that I’m not sure Christian quite got the consequence of that I’d like to elaborate on (don’t worry if you don’t understand JS, I’ll explain!):

	    if (specialSites[doc.domain] && doc.domain == "") {
		var d = doc.URL.substring(doc.URL.lastIndexOf("http"));
		win.google_page_url = d;
		win.google_page_location = doc.location;
		win.google_referrer_url = d;
	    } else {
		win.google_page_url = doc.referrer;
		if (! isTooSmall(win, doc)) {
		    win.google_page_url = doc.location;
		    win.google_last_modified_time = Date.parse(doc.lastModified) / 1000;
		    win.google_referrer_url = doc.referrer;

Now you don’t need to be technical to get this. That first line basically says this — “If this is coming through do something special.” As most of you probably know, is the serving domain used by the Right Media Exchange, which of course encompasses a significant chunk of remnant inventory being sold online today. So what Google is doing with this code is something “special” in case the publisher’s referring domain contains the url ‘’ — when the publisher is some party on the Right Media Exchange.

The next couple lines do some special manipulation on the referring URL. Now, it’s most likely that Google is simply pulling out the “&u=” parameter (the “URL.lastIndexOf()” ) commonly seen in tags, which would be the URL of the actual website hosting the ad. Google could then be using this to drive contextual results in case it is wrapped in one too many IFRAMEs and can’t pull out the “document.url” parameter it normally uses. But it bring to light another possibility.

Rather than accuse Google of doing something bad (which they probably aren’t), let’s move this to a hypothetical situation — a network (Network Evil) is buying from a publisher (Pub Good) and wants to extract additional information to build a more detailed behavioral profile of the end user. “Good Pub” knows the age and gender of his users and passes that to his adserver for premium advertisers. This particular publisher doesn’t share this information with ad-networks for privacy reasons and only uses it for direct brand advertisers where he hosts the actual creative.

One of the easiest ways for the publisher to pass information to his adserver is to insert a query string parameter at the end of an ad-call. For example, I could append “&gender=male&age=21″ at the end of my IFRAME ad-call. My adserver could then interpret these values and my media-sales team can then target specific advertisements to men or women of certain ages on my site. The problem with this method is that ‘Network Evil’ could easily sniff this information

All ‘Network Evil’ has to do is log and then harvest data from referring URLs. With a little bit of clever javascript (or server side code) ‘Network Evil’ can dig through the publisher’s referring URL and use the age and gender passed in the querystring to better target his ads and even store this information in his own cookie.

Of course with this simple example there are a number of easy workarounds. First off, any smart publisher won’t pass “&gender=male&age=22″ into the querystring and instead use obfuscated or encrypted parameters. This makes extracting the information much more difficult. But, as I’ve shown before in this post, there are some clever tricks you can do with Javascript already to increase how much data you can retrieve from a partner. Rather than post a list of nasty ideas and get them in peoples heads I’ll leave things to your own imagination.

This brings me to the whole point of this post. It seems these days that everybody is an ad-network. We have vertical networks, publisher networks, behavioral networks, data networks, optimization networks, cpa networks…. each of which when it comes down to is buying and selling online media in some shape or form — often from each other! This is the problem — our partners are our competitors and our competitors are our partners.

The question becomes — how do you monitor and control what your partners/competitors do with the inventory or creatives you give them? How do you stop someone form stealing behavioral data? How do you audit the creative an ad-network shows? Of course you can put contractual obligations in place, but how do you audit those policies? How do you audit your partner’s partners? If one network buys male users from you and then resells them to the another network, how do you control and audit that neither network is storing that data for future use?

To date the answer has generally been two things: rudimentary technology solutions and trust. When it comes down to it, most companies simply trust that their buyers and sellers are going to adhere to the negotiated contract terms. Some may go an additional step and place some rudimentary technology solutions in place to help audit, but honestly true auditing is incredibly difficult and challenging (look at Errorsafe which is still happening!). This is why slip ups continue to happen. Data leaks, adult creatives end up on premium sites and brand ads are shown on spyware apps.

Is this really the way ads are still being traded in 2008?

Related Posts:

  • That Which I Missed

    [...] How does Google treat Right Media redirects? Hmmm. [...]

  • » That Which I Missed Optimize Governance: Monitoring – Reporting – Governance

    [...] How does Google treat Right Media redirects? Hmmm. [...]

  • Paul Cook

    Interesting stuff, I’ve already seen this sort of demographic data appearing in referrers for a very large uk website being sold “blind” through multiple networks. Personal data like age,sex and level of education could be easily captured but our view is using it would be a serior breach of EU Data Protection Law (as is making it available).

    We were providing referrer reports for advertisers here on network buys but they now bury their ads in iframes. As you say (and google clearly do) it’s not hard to decode most adserver urls.

  • dc

    Great write-up Mike. I am a huge fan of your blog and really love your insight. After a year now, is there still no other levers other than trust & contracts? I would hope that audits now would be easier to apply…

  • ZIP Repair Software

    Thanks for this usefull information, i really need it.