There’s a lot to be said for doing as much work ahead of time (offline) and sharing the upfront cost of it. But there’s absolutely no reason you should not be able to ridicules amount of user targeting in real time and not even having the cpu break a sweat.

I don’t think there’s anyone out there with well designed adserver who’s run into machine limits. Real machine limits where their bottle neck is page fault, cache misses. Including us here.

Lastly, it is my opinion that if can’t get faster at what you do as you grow up (request / and data wise) then you’re not doing it right. In fact as we’ve grown and added more features our systems acctually got faster. We now able to serve more qps (by order of magnitude) per machine then we use to.

Well written post.

I am going to go a bit math geeky on you for a bit!:-)

The problem in dynamic ad construction, as well as in optimization with algorithmic feedback loops at core boils down to “set computations” with a notion of relevance ranking and scoring. In general, the problem in real-time bidding is isomorphic to the dynamic ad selection problem – modulo some plumbing. The problem is of matching a high dimensional incoming vector to a set of high dimensional content/document vectors. The constraints for selection are often expressed in SQL like constructs with keyword search like semantics. In the literature, these are often called Non-Ordered Discrete Data Spaces, where standard Euclidean geometry does not work. And standard SQL oriented databases don’t work at all, in terms of scale of computation and the flexibility demands. A new mechanism that does the absolute minimum in terms of fast computations, where business rules for marketing are encoded offline, we found to be the best approach.

Having fallen down a few times implementing parametric search engines (in such discrete data environments) in past lives – our team has solved a substantial part of the retrieval scale problem in ad serving. We have had to run at volumes approaching 30K qps at latencies of less than 1 ms response times.

That was a surprising burst of unexpected volume that came from RightMedia – within the first 2 seconds at the top of the hour reaching such peak volumes. It was scary and fun.

It is also important to point out to the brave souls attempting this sort of build out that all of your systems – logging, data aggregation, analytics system scale are also not to be treated lightly. Everything gets stressed significantly. There are substantial challenges in horizontal scalability on those fronts as well.

There are some folks that appear to think that throwing hardware in a cluster will solve all problems. With battle scars to prove it, not really. You still have to look at your own computational pipeline and address bottlenecks one at a time. But nothing gives like a great algorithm. But you know that already. Long live D.E. Knuth!:-)

–pradeep

Re: the (A^B)v(C^D) issue – to me it’s actually a question of on vs off-line processing.

The final actual logical (A && B) || (C && D) only going to end up one extra instruction anyway (assuming they’re all in registers) – far less than the time required to parse a cookie or HTTP header.

The most important thing I’ve found, and something that took me a while to convince others of, is that you can give up synchronisation. E.g. if you’ve got quotas you don’t want to be locking over something every request. It’s fine to have a delay in synchronisation, and soak up costs for the _tiny_ fraction of a percentage extra that may be delivered, at an incorrect price etc.

All that said, I’m working at what’s your low-end , >1K/s is a fairly big spike for me.

(I just came across your blog by the way – it’s a breath of fresh air to find so many people working on serving ads commenting on one blog)

My view on the industry is that what separates businesses that succeed vs. ones that don’t is maturity combined with execution. Good ideas are great but they don’t get sold easily and I agree scale is important.

Look forward to more posts!