Medscape Ad Serving

Here are some notes I'm pulling out of a mindmap I made for a talk I gave at a NetGravity user's conference.

This is about the history/evolution of ad targeting/delivery for the MedScape medical information website. I think the targeting of Advertising is a trend that's been under-explored online, so far.

v1: aliases (~1995)

• Mac platform (see Medscape And Frontier)

• static pages

• defined "slots" for ads using file aliases (Mac version of "shortcuts"), "randomly" inserted HTML for specific slots as files were produced. So one article would have HTML like::

• manually rotated ads through slots: a given ad would involve an HTML page on our site, and a Banner Ad image, and we'd made a shortcut for each and name is 'ad1.gif' or 'ad1.html', etc. If we had a "big" sponsor, we'd give their ad 2 slots.

• some crude sampling told us that we were delivering (roughly) twice as many pageviews as were implied by the hits to GIF images on the server (because of browser caching). So we'd run monthly reports of hits by GIF, and double the numbers for reporting to sponsors! Good-faith but crude. Click-throughs were reported based on pageviews of ad pages (which were on our site, since sponsors didn't have websites yet). Sampling also told us that ad views and clicks were pretty much proportional to our registration membership (e.g. if Physicians were 30% of our members, then they generated 30% of our pageviews, adviews and clickthroughs), so we applied those percentages to our summary numbers.

v1.5: rotate ads on high-traffic pages (~1996)

• still static content pages on Mac server

• but start delivering high-traffic (key navigation) pages from Frontier's cache, so those pages were dynamic

• wrote bit of Frontier code to handle ad rotation. Defined a table of ads for each content specialty/channel on the site, and each table would include data about the ads to run in that channel. Picked ad at random for each pageview, so they were evenly weighted (so for big sponsors, would insert their ads multiple times to give them greater weight). Wrote additional code to (a) flush cache when page content changed, and (b) manage ad rotation from HTML forms, so production staff could do instead of me.

• by this time we were running an Apple Search Search Engine. So I wrote some Frontier code to associate search terms with ads. Because there's such a diversity of user search terms, this never drove significant ad traffic, but was highly valuable to sponsors.

• same 1.0 hack process of reporting views. (At some point sponsors had their own sites, so we weren't hosting clickthrough pages, but the Mac Web Server (WebStar) had a way to make a file which would just trigger a redirect, so we counted clickthroughs that way.)

v1.99: rotate all (~1997)

• when we first migrated to the Win N T platform, we didn't want to deal with a fancy ad server, so our consultant wrote some simple server-side Java SSI code to rotate ads.

• still had static pages, delivered by Netscape Enterprise Server

• each page was assigned to a single primary specialty/channel, and that parameter was passed in the SSI tag for the ad

• similar channel ad rotation model as v1.5

• similar v1.0 reporting model

v2.0: NetGravity (~1997)

• at login, user assigned a Web Cookie defining their profession, specialty and country (we were mainly interested in Physicians, esp Americans)

• we also defined a "sticky" Referer cookie called Refpath which mapped to a specialty/channel (e.g. we had NSAPI code which would recognize when a page request was for a channel navigation page, and then set the Refpath cookie value to that channel, which would keep coming back with each future page request from that user during that session until they hit another navigation page).

• we paid NetGravity to write some custom code that would allow us to target ads on the basis of Refpath, Profession, Specialty (e.g. "cardiology") and Country cookie values.

• once we had the Verity Search Engine running, we also put in code to target on search keywords (along with other cookie values)

• now we could more-accurately report adviews and clickthroughs, even for specific audiences.

v2.1: individual targeting

• we ran into a small number of cases where we could close on big sponsors if we could do more arbitrary ad targeting (e.g. "all male Consumers from a defined list of zip codes")

• so we defined another cookie, and when each user signed in, we'd check the small set of custom rules to assign a value to that cookie. Then NetGravity would handle the targeting for us. (NetGravity had a pretty good model for allocating impressions across ad runs targeted on different/overlapping dimensions.)

One big problem we never solved (and to my knowledge nobody has solved) is inventory forecasting. The more you get into delivering custom niches of audiences, the harder it is to know whether you will be able to deliver the needed volume. Some factors include:

• Gross inventory forecasting: we saw significantly seasonality in our traffic, plus an overall growth curve. It's impossible to recognize (quickly) when either of those patterns is changing.

• Small niches create pockets of scarcity: overall, almost all websites have huge unsold inventory. But the more you try to drive up your pricing by delivering micro-targeted niches, the more likely you are to run short of inventory in some categories.

• Change in mix: this is actually something we didn't see. Crude calculations (niche group as a % of overall registrations) were predictive of even fairly short-term behavior (niche group as a % of monthly traffic).

• Overlapping ad rules: a given user's ad-view could be matched by a number of active runs (keyword, area of site, user profession, etc.). Which one will apply?

• Long Sales Cycle: you have to consider not just what you've sold, but you have to consider (a) how long you'll be able to keep re-selling a given ad run (we had many sponsors "committed" to a full year, but they could in reality cancel at any time), and (b) having a number of "interested" potential new sponsors, and not knowing which ones will actually close (or when).

In reality, this is such a hard problem to truly "solve" that it's probably best to just make a crude guess, err on the side of over-selling, then massage clients when problems arise. Not too many people will get seriously miffed when they get less exposure than they planned, as long as they're charged accordingly.