Browser Referer

In theory, when a browser requests a URL/URI from a Web Server, it includes in its request header the URL (if any) that it just came from (e.g. the page containing the link it's following).

Annoyingly, the official name is Referer instead of Referrer.

At MedScape, we used this data in a couple ways. The way that comes to mind first is that we had a Feedback link on every page of the site, which would go to a form that would generate a structured email to the appropriate party. We made the form dynamic, so it grabbed the Referer and stored it in a hidden field which was then included in the generated email. So, if a user described a problem, we'd know what page he was on at the time.

Of course, every web server log handling package provides a report of referring URLs, so you can see where you get users from.

Likewise, since the Referer URL includes any GET arguments, you typically see any search argument for links from a Search Engine like Google. So log reports include the list of search terms that led users to your site.

• a sad statement on people's understanding of the internet is that the most popular search term for most sites is the name of that site (e.g. most people came to Medscape by searching Google for "Medscape", instead of just putting "http://www.medscape.com/" in their browser Location dialog box).

• you see some strange search requests. There a weblog dedicated to this.

Some problems with this data:

• some weird browsers don't include it (this is pretty rare)

• lots of hits have no data for this because the user is coming from a bookmark, or typing in the URL

• sometimes the data is just plain wrong. You get a URL which just doesn't make sense (you can't find any link to your site, and that site doesn't look like it would have ever linked to you). I think this may be a bug related to use of the browser's Back button...

• in particular with WebLog cases, people are usually reading the homepage of a blog, which lists multiple entries. So when they click through from an entry to your site, the Referer is that site's homepage, rather than the ultimate archive URL that's appropriate. This might be the biggest argument for not implementing non-WiKi BackLinks.

  • although you could take one of those temp-home URLs, and (a) follow it to the source site, (b) find the specific item, (c) click on the Permalink to take you to the appropriately precise URL (d) click on the link that goes back to yourself, to get that added to the list, and (e) sort your list of BackLinks by most-recent-use (rather than number of hits, as many people do). You might also want to make a way to remove backlinks, so you can purge the temp-home items.