We have discussed the Privila network on Light Blue Touchpaper before. Richard explained how Privila solicit links and I described how to map the network. Since then, Privila’s behavior has changed. Previously, their pages were dominated by adverts, but included articles written by unpaid interns. Now the articles have been dropped completely, leaving more room for the adverts.

This change would appear to harm Privila’s search rankings — the articles, carefully optimized to include desirable keywords, would no longer be indexed. However, when Google download the page, the articles re-appear and the adverts are gone. The web server appears to be configured to give different pages, depending on the “User-Agent” header in the HTTP request.

For example, here’s how soccerlove.com appears in Firefox, Netscape, Opera and Internet Explorer — lots of adverts, and no article:

In contrast, by setting the browser’s user-agent to match that of Google’s spider, the page looks very different — a prominent article and no adverts:

Curiously, the Windows Live Search, and Yahoo! spiders are presented with an almost empty page: just a header but neither adverts nor articles (see update 2). You can try this yourself, by using the User Agent Switcher Firefox extension and a list of user-agent strings.

I expect the interns who wrote these articles will be displeased that their articles are hidden from view. Google will doubtlessly be interested too, since their webmaster guidelines recommend against such behavior. BMW and Ricoh were delisted for similar reasons. Fortunately for Google, I’ve already shown how to build a complete list of Privila’s sites.

Update 1 (2008-03-08):
It looks like Google has removed the Privila sites from their index. For example, searches of soccerlove.com, ammancarpets.com, and canadianbattery.com all return zero results.

Update 2 (2008-03-11):
Privila appear to have fixed the problem that led to Yahoo! and Windows Live Search bots being presented with a blank page. Both of these spiders are being shown the same content as Google’s — the article with no adverts. Normal web browsers are still being sent adverts with no article.

Update 3 (2008-03-11):
Shortly after the publication of an article about Privila’s browser tricks on The Register, Privila has restored articles on the pages shown to normal web browsers. Pages presented to search engines still are not identical — they don’t contain the adverts.