Last month, on the 4th April, I published a document describing how the Phorm system worked and blogged about what I thought of the scheme. The document had been run past Phorm’s technical people to ensure it was correct, but — it turns out — there were still a handful of errors in it. A number of helpful people pointed out that I’d misdescribed third-party cookies (which didn’t matter much because Phorm specifically uses first-party cookies), and I’d managed to reference RFC2695 rather than RFC2965 !
In my original document, I’d waved my hands a little bit about how the system worked if people had blocked cookies for specific domains, and so I swapped some more email with Phorm to better understand, and then published a revised version on 23rd April — so that the correct information would be available to accompany FIPR’s press release and paper on the various laws that the Phorm system breaks. However, there was one final thing that wasn’t dealt with by press time, and that’s now been explained to me….
The Phorm system does some of its tracking magic by redirecting browser requests using HTTP 307 responses. When this was first explained to me at the meeting with Phorm there were two redirections (a scan of my notes is here), but having thought about this for a while, I asked for it to be explained to me again later on, and it turned out that I had previously been misled, and that there were in fact three redirections (here’s my notes of this part of the meeting).
It now turns out, following my further emails with Phorm, that there are in fact FOUR redirections occurring! This is not because my notes are rubbish — but because Phorm have managed to recall more of the detail of their own system!
For full details of how I understand the system works (at least until some more detail comes to light), see the latest version of my explanatory document, but to give you a flavour of it, consider an example visit to www.cnn.com:
- The user wants to visit
www.cnn.com, but their request does not contain a cookie (forwww.cnn.com) with a Phorm unique identifier within it. They are redirected (ONE) by the Phorm system to www.webwise.net. - The user visits
webwise.netby following the redirection. If they do not have a Phorm identifier cookie, then they will be issued with a new identifier and redirected (TWO) elsewhere onwebwise.net. - The user visits
webwise.netfor the second time. If they still don’t have a Phorm identifier cookie then their IP address is marked as wishing to opt-out and they will be redirected towww.cnn.comand they won’t be redirected again for at least 30 minutes. If they do have a cookie (or if they had one at the previous stage) they are redirected (THREE) to a special URL withinwww.cnn.com. - The user visits the special URL, which the Phorm system redirects to a fake version of www.cnn.com that sets a
www.cnn.comcookie with their Phorm identifier in it, and redirects (FOUR) them to the URL they wanted to visit all along.
For the moment, this appears to be the grand total; there can be up to four redirections, and it is deducible from this description what happens if you refuse (or delete) cookies in the webwise.net and www.cnn.com domains. It is also apparent that if you resolve webwise.net to 127.0.0.1 that you’ll never get past the first redirection; and you will need to rely on the Phorm system spotting these repeated failures and turning off redirection for your IP address.
direct adjective: Straightforward in manner or conduct; upright, honest.
indirect adjective: Mechanism by which Phorm fools your system into accepting tracking cookies from third-party websites, even when those websites promise never to track you!
That covers the exception where there’s no webwise.net cookie because the user is systematically blocking cookies from that domain. It doesn’t cover what happens when there’s no webwise.net cookie because this is a new user or the user has cleared all their previous cookies.
The sentence “if there is no cookie then a new UID will be issued instead” must hide a lot of detail. Consent must be obtained before a UID cookie can be placed on the user’s machine. The user will need to visit what Phorm have called the ‘invitation page.’ At this point, all redirection must cease because the system will be waiting for the user’s response: do they want to take part in Webwise or not?
This implies that the first attempt to set a webwise.net cookie must be some form of dummy. If the dummy can be set, the user can safely be redirected to the invitation page, where the current sequence of redirections will stop. Only if consent is obtained can a true UID cookie be set. I assume the act of setting this cookie will start a new series of redirections, but this time starting with the special URL described at the end of paragraph 17.
We’re still no nearer to knowing what happens if, rather than cookies from webwise.net being blocked, only cookies from cnn.com are blocked. Is it considered such an exception that the system will pick it up by observing the browser going around in circles, or would a fifth redirection to check that that cookie has been successfully set be better? Potentially, a dummy cookie could be set at the very first redirection, which could be checked for when the fake http://www.cnn.com URL is requested, and which would be replaced using the response. This would avoid loops, without a fifth redirection being necessary.
This is worryingly complicated. Already, slight issues have been noted. As mentioned, there’s a problem if access to webwise.net is completely blocked. There may also be an issue with Opera, if the user sets it to block third-party cookies. I’d also like to know what Phorm think about the possibility of a user deleting just their webwise.net cookies. Any copies of the UID cached in other domains could cause the user to carry on being profiled for up to three more days, while the webwise.net status page and any adverts showing the user’s status will tell them that they are not being profiled.
@PA
The sentence “if there is no cookie then a new UID will be issued instead” must hide a lot of detail.
Yes, but this detail will differ between ISPs (as indeed will the exact nature of the layer 7 switch), and hence it is out of scope for this description, which is intentionally generic.
This implies that the first attempt to set a webwise.net cookie must be some form of dummy.
I don’t think that is necessarily the case — but you will need to look at specific implementations (which hopefully you will find impossible, because they aren’t deployed!) in order to know for sure.
We’re still no nearer to knowing what happens if, rather than cookies from webwise.net being blocked, only cookies from cnn.com are blocked.
This is covered in paragraph #27 of the document.
There may also be an issue with Opera, if the user sets it to block third-party cookies.
These are not third-party cookies, so I doubt it. We also don’t know if Opera is included in the list of browsers that they are prepared to mess with (see paragraph #36).
@ Richard Clayton
This is covered in paragraph #27 of the document.
Paragraph #27 doesn’t say how the infinite loop will be detected. I had assumed the infinite loop detector would be outside the normal sequence of redirections. Until your latest update, we didn’t know how detecting the blocking of webwise.net cookies would be implemented. As one extra redirection has already made an appearance, could we see any more?
These are not third-party cookies, so I doubt it.
When I tested Opera, its behaviour was consistent with it treating all cookies following a redirection as third-party cookies.
http://www.badphorm.co.uk/e107_plugins/forum/forum_viewtopic.php?4014.post
When I chose Opera’s non-default option of refusing third-party cookies, cookie setting and sending stopped completely after the first redirection in a sequence of redirections. Another poster on that thread said they felt that this is a correct implementation of RFC 2965.
We also don’t know if Opera is included in the list of browsers that they are prepared to mess with (see paragraph #36).
The BT Webwise FAQ/Help page was updated on 2 April. It implies Opera is on the list of user agents whose traffic will be processed. If it’s going to be skipped, like Safari, I don’t think BT would have put Opera in the proven-to-work list.
http://www.beta.bt.com/bta/forums/thread.jspa?threadID=3050
http://www.webwise.bt.com/webwise/help.php
@ Richard Clayton
I think it’s a maximum of three redirects per HTTP get.
Assuming users enable webwise.net cookies:
* Whenever the user visits a new site they get redirected three times:: (1) to webwise.com to get a Phorm cookie, (2) to a fake version of the desired site to get a fake site cookie, and (3) finally to a real version of the desired site.
* When they visit the same site again, they go straight to the real site, but Phorm reads their cookie along with the rest of the transferred data.
Assuming users disable webwise.net cookies:
* Whenever the user logs on and visits their first site for a while, they also get redirected three times, exactly as above.
* When they visit the same site again during the next 30 minutes, Phorm tries to read the fake site cookie, notices its not there, and kindly decides not to read the rest of the transferred data.
* Whenever the user visits a different site within the next 30 minutes, they get redirected two times: to webwise.com which tries to read the Phorm cookie and notices its not there, then to real version of the designer site.
As you say, blocking webwise.net prevents you from accessing the Web, which has the desirable side-effect for Phorm that you can’t easily block adverts hosted on their domain.
Personally I don’t see why they don’t just track subscribers based on their IP addresses. Their system effectively falls back to doing this when there’s no Phorm cookie, to check whether 30 minutes has passed, when it will try to install a cookie again. Possibly this is a Data Protection Act requirement?
But given that Phorm are going to follow the complex logic above, they seem slightly vulnerable to a wget script which accesses random IP addresses, or an auto-refreshing Web page which loads images from random IP addresses.
I think it’s a maximum of three redirects per HTTP get.
Assuming users enable webwise.net cookies:
* Whenever the user visits a new site they get redirected three times:: (1) to webwise.com to get a Phorm cookie, (2) to a fake version of the desired site to get a fake site cookie, and (3) finally to a real version of the desired site.
Nope — you’ve missed out the second visit to webwise.net to check that the webwise.net cookie at #1 was accepted.
If my blog post isn’t clear — then try reading the latest version of the document!
Simply forgotten or redesigned in a panic?
richard, what are we missing here (if anything) can you explain?
see comment 4 (revrob)
“This seems surely just to be an answer to a question I raised a while ago with BT (never answered) about the way a Phorm UID is constructed and whether it was truly random or contaiined an ISP identifier to make sure the different ISPs didn’t issue overlapping Phorm UIDs
It seems that indeed the cookie and presumably the Phorm UID will contain an ISP identifier.”
http://www.badphorm.co.uk/e107_plugins/forum/forum_viewtopic.php?5930
http://www.cableforum.co.uk/board/12/33628733-virgin-media-phorm-webwise-adverts-updated-page-472.html#post34558573
sorry about that, and sammy’s point,just above that…
“When originally asked about Phorm- KE and Co stated that the profile created can be transfered from one ISP to another.
ie. Once a profile is created using my VM connection, I can use another PC or lets BT, and the Profile created will deliver the same ads, as if I was on my home connection.
Which clearly shows something has been missed, in respects of our analysis – to deliver this Phorm must have the capability to transfer the data from VM to BT, then BT to CPW and so on.
To do this will depend on far more more than a mere Cookie dependant on your browser.
IMO Phorm’s System, using the methods they claim to, must Intercept clearly indentifiable PII to reference with the profile they save for the user.
“
BTW Richard,
Tharrick and ravenheart both got a reply from EU Commisioner today, read it here
http://www.cableforum.co.uk/board/12/33628733-virgin-media-phorm-webwise-adverts-updated-page-485.html#post34560783
scans here
http://www.cableforum.co.uk/board/12/33628733-virgin-media-phorm-webwise-adverts-updated-page-486.html#post34560921
and a news post related to this official EU stance
“…The data concerned in this particular matter i.e. the content of search queries, constitute communication within the meaning of this Directive and the URLs used in the packets constitute traffic data. This data should therefore be protected appropriately…”
here for a digg and link
http://digg.com/tech_news/EU_Commission_respond_to_the_publics_complaints_about_Phorm
more twists today richard, from Alexander’s new NoDPI page
http://digg.com/tech_news/BT_commited_113_million_allegedly_illegal_acts_in_8_days
“BT commited 113 million allegedly illegal acts in 8 days
nodpi.org — [EXCLUSIVE] An article summarising an internal report by BT regarding their covert trials of PageSense (Phorm) in September 2006. IP addresses were used (despite BT assuring the public and ICO that no personally identifiable data was used) and 130 000 charity ads were hijacked and replaced with Phorm’s ads.
…
”
http://www.cableforum.co.uk/board/12/33628733-virgin-media-phorm-webwise-adverts-updated-page-532.html#post34567434
Phorm counter-measures
1)
Edit your ‘Hosts’ file…
Start –> Run –> Type:
notepad “c:windowssystem32driversetchosts”
Add the following line to the bottom of the file:
127.0.0.1 oix.net
127.0.0.1 oix.com
127.0.0.1 phorm.com
127.0.0.1 webwise.net
127.0.0.1 webwise.com
127.0.0.1 sysip.net
127.0.0.1 qkilbdr.net
127.0.0.1 121media.com
127.0.0.1 openinternetalliance.com
127.0.0.1 openinternetalliance.net
127.0.0.1 youcanoptin.com
127.0.0.1 youcanoptin.net
127.0.0.1 youcanoptout.com
127.0.0.1 youcanoptout.net
File –> Save
File –> Exit
(You may, at first, have go into the file’s temporary and untick the ‘Read-only’ box).
(HEY MOD! Have I got the first bit right, this time)?
2)
Visit http://www.dephormation.org.uk/ and Download the Dephormation v2.1 Firefox Add On.
“The Dephormation Add On ensures that your decision to permanently opt out of Phorm profiling cannot be undone in Firefox.
Optionally, the Add On can also alert you to sites using Phorm/ Webwise/ OIX profile based advertising.
With each page you view in your browser, a Phorm ‘opt out’ cookie is set automatically, and the Phorm UID cookie is randomised. Even if you delete all your cookies regularly”.
3)
Visit http://www.torproject.org/ and install TorBrowser, a Windows Browser Bundle (Containing Tor, Torbutton, Polipo, and Firefox).
@v4vendetta
#1 no don’t do this, it will break things
#2 this should work (and if it doesn’t I expect they’ll fix it)
#3 this is rather over the top and will affect your browsing in other ways
might I suggest #4, which is to change ISP ? though even that may not be needed since no-one has rolled the ‘service’ out yet.
If such a System goes “live”, I would actually recommend #1.
Your ISP has a contract with you to provide access to the Internet through their switching device, the moment they cut off your Browsing they have cut off your Access & they are obliged to remedy this!
@Jonah
If you do #1 then, because of the way Phorm works, you will not be able to browse to any website. I don’t think that the ISP is likely to “remedy this” by any other method than telling you that putting 127.0.0.1 in your hosts file for the webwise.net domain is anything other than a damn fool thing to do. The other entries are unlikely to have any practical effect one way or another. Have a further read of how the system works!
Richard I fully understand the implications of doing this but BT have a contract with me to provide WWW internet access & as far as I know they do NOT have the right to intercept my traffic.
The moment I am blocked from access to the WWW I will politely ask BT to properly reconnect me to the WWW or give me a MAC to go elsewhere as “they have broken their contract to provide WWW access”!
Since BT intercepted my traffic in 2006 & 2007, I am very likely to have that happen again!
Something I meant to ask some time ago, if a Web Server does this is the Phorm/Webwise System supposed to follow with Profiling or abort the Operation?
{www.anywhereh.com redirection code to http://www.anywherh.com:7034}
In other words does it only Profile on Port 80 or on all ports when using the HTTP Protocol?
@Jonah
In other words does it only Profile on Port 80 or on all ports when using the HTTP Protocol?
See paragraph #3 of my report!