Last week I spent several hours at Phorm learning how their advertising system works — this is the system that is to be deployed by the UK’s largest ISPs to pick apart your web browsing activities to try and determine what interests you.
The idea is that advertisers can be more picky in who they serve adverts to… you’ll get travel ads if you’ve been looking to go to Pamplona for the running of the bulls, car adverts if you’ve been checking out the prices of Fords (the intent is that Phorm’s method of distilling down the ten most common words on the page will allow them to distinguish between a Fiesta and a Fiesta!)
I’ve now written up the extensive technical details that they provided (10 pages worth) which you can now download from my website.
Much of the information was already known, albeit perhaps not all minutiae. However, there were a number of new things that were disclosed.
Phorm explained the process by which an initial web request is redirected three times (using HTTP 307 responses) within their system so that they can inspect cookies to determine if the user has opted out of their system, so that they can set a unique identifier for the user (or collect it if it already exists), and finally to add a cookie that they forge to appear to come from someone else’s website. A number of very well-informed people on the UKCrypto mailing list have suggested that the last of these actions may be illegal under the Fraud Act 2006 and/or the Computer Misuse Act 1990.
Phorm also explained that they inspect a website’s “robots.txt” file to determine whether the website owner has specified that search engine “spiders” and other automated processing systems should not examine the site. This goes a little way towards obtaining the permission of the website owner for intercepting their traffic — however, in my view, failing to prohibit the GoogleBot from indexing your page is rather different from permitting your page contents to be snooped upon, so that Phorm can turn a profit from profiling your visitors.
Overall, I learnt nothing about the Phorm system that caused me to change my view that the system performs illegal interception as defined by s1 of the Regulation of Investigatory Powers Act 2000.
Phorm argue, with some justification, that their system does not permit them to identify individuals and that they meet and exceed all necessary Data Protection regulations — producing a system that is superior to other advertising platforms that profile Internet users.
Mayhap, but this is to mix up data protection and privacy.
The latter to me includes the important notion that other people, even people I’ll never meet and who will never meet me, don’t get to know what I do, they don’t get to learn what I’m interested in, and they don’t get to assume that targeting their advertisements will be welcomed.
If I spend my time checking out the details of a surprise visit to Spain, I don’t want the person I’m taking with me to glance at my laptop screen and see that its covered with travel adverts, mix up cause and effect, and think — even just for a moment — that it wasn’t my idea first!
Phorm says that of course I can opt out — and I will — but just because nothing bad happens to me doesn’t mean that the deploying the system is acceptable.
Phorm assumes that their system “anonymises” and therefore cannot possibly do anyone any harm; they assume that their processing is generic and so it cannot be interception; they assume that their business processes gives them the right to impersonate trusted websites and add tracking cookies under an assumed name; and they assume that if only people understood all the technical details they’d be happy.
Well now’s your chance to see all these technical details for yourself — I have, and I’m still not happy at all.
Update (2008-04-06):
Phorm have now quoted sections of this article on their own blog: http://blog.phorm.com/?p=12. Perhaps not surprisingly, they’ve quoted the paragraph that was favourable to their cause, and failed to mention all the paragraphs that followed that were sharply critical. They then fail, again how can one be surprised? to provide a link back to this article so that people can read it for themselves. Readers are left to draw their own conclusions.
Update (2008-04-07):
Phorm have now fixed a “tech glitch” (see comment #31) and now link to my technical report. The material they quote comes from this blog article, but they point out that they link to the ORG blog, and that links to this blog article. So that’s all right then!
Richard, Congratulations on this – haven’t got to read the technical document yet, will do so when I have a glass of wine to hand. Toby
Richard thank you for dealing with the phorm issues, I am pleased that at last someone has seen this for what it is illegal inerception of my browsing habits.
I am about to download this document to read it later on when I have time to relax with no interuptions.
Sir you deserve a drink from us all.
Florence
Excellent investigative piece Richard, well done!
You’ve made the news already with this: http://news.bbc.co.uk/2/hi/technology/7331493.stm
Thanks for all the hard work, very much appreciated.
This tawdry scheme (Phorm) is an appalling attack on our basic privacy.
It must be stopped.
Thanks very much for de-mystifying this. A few quick observations:
a) There is no genuine opt out system. Whatever you do, your HTTP traffic is still going to be intercepted, regardless of whether or not you choose to receive the targeted adverts.
b) Cookies useless in this situation. Any one of my four kids would probably sign me up whilst browsing [what they thought was] the CBeebies website.
c) I’m pretty sure there are a lot more than 25 webmail sites out there. This, together with the other promises to avoid certain sites and keyphrases/names is, at best, going to be more of an art than a precise science.
d) A non-technical point, but the whole thing seems rather grubby – a breach of trust (and quite possibly law) by the ISP, who should be looking after the best interests of the customer. To use FUD to hook users into a cool new free “anti-phishing” tool, when actually they’re doing the digital equivalent of listening into their conversations in order to sell them stuff is pretty low. This is especially true when the main benefit, the “anti-phishing filter” isn’t all that.
In short, the only problem for which this system is a solution is the fact that you may not yet have worked out how to install Firefox and the relevant adblocking extensions.
Thanks for this – will look at the detail once the evening’s imbibing has decayed appropriately.
According to the document the scumware forces HTTP 307 responses. Is 307 implemented in HTTP 1.0? I wonder what happens if I start modifying HTTP headers to HTTP/1.0 in my proxy.
Thanks for this.
There seems to be two possible actions by website operators. Firstly robots.txt could be used to stop the scanning of pages on the system.
Secondly the website may be able to use the cookie that has been set in its name. The paper states that this cookies (cnn.com etc) would be stripped on on the way to the server – effectively hiding Phorm from the server.
However a site malicious to Phorm may attempt to delete or even change the value of the cookie. It is not clear if these commands would be stripped on the way back to the browser from the server.
In either case the server could detect whether Phorm was in operation by simply attempting to set the cookie and then observing that it did not come back (of course a second cookie would have to be set to ensure that the browser is not rejecting all cookies from that server)
Sites could display a “Phorm is watching this connection” logo when it is detected.
From a user’s perspective, given that it only inspects packets on port 80, there’s always the option of asking websites to open up other ports. Alternatively some system that automatically request random web pages when the connection is idle would decrease the signal to noise ratio.
Thank you for a very readable report – especially when there has been so much guesswork floating about as to how Phorm actually proposes to deploy.
Hi Richard,
Great report.
Some minor obs;
– The analysis I’ve done on the UID cookie suggests to me its not a simple number encoded using standard base 64 encoding (as indicated to you by Phorm). If the method of encoding is simply base 64; Phorm should publish the specification so we can decode the cookie and be reassured. To me it seems to be 3 fields, with a non-standard base64 encoding alphabet.
– Requiring web site owners to ‘opt out’ of parasitic marketing (which is likely to benefit only lazier competitors) seems a different value proposition entirely to indexing content for searches (which is likely to be beneficial to the web site indexed). So web site owners should not be required to opt out, particularly so given Phorm is only being rolled out in the UK at present.
– I don’t understand how Phorm can differentiate applications that use the underlying capabilities of Internet Explorer (which present a byte for byte identical user-agent) including Microsoft Office, and Open Office. File/Open/URL is enough to create a web request which cannot be distinguished from IE.
– There are a category of applications which don’t present an address bar interface or store cookies (so would never be able to present an opt out page and/or would not be able to store an opt out cookie). Online help pages embedded in an application for example.
– It all rests on the ‘at present’ predicate
I’ll stop there.
I think the overall conclusion you’ve reached were right, and its an excellent analysis given the limited time available to you.
many thanks,
regards
Pete
Many thanks for this very readable report, which blows away a lot of the smoke around phorm and its methods. As a website operator I’m particularly grateful for the explanation re cookies. I find the idea that my work can be used without explicit consent to plump up someone’s bottom line extremely offensive.
Regards
That’s an excellent idea about sites warning users about phorm. It would be great if someone could package up a phorm-detector so everyone could put it on their websites.
Great article, there’s a flood of new Phorm info hitting my browser this morning, and it’s good to see some of the better researched articles hitting the blogosphere.
Congrats on hitting the BBC with this. I also just finished reading another article on phorm at..
Startup Earth
It seems you can’t trust big media to give you the full story, luckily, there’s blogs like these to spread the truth about phorm’s invasive plan.
So again a corporation assumes that all ‘internet traffic’ is from web browsers… a patently false assumption. There are quite a few things that traverse the net on tcp/80 or tcp/443 that are not from browser applications.
I’ve still to read the pdf, but I assume there’s some discussion of SPOF for users in BT-land when the Phorm system(s) fail? Additionally, why would a network operator put a large portion of their user satisfaction at risk with a third/fourth party they don’t have a direct strangle-hold over?
These schemes (Phorm/BT, PageSense|ProxySense/VZ, Paxfire/VZ/Rogers/others) always seem like such a bad idea for the network’s users.
-Chris
Richard,
Many thanks for the report, it’s very detailed and has proved very helpful in dispelling confusion around the technology; we’re very pleased that you agreed to come in. I’ve posted a response to your security question and will post later on http://blog.phorm.com/
Thanks too for providing clarity around the PII question on ukcrypto and for reiterating that our claim of not storing personal information is correct. We eagerly await the A29 ruling on Monday and hope for a a positive outcome: IP addresses to be designated PII. We also hope for further measures to be put in place to limit timeframes for data storage.
Radha
Server side phorm detector prototype now available to play with. 🙂
Richard,
Your report has been very informative. Thank you for doing this.
Now the final question is why doesn’t Phorm and the ISPs that are selling their customers out realise that we do not want this?
No matter what spin they put on it we don’t want it.
Richard,
Thank you so much for going to the trouble of publishing this detail. It is exactly what I’ve been looking for since getting a basic outline from the diagrams on the Register site.
I am – for the moment – a BT customer and, like you, am not at all happy. I feel that this “trojan horse” proposal breaches the law, and an important psychological contract, and I will be unable to stay with BT if it is implemented. I have asked BT here if I was included in the early trials, and would encourage others to do the same. http://webwise.bt.com/webwise/contact.html
It’s not just a technical issue, but it has been a great help to understand it. Thanks again.
How to learn IPs that correspond to a particular channel:
1 Buy the channel you are interested in
2 Supply to the channel adverts that phone home when viewed
3 Run a web page that serves adverts
4 Correlate times that adverts are viewed with times that particular IPs view your web page
Or did i miss the bit where they say they won’t allow adverts that phone home?
But even without that, when someone clicks a link in a phorm advert, they are telling the advertiser “my browsing habits match the channel you paid for”. This is an extra bit of information, that woldn’t be revealed by clicking any other type of web advert. Will phorm adverts come with warning lables? Will they tell me what i am admiting to by clicking on them?
@tom
Or did i miss the bit where they say they won’t allow adverts that phone home?
No, you missed that the advertiser doesn’t serve the ads and therefore doesn’t learn IP addresses. The adverts are served by the anonymiser machine, which — being in the ISP network — is alteady aware of IP addresses.
Also, the phoning home is pretty irrelevant. Once you click on the advert you will go to the advertiser’s website and they will start the process of knowing all about you!
Having read the PDF, I’ve a couple of queries:
1. Does setting the opt-out webwise.net cookie cause web transactions to bypass the Profiler?
2. Does disabling webwise.net cookies cause web transactions to bypass the Profiler?
3. Does disabling webwise.net cookies function in exactly the same manner as having an opt-out cookie?
This I assume would also cover apps that don’t support cookies mentioned above.
It would appear that the profiler can be bypassed by blocking an app’s “User-Agent” string. Firewalls would be the best way to do this.
So the anonymiser and profiler are on boxes that are (a) gifted to the ISP (b) contain software written by Phorm and patched by Phorm..these devices according to your report know the users IP address and UID.
This is not very private is it. We have to trust Phorm that it isn’t storing the IP address. The fact is, that it could store it if it wanted to. you missed that on point 79..it really doesnt matter that the ISP “controls” the box, the fact is that it is Phorm software on it (and the ISP wont even get the source code for that)..phorm can patch (i.e. change the behaviour) of these boxes at will. The fact that the anonymiser is phorm issued is quite shocking and is the part of the system that should be 100% out of their control as it really holds the key (or should do) to user privacy. How can a company set up to sell as much data as possible (advertisers will want as much data as they can get!) also be responsible for the anonymising of it and expect to be taken seriously?
points 39-45 just show how underhanded Phorm are. They won’t tell website owners precisely what they need to do to block Phorm, and perhaps are even suggesting that the only way to block them is also to block googlebot etc? there should be more explicit instructions given by Phorm on this front. if Phorm were upfront about this they could easily specify and publish a dedicated Phorm robots agent for this. Theres a world of difference between search engine robots and Phorm level-7 stuff!
point 37. Most sites do not use basic auth, they use an application user/pass authorisation, then use a cookie to persist a session id (like PHPSESSID used in PHP sessions). Phorm didn’t address this, so therefore i take it all sites that use session ids in this manner still get trawled.
—
Richard,
can you clarify the advert serving part. once the opted in people get the HTML back, where will the advert image/embed/whatever *appear* to come from, is it from webwise.net or the originating server? i.e. will tools like adblockplus be circumvented by low level devices like phorm?
—
Amk,
see points 27-30 in the pdf but consider this: an interview with Phorm’s CEO stated (http://www.theregister.co.uk/2008/03/07/phorm_interview_burgess_ertegrul/page3.html) this:
“So if I’m opted out, data passes straight between me and the website I’m visiting? It doesn’t enter Phorm’s systems at all?
MB: What happens is that the data is still mirrored to the profiler but the data digest is never made and the rest of the chain never occurs. It ought to be said that the profiler is operated by the ISP, not us.”
what he fails to mention is that the profiler is in fact a Phorm supplied bit of software+hardware.
it appears not, and how could it, they have to at the very least, collect your data, so it can be processed and determined if you have an opt-out cookie in the first place.
it appears the only way your data wouldnt get to the Phormed Layer7, would be a totally seperate ISP intervension that routed your data traffic away from the gifted Phorm kit.
@amk
1. Does setting the opt-out webwise.net cookie cause web transactions to bypass the Profiler?
2. Does disabling webwise.net cookies cause web transactions to bypass the Profiler?
3. Does disabling webwise.net cookies function in exactly the same manner as having an opt-out cookie?
1. Yes 2. Eventually 3. No — see paras #28 and #29.
This I assume would also cover apps that don’t support cookies mentioned above.
in essence yes, but since they won’t have recognised User-Agent strings, they won’t be processed in the first place
It would appear that the profiler can be bypassed by blocking an app’s “User-Agent” string. Firewalls would be the best way to do this
This may impair your experience on some websites, and firewalls tend not to offer this sort of functionality. However, there are firefox add-ons that are relevant. See the Privila story recently blogged about.
Richard,
Phorm’s CEO stated at http://www.theregister.co.uk/2008/03/07/phorm_interview_burgess_ertegrul/page3.html:
“So if I’m opted out, data passes straight between me and the website I’m visiting? It doesn’t enter Phorm’s systems at all?
MB: What happens is that the data is still mirrored to the profiler but the data digest is never made and the rest of the chain never occurs. It ought to be said that the profiler is operated by the ISP, not us.”
so has that changed? as your answer to amk said it skips the profiler.
It seems the only way to full opt out of this is to change ISP. It concerns me that if Virgin Media go ahead with this, then that leaves many people with an option of either VM (with the invasion of privacy) or no internet access at all. Residents in much of Cambridge University’s accommodation can only use VM for their internet access and are not allowed by the university to use another ISP.
A quick look over personal firewalls suggests they won’t, although some (Sunbelt) will filter Referrer headers.
Proxies, including Privoxy, will certainly be able to filter User-Agent headers.
Privoxy
I’m wondering if a proxy could be configured (or modified) to take and serve the webwise.net opt-out cookie so a network behind it could opt-out together. Possibly a custom Squid redirector? Squid would hardly be a good solution for the average home user, but could be good for business customers. And nerds.
Am I right in thinking that the browsing performance for those who opt out could be worse than those who consent to having their web browsing classified by Phorm?
This from s26 of Richard’s report, the apparent fraudulent placing of 1st-party cookies within the domain of the website being visited by Phorm so that further visits to that website will not require a 307 redirect, hence Phorm’s claim that “roughly 99% of the stream is untouched, with no redirect at all” in this blog.
It follows that all HTTP GET requests will be redirected at least twice in order to read the presence of the opt-out cookie, contrary to their claim on the above mentioned blog, unless they plan refinements whereby IP address of those opted out are remembered by the profiler. Or did I miss this in Richard’s report?
The day Virginmedia use Phorm is the day i have no further use of the Internet, i will cancel my subscription.
While the ICO statement concerns itself with the data processing of Phorm I would question what rights BT or other ISPs have to process my traffic data for this purpose. Sure they have rights to process my data for the purpose of providing me a broadband service e.g. for bandwidth management, security etc. Where/when did they tell me they would also be collecting my traffic data for this new purpose?
First, thanks Richard for your report.
I, like many many people, run hobbyist, special interest, websites with ‘semi-private’ areas. That is they require a user ID and password to access certain pages but the content is not encrypted via SSL.
I have been following debates about Phorm for a while now in effort to find out how I can stop ‘private’ areas of my websites being intercepted by the Phorm system.
From your report (para.26) it would seem that I can add some code to my site that will can check for a ‘webwise’ cookie and act accordingly (ie. deny access to content – a somewhat drastic option). Is this correct, or will the cookie always be removed before the site gets to see it (para. 22)?
It also seems (para. 39+) that I can stop Phorm from intercepting my site content my inserting the appropriate User-Agent (if I know what it is) into a robots.txt file. Is this also correct?
These ‘solutions’ do of course take on trust that Phorm and the ISPs involved will implement & honour what you have detailed in your report, and given the lack of transparency so far I remain to be convinced.
As a website owner I do not want my site content intercepted and require some way of being able to identify if Phorm is being used.
One other question: Your report (para. 4) says that this works at the HTTP level and so can see all parts of the request. Many blogs use cookies to hold user ID and password (e.g. WordPress – though password is encrypted). Are website cookies going to be available to Phorm?
Again good work on the report.
Regards
Ian.
Richard,
Your last post is rather misleading. The link to your report is up on our blog (http://blog.phorm.com/) — it was added yesterday evening after a tech glitch (as we note on the comments page). There is no intention to not display your report in full. As I have made very clear, here (look up ^^^^) and elsewhere, we appreciate the report and welcome it. We’d be nuts, frankly, not to publish in the light of how we feel and have openly said we feel about the report.
You also overlook the fact that in the very post you highlight (where you claim we offer a partial view) there is a link to the Open Rights Group site, where your intro is there for all to see plus links to your report (not to mention unfavourable mentions of Phorm if you scroll down…)
I wonder if you’ll post this on your front page…
Best wishes,
Radha
It seems to me that a web site can identify your browsing habits by using the Photm cookie.
1. visit example.com using the phorm system.
2. example.com (a non-phorm particpant) captures the webwise cookie and creates a request to to the webwise adservers.
3. Analyses types of ads returned and deduces my browsing habits.
This leaks the information that Phorm gathers to the point that any web site, participating in phorm or not, can infer the types of site the user visits. If the website in question has user registration/tracking then it can correlate your ID with your phorm UID.
So regardless of the privacy and data issues within Phorm, the data that they leak to 3rd parties is very undesirable.
Thanks for the front page clarification! 🙂
Nice piece of analysis. Redirection and forged cookies – how does that sit with the ISP remaining a “mere conduit”?
Sadly, my recomendation to APIG that falsifying the source of a communication become an offence under the amended CMA was ignored.
Worth a peep at the Universal Declaration of Human Rights too – Article 12 is the one of interest:
“No one shall be subjected to arbitrary interference with his privacy, family, home or correspondence, nor to attacks upon his honour and reputation. Everyone has the right to the protection of the law against such interference or attacks.”
So assuming Phorm are protecting my privacy (I have some reservations that they are adequately) they are arbitrarily interfering with my correspondence. It’s time that those who make the statute stand up and offer us the protection of the law instead of brown-nosing anyone wafting a suitably large bundle of cash.
Well I have moved from VM to an ISP that assures me they are not going to use phorm or anything like it. I will still stand and fight phorm as this is an invaision of human rights. I think perhaps the best way to get the government to stand up and listen to us is to give them a swift kick to remind them. They are only in government due to us voting for them. Let us give them this message in the may elections.
Re: Barrie
One thing to point out, and a response Phorm might try and use, is that in the ‘default’ situation what you describe couldn’t happen, as the L7 switch strips off the forged cookie before it gets to example.com.
It would still work however if someone who used an ISP using phorm in one location, got the forged cookie for example.com, then connected to it again through an ISP not using phorm, as the cookie wouldn’t be stripped in that situation. Another thing that *might* work (I’m not an expert on cookies and browser security so don’t know for sure whether it would get sent in this situation), is redirecting users to a custom port, e.g. example.com:81, at which point the L7 switch will I believe, ignore it…
“add a cookie that they forge to appear to come from someone else’s website”
I know it’s a terrible pun, but bad phorm chaps.
They ‘say’ they ignore other UI strings, but how do they ignore all Microsoft products that are not Internet Explorer? ALL their products including the PS3 etc, use the same UI string.
They have no moral right to even think they can make money out of my web habits – at least not without reimbursing me – and I don’t mean the figment they call WebWise.
For the kinds of sites that are likely to match Website channels (sports, travel, etc.), opting out of search engine indexing solely to avoid Website co-option is not a viable option. Thus, I don’t see any way to describe it as even “going a little way towards” getting permission to scrape the site. At best, it’s modestly privacy-protecting.
Unless there’s some way for a web server to distinguish a request for robots.txt from Phorm and one from Google and react accordingly?
@Barrie & @Alex Brett
I think another feature of their system block that particular leak: to request an advert example.com merely tells the browser to request an advert from phorm – the advert goes direct from phorm to the browser and example.com never gets to see it.
I think you could do it by collecting {UID/IP} from example.com by directing browsers to an https page (so phorm doesn’t strip the cookie), then use a machine served by a phormed isp to work out UID->channel mappings (thanks to the roaming feature, this machine can pretend to own the UIDs you harvest from example.com).
@Richard Clayton (comment 21)
Thanks, but I think the idea I was getting at still stands: the advertiser must have some say over the content of the advert they are buying. By ‘adverts that phone home’ I meant that when the phorm system displays an advert, its (advertiser chosen) content would cause information to flow back to the advertiser, and even if the information is just a time-stamp, it might be used to associate IPs with a channel.
The two issues I find interesting are:
1. When I click on a phorm advert, I reveal to the advertiser that my browsing habits match the channel they paid for. They can then remember this for as long as they care to. How much do I care about that? How does this compare with the information collected by conventional web advertisers?
2. Can the advertiser (or anyone else) gain information about the channels I match without me clicking on their advert?
Issue 1 is about the morality of channel-based classification of browsing habits. A lot seems to rest on the construction of the channels – they quote some limitations they are imposing, but I am still uneasy.
Issue 2 is a technical question: just how invasive is the system? How much information leaks out of it?
Also, I would like to see what an economist makes of their claim that their system will tend to reduce the quantity of adverts on the web.
To me it’s rather like arguing that putting your business’ phone number in the Yellow Pages gives advertisers implicit permission to listen in on phone calls between you and your customers.
Is it a coincidence that the former CTO of BT is now CTO at Phorm…………
what questions were asked at the “Focus Group”, along the lines o “would you like your ISP to protect you from all the bad things on the internet without it costing any extra and not having any new software to install / update on your computer?”
well its not really the place other than ro ask did anyone look to AOL for any of this interception tech
the reason is simple if you follow the logic
Stratis Scleparis seems to like this interception cashcow,and was chief technology officer at BT Retail, he’s now with Phorm as we know.
he also held senior technology management roles with leading firms Orange UK plc (formerly Freeserve/Wanadoo), AOL Europe and the BBC.
we can discount the BBC (cant we?) but its already been reported by ElReg about Orange Uk plc suppllying customer data in the Mobile Phorm story, did we forget about that one people?
http://www.theregister.co.uk/2008/03/12/mobile_phom/
“…Orange UK apparently “supplies Xiam with data including billing information, mobile browsing logs and purchase history…”.”
so given Stratis Scleparis’s likeing for this tech, is it not reasonable to look at AOL ?
if anyones interested , there a new petition up regarding BT and RIPA
http://www.petitiononline.com/BTRipa/petition.html
Surely the key question is, how does this compare with other systems for targeted advertising? I presume that other systems, whether they use tracking cookies or search engine data or what, also keep correlated information. The Phorm approach is clearly different on a technical level but how does it rate on policy compared to existing systems?
From what you’ve written here, I glean the following impressions:
– as the system is hosted by the ISP, it occupies a different relationship with the user. Whereas a user can decide on a case-by-case basis whether to trust individual sites or cookies, Phorm is all-or-none.
– Phorm keeps personal information separate from tracking information, as is done by the best existing sites but not by most.
– The system should have better methods for users and web sites to be able to opt out of the system.
There is also some jiggery-pokery with cookie impersonation, which I suspect will only ever be of concern to techies – is there any reason why an average user should care about the method used for tracking activity, as opposed to whether the activity is tracked at all?
@45 good way of putting it.
I run a few web sites and one in particular we really want a good Google ranking so that people who might benefit from finding it can. We really do NOT want Phorm to access it and then feed people ads that will probably be exactly the kind that they do NOT want, and the second I learn how to deny them I’ll do it.
I don’t want them on my own business site either because I’m amazingly less than keen on potential customers being bombarded by ads from others offering a similar service. Duh!
Finally as a BT broadband customer I do NOT want my traffic going through any extra servers and to be fed fake cookies. I want to be connected directly to the site I asked for, that’s what we PAY them for after all.
So, ISP(s?) are being talked into voluntarily installing software and hardware that intercepts user traffic and the ISP(s) argue legal responsibility for doing so, and by doing so they might risk voiding the ISP’s safe harbour claims? Possibly to lose many customers? Possibly real damage to brand trust? Maybe even global issues arising from this? No, impossible, simply not true! That would be bad business and would piss off investors. So it sounds like they found something really worth while investing into. Must be big.
I demand my basic human right to privacy in the short, medium and long term. I demand basic human rights for my descendents (sp? not important, message is important here).
Ethics? Morals? Inconvenience and interrupted user experience? Time? Total costs to users (implementing interception detection, breaking existing software, to exclude webwise via robots.txt is to opt out of search engine indexing, etc)? Lessons from history? Present versus future state of the software? Net neutrality? Monopoly (unfair) over the advertising industry? National and later global broadcast capability via adverts (yes, this means more voice than any politician; read: power and control, again monopolistic in nature)? Inability for relevant policing bodies to act in a rapid response manner? Nobody actually looking at the software’s source code to confirm if this is actually the case that’s being presented to us, the dumb users who weren’t supposed to know until we found out and now it’s all about transparency? No regulation of updates to the software by a disinterested third party? So many questions. Not one answer.
I enjoy the service I get from Virgin Media, but if this happens, I’m gone – interception is not a reason to sign up with any ISP – think: clean conduit ISP, fast, 1:1 contention, ethical, price, and throw in static IPs, DNS and rDNS. VM would be a hero of sorts if they said ‘no’ to interception of any form (I accept warranted interception, but this is different). It would probably swell their profits like crazy with new user sign ups. We don’t care if that is the obvious route, just do it. We need a hero. We need an ISP who understands these important issues. I personally don’t even care if VM have to raise my Internet bill by a few quid (do the math) – a whole lot more value for money than this interception promises for both me and the bottom line of the ISP. Did you know I don’t even see adverts!!! My eyes avoid them and dart towards actual content of the page that I’m after. I point blank refuse to click adverts. This is the real issue, nothing to do with targetting. Hell, people even had to explain how wikipedia works to these “clued up” people. They live in a bubble IMHO and frankly I would regret seeing anyone who is like that to rule anything at any level.
Even if an ISP offers opt-in/out at the ISP account level (via different routers etc), I will vote with my feet.
I DID NOT SIGN UP FOR THIS INTERCEPTION SERVICE AND DO NOT WANT IT. THIS IS LIKE BAIT AND SWITCH! (yes capital letters is Internet speak for shouting, I’m upset and I personally have no voice of consequence against these people who hide behind businesses but seek to affect my individual rights directly)
There is no use case of this technology that satisfies me personally. You can’t force me to share your humble opinion.
YOU MAY NOT USE MY DATA TO MAKE UNATTENDED DECISIONS ABOUT ME. PERIOD.
Richard
Great technical write up – it tells us a lot more about how the intercepts are done.
Several other people on here have raised the issue of the robots.txt file. Phorm say they will honour it but wont tell anyone what to put in it to stop Phorm, and only Phorm, from scraping a site. Most website owners are happy to allow things like Google and Microsoft Live scrape the public areas of their site as it brings them more traffic from search results.
Do you know if Phorm (who at least were reading here) are going to release the information for robots.txt? I assume that if they say they will simply piggy back on the robots.txt entry for one of the major search engines then that company will have some rather unpleasant words for Phorm.
It is completely unreasonable for Phorm to expect the entire world wide web to suddenly run about, changing their web sites based on a whim of Phorm’s. No RFCs, nothing.
Phorm are doing exactly what these laws were set up to prevent. Why is it taking so long for this to be binned?
ISP’s recently stated that they could not be expected to “police” the Internet because they do not have the technology to monitor every customers activity. But Lo and Behold they do have the technology to monitor every customers activity if it means commercial gain.
They say they cannot block or monitor a minority of people using illegal file sharing sites or pedophiles visiting porn sites but they can block suspected fishing sites?. I do not require any external anti-phishing device, if I come across a phishing site I report it to the relevant bodies. It would now appear that those reports will be put towards creating revenue for webwise to use as a sales tool. This whole idea smells of hypocrisy by the ISP,s and a gross invasion of privacy because there is no real opt out available except to change ISP
@Tony
ISP’s recently stated that they could not be expected to “police” the Internet because they do not have the technology to monitor every customers activity.
that’s not exactly the reason they will have given… the fundamental reason is that the technology has significant limitations in what it can detect and the blocking is easy to evade. Phorm aren’t trying to solve quite the same problem (they only obtain a rough idea of what a webpage is about, far too vague to consider whether it should be blocked or not).
Also, their phishing detector has almost nothing to do with the rest of their system (the cynical would say it was just bolted on the side to help sell the system to customers) and has all the limitations of every other blacklist-based detector… so it’s better nothing, but uncritically relying on its opinion would be unwise.
I have just been browsing the BBC tech news and found this interesting item of news
http://news.bbc.co.uk/1/hi/technology/7246403.stm
A spokesman for the Internet Service Providers Association (ISPA) said the 2002 E-Commerce Regulations defined net firms as “mere conduits” and not responsible for the contents of the traffic flowing across their networks.
If they didn’t do traffic management we would all complain
He added that other laws on surveillance explicitly prohibited ISPs from inspecting the contents of data packets unless forced to do so by a warrant.
I admit that this is a snippet but the article is worth reading because it again shows the manipulation of the law by the ISP’s. The technology in question (phorm) is a packet sniffer being used without warrant. So where do the ISP’s stand on this matter. It would appear that their own spokesperson is pointing out the illegality of their own proposed actions.
Interesting link Tony – it also contains the following : “He added that other laws on surveillance explicitly prohibited ISPs from inspecting the contents of data packets unless forced to do so by a warrant. ”
Virgin and BT are both members of the ISPA. I assume that they will be booted out of it if they implement phorm.
http://www.opsi.gov.uk/si/si2002/20022013.htm
”
Mere conduit
17. – (1) Where an information society service is provided which consists of the transmission in a communication network of information provided by a recipient of the service or the provision of access to a communication network, the service provider (if he otherwise would) shall not be liable for damages or for any other pecuniary remedy or for any criminal sanction as a result of that transmission where the service provider –
(a) did not initiate the transmission;
(b) did not select the receiver of the transmission; and
(c) did not select or modify the information contained in the transmission.
(2) The acts of transmission and of provision of access referred to in paragraph (1) include the automatic, intermediate and transient storage of the information transmitted where:
(a) this takes place for the sole purpose of carrying out the transmission in the communication network, and
(b) the information is not stored for any period longer than is reasonably necessary for the transmission.”
the key words are “for the transmission”
the ISPs fails on all counts when it involves add-On 3rd aprty profit module from Phorm.
@davidM
I think you misunderstand the law you quote. It provides an immunity (but perhaps not the only immunity) for an ISP if they would otherwise be responsible for content.
The Phorm system interferes with transmission (because of all the redirects) but after that there is no initiation, no selection, and no modification of what flows between the remote site and the user.
Hence, in the unlikely event that you can hold the ISP responsible for the content (and this would be uncommon because there are a lot of other protections for the ISP besides this fallback transposition of the ECommerce Directive) then it is far from clear that the ISP will be unable to avail themeselves of this particular statutory protection.
To put it another way — Phorm’s system may well not be lawful to operate for several reasons. A very minor risk that the ISP could lose one of many statutory protections from being sued for someone else’s actions is not especially relevant.
IANAL
A: fails, they did initiate their part of the interception.
B:fails,they did initiate their part of the interception.
C:fails BIG time, they did modify the information contained in the transmission.
2A: fails, its not part of the basic “mere Conduit” its part of this 3rd party add-on for profit and not aparty to the consumer contract.
2B fails, no consent from the user to collect the data, never mind process it or store it in computer ram or on harddrive.
it would seem by freely entering into this Phorrm contract, to freely monitor 3rd partys (the ISP’s end users and websites) datastreams, they are clearly giveing up the right to their current EU legal protection under the “mere Conduit” directive.
BTW. heres some of the exclusions they cant ignore
“Exclusions
3. – (1) Nothing in these Regulations shall apply in respect of –
(a) the field of taxation;
(b) questions relating to information society services covered by the Data Protection Directive[9] and the Telecommunications Data Protection Directive[10] and Directive 2002/58/EC of the European Parliament and of the Council of 12th July 2002 concerning _the processing of _personal_ _data_ and the _protection_ of _privacy _in the electronic communications sector (Directive on privacy and electronic communications)[11];….
….more.
“
am i wrong in any part of this posible interpretationrichard, if so were did i go wrong?.
it seems at least one logical way to interprate it in my Opinion, is it not?
@davidM
No you’ve still missed the point — what liability do you think will result from the part of the communication where 307 redirections have taken place — where (a) and (b) apply ? Liability is only likely to occur (and it’s pretty unlikely even then) when the traffic flows to the remote site and that is not particularly interfered with.
They do modify (albeit in a very subtle way) the traffic to the remote site by arranging for an extra cookie — but that doesn’t actually reach the remote site… so diifficulties here as well.
So I don’t agree that ISPs are giving up their mere conduit exemption. Even if they did, so what? they are still protected by other legislation (such as the Defamation Act 1996), by common law concepts such as mens rea etc etc.
It may not be wise to put mere conduit at risk, but (a) I don’t think it is especially obvious that they have actually done so, and (b) even if they have, it’s not the end of the world for ISPs in terms of the legal risks they actually run.
You could make a much, much stronger case that the blocking of illegal images of children (as listed by the IWF), has cast mere conduit protection to the winds for some time now — since most. UK ISPs have such systems in place. The loss of “mere conduit” protection that this engenders has not caused the sky to fall.
So sorry, but this is an unconvincing legal avenue.
thanks richard, yes, wrote 60 before i read your 59 as i wanted to keep the mere conduit post clean, your to quick for me 😉
im taking in what you say and will consider it…
perhaps other readers might also give a view to what you said.
thanks
Does this mean that not only everything I read, but also anything I write (except to a secure website) might be intercepted? People might be more concerned about Phorm if they knew this…
Will all opt-in pages include the words :-
“As you browse, we’re able to categorize all of your Internet actions, we actually can see the entire Internet.”
It might be worth pointing out that anyone who wants to know more, merely has to google the phrases :-
“we actually can see the entire Internet”
or
“we’re able to categorize all of your Internet actions”
Worryingly, from the search terms returned, this seems to be a quote universally associated with Phorm.
Have Phorm a trademark on these phrases?
Not a very good result for a company barking on about “privacy”.
Richard,
Mere Conduit argument aside, you cannot honestly believe that this technology doesn’t breach RIPA, DPA, Fraud Act, Torts (Interference with Goods) ActPrivacy and Electronic Communications (EC Directive) Regulations.
RIPA is requires consent from all parties in a communication (s1)
DPA classes -any operation- as processing, so even the very first stage where the Layer 7 technology uses DPI on the traffic data to determine consent, it is in breach of the Act. You can’t break the law to find out if you have permission.
Fraud Act clearly makes the 307 redirects illegal under section 2.
Trespass to Goods (interference with goods) also seems very clear.
PECR is just full of clauses this technology contravenes.
The very fact they they need to use DPI and interception to determine the presence of a cookie is where this entire system falls down and is quite plainly very illegal violating multiple Acts.
The only possible way this system could be legal would be for either a: over half a dozen laws change -or- b: the opt-in/out is done at a different point in the network that doesn’t go anywhere near the Layer 7 tech.
As for the trials in 2006/2007 it is atrocious that action by the Home Office hasn’t already occurred, the list of offences is remarkable and on a scale I can’t find ever happening before.
We have the following problems with the 2006/2007 trials:
European Convention on Human Rights
Human Rights Act
Regulation of Investigatory Powers Act
Privacy and Electronic Communications (EC Directive) Regulations
Computer Misuse Act (even the English version of s3 of the Act is easily representative of the trials which is notoriously difficult to use in England and Wales).
Fraud Act
Torts (Interference with Goods) Act
Data Protection Act
Copyright, Designs and Patents Act
That is a pretty impressive list of violations even for a company borne out of Spyware.
Then there is the Council of Europe’s Convention on Cybercrime which is supposed to illustrate how we should be behaving yet when applied to the trials of 2006/2007 they are so out of sync with the convention it becomes sinister.
Time and time again people have asked about the secret trials which given the numbers of users must have involved literally millions of interceptions, bringing the count of likely criminal violations to a staggering amount never seen before in the history of a telecoms.
I can’t even type anymore I am so infuriated by the whole thing, seriously the whole affair is just utterly repulsive at every level.
Alexander Hanff
Oh Richard, the first paragraph of my last comment was rhetorical by the way, I wasn’t suggesting you think it isn’t a breach of RIPA, you have clearly stated otherwise.
Alexander Hanff
What has everyone made of the revised statement from the ICO?
http://www.ico.gov.uk/about_us/news_and_views/current_topics/phorm_webwise_and_oie.aspx
It is certainly better news than the initial one, but I can’t help feeling that it is still a bit toothless.
is anyone not already reading the CF threads going to the london PIA meeting tonight?
pass the word, do it NOW, and get there if you care about this Interception for profit, and can make the effort, people…
and also felicitations to Alexander Hanff above for being invited to the speakers podium and being the voice of the people tonight, well done.
http://www.cableforum.co.uk/board/12/33631213-phorm-public-meeting-official-thread-page-3.html
Very, very interesting analysis/article.
Being a Virgin subscriber I’m very concerned about this. Being a web developer, I’m even more concerned.
The privacy angle has already been discussed, but there is another slightly less concerning angle from the point of view of application and site security. Not only is information about the user collected (and supposedly anonymised), but there is also a wealth of information about each site and it’s user population that could be collated as well.
I haven’t seen anybody comment from this angle, but what’s to stop Phorm collecting statistics and information about a website and it’s usage statistics and selling that information on, or even more worryingly offering some kind of service to a competitor to watch a given web site.
As an example, imagine a competitor to say Amazon.co.uk, being able to analyze the usage patterns of users on Amazon’s site, down to which books are the most popular at the moment.
Imagine also mining the data and classifying users, you would then be able to provide information about the demographic make up of that site’s users.
Nothing, other than the kind of anonymous HTTP proxies that we’re already used to seeing should get between the user and the data their receiving from any websites they visit.
Great post. Huge response. I guess people really care about this.
My reading of the article suggests two simple ways to bypass phorm:
(1) block all cookies from *.webwise.net in which case your traffic bypasses phorm’s logging servers after the first couple of requests.
(2) set a non-standard User-Agent string (ditto).
Accepting the “opt-out” cookie is not a good alternative, as you’d have to opt-out again whenever you delete your cookies.
Excluding 25 Webmail clients is totally inadequate. The information entered on contact forms is probably more sensitive, there are millions of these, and they are not all https:, for example:
http://www.webwise.com/contact.html
Excellent explanation of the Phorm process. Assuming it is accurate, then I believe the following JavaScript added to a web page will warn if Phorm has added a cookie. I’ve left out the script tags in case the blog software dislikes them.
function hasPhormCookie()
{
var found = false;
var c = document.cookie
if (c.length>0)
{
// If phorm change the name of the cookie they insert
// you need to change the comparison string to match
if (c.indexOf(“webwise=”, 0)>-1)
{
found=true;
}
}
}
if (hasPhormCookie()==true)
{
document.write(“Phorm is monitoring your traffic”);
}
Apparently, it dislike the “return found;” line in the function as well.
Comment 72 is an interesting way of detecting the phormjacking of your page. However, I’d be even more interested in a way to *prevent* it. Anyone found anything?
If you don’t want Phorm (or any of the other packet sniffing systems) to read your page, use https. Simple and straightforward and that should be an end to it.
More complex (and very Phorm-specific) disruption would involve changing the content of Phorm’s cookies (the ones in your website’s domain). However, until there is an operational trial, you cannot be sure what the deployed technology will exactly be.
Me again. I’ve now read your comprehensive PDF document (thanks for making it public).
I may be missing something, but I now figure that, as a website developer, if I set a cookie on all my domains with the name “webwise” and the value “OPTED_OUT” then pages in these domains will not be diverted/profiled, even if a person is visiting them via a phormed ISP.
I got this from paragraph 28 of your document. If my interpretation is wrong, please add a comment to let me (and others) know – many thanks.
Sorry, didn’t see comment 75 before I posted 76. We seem to have both hit on the same area – so I guess I’m kind of right, but would need to see if they changed the name/value of the false first-party cookie after they go live. Thanks for the feedback 🙂
If Phorm adds an extra cookie without the web site owners or customers knowledge and that causes a problem (there are maximum numbers of cookies in all browsers except safari) who would be held accountable.
Here’s a silly example (because you’d be mad to implement something this way; you should be using HTTPS; and I have wildly exaggerated the possible consequences). Let’s say I run a charity web site, my 20th cookie holds the currency for a donation via bank transfer. The Phorm cookie is added without my knowledge, which means that the IE6 browser (limited to 20 cookies per domain) drops the currency cookie (now cookie 21). My web site happily transfers 50,000 pounds to Save the White Whale instead of 50,000 pesetas. The customer is bankrupted by hideously unfair overdraft charges but Ahab is confounded once more.
Who should the customer sue?
P.S.
You could presumably disable the Phorm cookie by adjusting the JavaScript to delete the webwise cookie if it finds it. Phorm would do it’s infinite loop detection and blacklist the web site. I didn’t want to include code for that in case my ISP took umbrage and disconnected me.
Thankyou for your work looking at Phorm, it is really appreciated.
Forgive me if I’m wrong, but with regard to using HTTPS.
If the Customer uses HTTPS did not part of your summary describe that the Phorm System cannot remove the UID from an HTTPS transaction.
This leaves the door open for any HTTPS Website (rogue or otherwise) to match the UID to a Real IP address or E-mail account, with or without the help of Javascript.
The use of Javascript could also lead in some cases to the Computer Users Account name being discovered!
How can this be called an advancement in Privacy as the ISP’s are trying to claim.
Has anyone even looked at how this is going to affect businesses? I know that where I work, such a deep intrusion into what business we do with other companies and what is sent between us would NOT be welcomed. Matter of fact, considering much of it is proprietary, it may even be grounds for a lawsuit if they’re attempting deep analysis of data packets.
A question: In the article you mention that the Channel Server would located at the ISP site. Does this imply that it would have a domain name similar to the ISP’s main one or something different?
The article seemed to confuse the Anonymiser and Channel servers. Are they one and the same?
@Alan
A question: In the article you mention that the Channel Server would located at the ISP site. Does this imply that it would have a domain name similar to the ISP’s main one or something different?
the article does not imply that it would have a host name at all, leave alone which domain it would be in
The article seemed to confuse the Anonymiser and Channel servers.
sorry!
Are they one and the same?
no
All of this appears to ignore the fact this is not merely cookie insertion & forgery, it is also “Browser/Application Hijacking” & “Browser/Application Forgery”, anything can be added or removed without the express wish of the Account holder!
I have read all the technical and legal analysis documents.
I thought I had a rough understanding of how this system’s hardware will sit.
As I understand it, there is a profiler/anonymiser that sits within the ISP network (or is supposed to) which collects the data and anonymises it. This is passed to a Channel server. Where does the channel server sit? Is this meant to be in the same location as the profiler or is it located elsewhere / not within the ISP network?
The channel server is located at the ISP (see #53 in the tech document)
Richard, re. your Update (2008-04-06) where you notice that, “Phorm have now quoted sections of this article on their own blog: http://blog.phorm.com/?p=12. Perhaps not surprisingly, they’ve quoted the paragraph that was favourable to their cause, and failed to mention all the paragraphs that followed that were sharply critical.”
Just noticed that they are still using the same single paragraph from your report to justify their system. Spotted this in a reply to an interview ISPReview had with Alex Hanff.
http://www.ispreview.co.uk/articles/08phorm/03.shtml
Thank you for your efforts regarding this matter. I find BT’s behavior very offensive and i regard any snooping as illegal and it amounts to wiretapping/eavesdropping.
How on earth the authorities have not gotten involved yet is beyond me but then again they cant even keep track of their own records as has been proven time and time again in this last year alone.
We will have have to vote and move with our feet/wallets when this goes live. I would like to know if i can get out of my contract based on not wanting to take part even though i have still got 6 months or so left.
thanks much for your diligence and hard work.
The worse thing about this PHORM, as in all these cases, is IT IS THE THIN EDGE OF THE WEDGE.
Once this is deployed and assimilated -and forgotten about by the main stream media- it will then be no problem for these details to be sold to WHOEVER THEY LIKE.
@88
From what ive read the ISP companies will be required to amend thier terms and conditions to implement Phorm, with a change of terms and conditions you have the right to ‘not agree’ to them to which I doubt they would have any legal standing in terms of remaining contract lengths as they have pro activly changed thier service to you.