mozilla / lightbeam

Orignal unmaintained version of the Lightbeam extension. See lightbeam-we for the new one which works in modern versions of Firefox.

Lightbeam for Firefox README

This is an add-on for Firefox that helps to visualize sites that may be tracking you around the internet. The add-on is available from https://addons.mozilla.org/en-US/firefox/addon/lightbeam/.

Prerequisites

• Mozilla Add-on SDK
• Firefox 38 or higher.

If you only want to see the demo, you don't need any of these; all you need is a modern browser.

Quick Start

First, install the Mozilla Add-on SDK if you haven't already.

Then, check out the Lightbeam repository and enter it (do this where you want the lightbeam directory, not in the addon-sdk directory):

git clone git://github.com/mozilla/lightbeam.git
cd lightbeam

Finally, run:

jpm run

This will start Firefox with a temporary profile that has Lightbeam installed. Just click on the Lightbeam icon at the bottom-right of the browser window to open the web front-end.

At this point, any changes you make to the web front-end simply require reloading the tab containing it. Changing the add-on, however, will require quitting Firefox and running jpm run again.

Software Used

The following software is bundled with the repository and doesn't need to be manually obtained.

• D3
• PicoModal
• parseUri

lightbeam's Issues

Provide audio output

If the aim of this is to freak people out about being tracked, how about a version with audio?

First, I thought: why don't I have this open in the corner while I'm browsing, so I can see the changes? Then I thought: it would be much better if I could hear myself being tracked.

Imagine this: a low "bloop" when a new tracking website appears. Each time a link appears to an existing tracking website, you get a "bloop" one note up the scale from the last one. I.e. the pitch depends on how many links are already there.

In other words, after a bit of browsing, new links to Doubleclick would be high-pitched "beeps". As the pitch rises, so would the sense of someone keeping a very close eye on your browsing. As the user surfs around, they click a link and hear "boop" "beep" "beeeep" and think "Goodness, I click one link and three companies are tracking me!"

I guarantee this sort of thing will drive adoption of anti-tracking software. Try it! (Use, of course.)

Gerv

SVG region doesn't resize when window size is changed

So for example, if I make my window bigger, the SVG doesn't expand to fill the window -- there's an invisible line at the edge of the SVG where the graph gets cut off. The extra space goes unused.

Should add an onResize handler to the window that resizes the root SVG element.

Export your graph as an image

So you can share it with other people easily. Maybe even have a "share" button that posts your exported image to your social network / photo hosting service of choice?

Graph isn't displayed on demo mode

When running Collusion in Firefox under "demo mode" (i.e., without the addon installed), the graph doesn't display as of commit 0abe93e.

Graph gets all flickery when you're dragging a node

Because the act of dragging a node means repeatedly mousing over and then mousing out of a the node you're dragging, and possibly other nodes as well, which keeps triggering the fadeout/fadein of the other nodes.

Need to disable the fadeout/fadein while you're dragging. Leave it in one state. The mouseover/mouseout handlers should not do anything while you're dragging.

How to implement this, though? There's not a way to poll the current state of the mouse button - we have to use an event listener. Thing is, force.drag is capturing the mouse down events on the nodes, so if we try to add an extra listener for mouse down, either force.drag won't get the event, or our extra listener won't get the event.

Possible solutions:

1. have our listener trigger first and not consume the event (how to do this?); maintain a variable to track state of the mouse button, have the mouseover/mouseout handlers return immediately if the mouse button is down. Will also need to catch the mouseup event whether it is in a node or not -- you might let go in a node or you might let go over the background, so probably there needs to be multiple mouse up handlers.

2. This would all be much easier if we could just ask the force object in d3.layout.js to tell us whether the mouse button is down or not, since it is already maintaining this state internally. However, it doesn't expose its internal state (the variable is hidden using closures). There's a fairly trivial modification that would make it expose this state:

   force.isDragging = function() {
   return (d3_layout_forceDragNode != null);
   };

... which our code could then easily use to solve this problem. However, modifying a 3rd-party library could cause maintainance problems especially when that library issues an update of its own.

Favicon not always found

For some sites, domain.tdl/favicon.ico does not resolve. The result is that we create an svg:image that fails to load.

Could try looking at some other URLs to find a favicon, or we could detect the failure of the image to load and draw something else in its place.

Currently using a hack where we draw a circle and then draw favicon over it, so if the favicon doesn't load the circle is still visible. But it's not great.

Scale graph to better fit available screen area

Right now when graph is small, most of the screen goes unused. It could draw a lot bigger.

When the graph gets big, it seems to be trying to fit inside the available vertical space, but sometimes it's not quite perfect and then the vertical scroll bar appears. The main window area should not be scrollable! (although the sidebar might have to be scrollable if there's too much text in it).

Also, the graph when big uses up all the vertical space, but not all the horizontal space, since it seems to want to be vaguely round. I wonder if there's a way to get it to spread out more horizontally to use that space better?

Let me whitelist sites

If I know a site is OK and don't want to see it on the collusion graph anymore, I should be able to click the node and say "This one's OK", then have it hidden by default.

Report to user whether sites are respecting Do-Not-Track

If user has Do-Not-Track turned on, Collusion should tell the user as much as possible about whether or not a site is respecting that setting. (Not sure how technically possible this is, but it would be really cool.)

Also, have an option on the Collusion page to turn on Do-Not-Track.

Following a link to another site always indicates a behavioral tracking relationship

Clicking on the "privacychoice.org" link from collusion.toolness.org, for instance, connects the two domains on the collusion diagram and claims that privacychoice.org tracks your behavior across toolness.org.

All this means is that privacychoice.org sets a cookie, and that it knows I came to it from collusion.toolness.org. While they could use this for "tracking", it's far less insidious than what we consider behavioral tracking, particularly since the user has to actually click on a link on a specific page for the linked-to site to know where the user came from. Including e.g. an image or script to the tracking site on every page of one's site is much more akin to what we normally consider tracking to be.

Add data about +1, Like buttons to Collusion tutorial

It's mentioned in the last page as a reason to try the addon, but it'd be nice if the data was actually in the tutorial.

First suggested by knowtheory here.

Exceptions thrown when serving pages from 127.0.0.1

Atul, I am getting the following exceptions when serving up the browserid.org pages from localhost. I am using http://127.0.0.1:10001 as the URL. The following exception is thrown, as seen in Firebug:

An exception occurred. Traceback (most recent call last): File "resource://jid1-f9uj2thwoam5gq-at-jetpack-api-utils-lib/observer-service.js", line 174, in null this.callback(subject, data); File "resource://jid1-f9uj2thwoam5gq-at-jetpack-collusion-lib/main.js", line 74, in null var domain = getDomain(channel.URI.host); File "resource://jid1-f9uj2thwoam5gq-at-jetpack-collusion-lib/main.js", line 61, in getDomain return eTLDSvc.getBaseDomainFromHost(host); [Exception... "Component returned failure code: 0x804b0051 (NS_ERROR_HOST_IS_IP_ADDRESS) [nsIEffectiveTLDService.getBaseDomainFromHost]" nsresult: "0x804b0051 (NS_ERROR_HOST_IS_IP_ADDRESS)" location: "JS frame :: resource://jid1-f9uj2thwoam5gq-at-jetpack-api-utils-lib/securable-module.js -> resource://jid1-f9uj2thwoam5gq-at-jetpack-collusion-lib/main.js :: getDomain :: line 61" data: no]

Legend for the graph

The start of a table-legend for the sidebar

	Confirmed Tracker
	Unclassified Site
	Site You Have Visited
	Site You Have Not Visited
	Data Sent From Site A to Site B

from the css.index file:

legend {

font-size: 8pt;
font-family: "Helvetica Neue", Helvetica, Arial, sans-serif;

}

//trying to set the font so it matches the ohters

Heuristic for sorting sites by level of sketchiness

The "known trackers" list is a good start here, but we could also make use of additional information about the site -- e.g. what does privacychoice.org know about their privacy and data-sharing policies? How promiscuous are they with incoming information and outgoing information? How frequently does it slurp data in? etc.

The judgment of the site's sketchiness level could be used to further color-code the site, or if we implement picture-in-picture, the browser could show the site's sketchiness level when you visit it (similar to Ghostery).

Users should be able to export their collusion graph as JSON

In particular, Sid would like this so that he can make more awesome visualizations with it.

Mark nodes with site favicon

Right now, in order to see which entities are which in collusion's network map, one needs to mouse-over the relevant node.

It might be easier to get useful information when glancing at collusion's network map if the grey and red nodes were slightly larger and contained the relevant site's favicon.

Add unit testing suite

A number of folks would like to extend and build upon Collusion, but doing so would likely introduce regressions, so adding a suite of unit tests to guard against this would be rockin'.

Make it possible to block/unblock entities in the Collusion graph

It'd be nice if you could e.g. shift-click on an entity in the Collusion graph to prevent it from being colluded-with.

Perhaps one level may be to block cookies, but another level could be to immediately cancel any network requests to the domain.

The graph should still indicate when an attempt to contact a blocked domain occurs; perhaps coloring all the arcs red instead of gray could indicate this.

Display cookie contents if available in plaintext

Collusion should help the user find out not only what sites are colluding, but what data their collusion consists of. What are they saying about me?

If the cookies contain plaintext data like, say, usernames or email addresses, then give the user a way to see what that data is. (If the cookies just contain session IDs that have to be looked up in a server-side database somewhere to have any meaning, then we're out of luck.)

Search filter or checkbox filter to show/hide sites

To help the user deal with a graph once it gets large, it may be useful to provide tools to inspect a subset of the graph by category or string match.

E.g. as I type into the search filter box, all nodes fade out except the ones matching the term I'm typing (and their direct connections?) This would let me for instance easily focus on just the Google-related sites in my graph.

Related feature: check boxes to show/hide categories of sites ("sites I've visited") ("known trackers") etc. Nodes that don't match a checked box would fade out.

Collusion merges everything under .co.jp into a single co.jp domain

First mentioned here.

Wrong favicon sometimes appears on node

Can't figure out exactly how to reproduce it, but sometimes I've gotten collusion into a state where the node you're hovering over has the wrong favicon. (The right sidebar identifies the site and its links correctly, however).

Reloading consistently fixes the problem. That seems like a big clue.

Rotate graph in 3d space?

Hard, computationally intensive, would require camera controls, and not actually that useful from a data visualization perspective, but damn would it make a cool demo.

Physics simulation gets very slow with a lot of nodes

It works great for ~25 nodes, not so great with ~100 nodes.

Some ideas of things to try:

1. Profile and see if the slowdown is more in the physics simulation itself, or more in the rendering
2. See https://github.com/mbostock/d3/wiki/Force-Layout - there is a parameter "theta" that can be tweaked for a faster but less accurate physics simulation.
3. maybe don't try to redraw positions every tick, but instead only redraw once per X milliseconds (and CSS animate between these positions?)
4. maybe once the graph reaches a certain size, "pin" some of the nodes so that they no longer move, so we don't have to simulate forces on them or animate them; only the newer nodes joining the graph would be free to move relative to the pinned nodes.

Integrate browser fingerprinting data i.e. "how unique is my useragent"

Look at say http://panopticlick.eff.org which shows how unique your browser fingerprint is based on all the bits of entropy that it reports to any website that asks.

This could be integrated right into Collusion - it could tell the user "Your browser fingerprint is very likely to be unique". We don't know for sure, of course, so this would have to be a heuristic.

Maybe provide some options for the user to turn off some of the reporting features that expose those bits of entropy?

Add privacy policy

Sid Stamm suggests something like:

When you're using the add-on, we collect sites you visit solely to show you how they're connected. We don't keep them and don't give away the information to anyone except you.

Put it on a proper-looking Mozilla Labs web page

Cull out toolness.org from Collusion graph

Partly as a result of #6, collusion.toolness.org looks like it's colluding with all kinds of entities because it's requesting favicon.ico from them, which sometimes results in a cookie being transmitted. Until #6 is fixed, it might be nice to at least cull out toolness.org from the list of colluders.

Suggested by @ianb.

Play back your graph in order as a movie / export movie?

Would be a cool visualization tool, for sharing not only what your graph looks like now, but how it got there.

Enbiggen the visualization to the size of the window, or don't overlap text atop graph

For large monitors, the collusion graph's default size of 640x480 is tiny, and having the text overlap that tiny area reduces the visible area even more. It'd be nice if the graph dynamically sized to the size of the window, or at least if the text didn't overlap the graph area if there was enough room on the sides.

Display sites you *actually visited* differently

Should have a way to easily tell which sites on the graph were ones I went to on purpose, vs. the ones that know about me only because they colluded with the ones I visited.

Maybe pin the actually-visited sites to a certain area of the graph (e.g. put them at the top, let everything else hang down from them) or maybe outline them in a different color?

Sidebar

Have an option to keep the Collusion graph visible when you are browsing other pages: Put it in a chrome panel that hovers over the corner of your browser window, or show it in the background of the toolbar (animated persona?)

Make it possible to record and play back Collusion sessions

It'd be awesome if someone could record a session and tweet it for others to play back.

Glow for visited sites does not always appear immediately

Sometimes a visited site will not have the glow effect it's supposed to have. Reloading the page always fixes this problem, so I surmise that the node has the correct data about its visited status, but the class of the glow element is not getting set correctly when new nodes join the graph.

Running Collusion on Safari results in SyntaxError exception

As of commit 0abe93e, Safari raises SyntaxError: Expected token '(' on line 283 of graphrunner.js.

Plot tracker sites on world map if lat/long data avialable?

This is hard and it might not be all that informative, but it would make another very cool demo. See Barry's mockup of how this might look.

"Report your findings to Mozilla" button

It would take the JSON representing your graph and upload it to a Test Pilot - like service (possibly even using the exact same Test Pilot backend) so that we can aggregate data from many users and put together a report about who are the worst colluders on the web.

This would of course need its own privacy policy, would need to be suitably anonymized, etc.

Preserve the Collusion graph between restarts

It might be cool for either the add-on to persistently store the data, or for collusion.toolness.org to use HTML5 localStorage or webIndexedDB to keep the graph. The user should be able to clear the graph somehow.

Graph is currently too dense-looking

When the number of nodes gets high, the middle of the graph especially turns into a tangled mess of spaghetti. Lots of observers have complained about this and asked if we can spread it out more, allow nodes to be farther apart, etc.

We can try tweaking the force-directed-graph layout parameters and see if that helps. But the forces are already trying to push the nodes apart from each other; the density of the graph reflects the density of the connections in the underlying data. I suspect the nature of the phenomena Collusion is observing means that the graph is not disjoint and will not be easily separated. Not sure what the right thing to do is here.

Also show tracking via 3rd party includes (images, iframes)

Another method sites collude is by embedding 3rd party resources (small transparent gifs, iframes, Facebook like buttons) etc. inside pages. Would it be possible to have Firefox note the loading of resources from sources not on the current domain, and integrate this data with the Collusion graph? (Need to do some research on the frequency of "false positives", i.e. innocuous uses of 3rd party includes that are not really tracking.)

Data corruption with the lists

I think there's a bug in the list at the side of the visualization; sometimes, the number of entries in the list is not the same as the number of links to the node. And sometimes when you follow the links, the entities at the end are not the ones in the list...

If there was a reset button (a different issue!) it would be easier to debug!

Gerv

Disabling cookies results in no change to Collusion graph

If a user turns off support for cookies, deletes all existing cookies and restarts Firefox, the collusion graph doesn't change much (or at all).

This is because Collusion's code actually looks for either a Cookie: header in the HTTP requests Firefox sends out or a Set-Cookie: header in the HTTP response to determine whether a link should be created in the Collusion graph. Even when cookies are disabled, the Set-Cookie: header is still provided by remote servers, which is why the graph still indicates that you're being tracked.

We might be able to fix this by talking to Firefox's cookie manager instead of (or in addition to) just inspecting HTTP requests/responses. Or I suppose we could ignore Set-Cookie: headers in responses entirely, and only create graph links when Cookie: is in the request header.

Re-style UI to look like Barry's mockups

See mockups here:
http://people.mozilla.com/~jdicarlo/collusion/collusionV2black-1.jpeg
http://people.mozilla.com/~jdicarlo/collusion/collusionV2black-2.jpeg
http://people.mozilla.com/~jdicarlo/collusion/collusionV2black-3.jpeg
http://people.mozilla.com/~jdicarlo/collusion/collusionV2black-4.jpeg
http://people.mozilla.com/~jdicarlo/collusion/collusionV2black-5.jpeg

Barry says:
here is a stab at making things a bit more ominous with the black background and using the favicons. here the favicons are 20x20 as 16x16 seemed a little too small… if scaling them up becomes an issue we can keep them at 16x16

• red rings indicate sites tracked by privacychoice.com
• blue glow indicates sites navigated to directly
• red glow only appears on hover of privacy choice.com tracked sites
• gray ring sites that do not track on privacy choice.com
• hovering gives you the URL of the site related to the favicon as it isn't always obvious
• stuff on the left can change as we know more about the development and what is feasible with in the time we have to get things done.
• would like to see a way to get the nodes a bit more spaced out than how they are currently done.
• would be ok by me if the node box had a way to pan around in it if the tree gets too big
• christian heilmann thought there may be a way to make the node tree actually have 3rd dimension but then it becomes an issue of how nodes are placed in the space.
• glows can be a simple JPGs that get positioned. they could also only show on hover if that makes things better/easier.

Change name from "collusion"

Foundation team is going to come up with a different name that sounds less scary.

Include instructions on how to use the add-on

There aren't any actual instructions on how to use the add-on, so writing some would help.

List incoming links as well as outgoing ones

When you click a node, the list on the right shows the outgoing links. It should also list the incoming ones ("the following sites track your behaviour when you visit this site:")

As a bonus, make it possible for the size relationships to be inverted - instead of size being proportional to outgoing links (how many other sites does this one track), they can be proportional to incoming links (how many different companies does this company allow to track you?)

Gerv

There needs to be a "reset" button

Like it says :-) Either in the web page, or as part of the browser UI (and documented on the web page).

Gerv

Sequence visualization?

How about a type of visualization where nodes are arranged according to the order in which sites were visited (with non-visited sites hanging off the sides)?

Also read Flash cookies, if possible

Flash cookies are a major means of tracking people on the web; Collusion doesn't currently know about them. I'm not sure how much data we can actually get from the plugin about the individual cookies it's storing and what sites it allows to have them. Will need to do some research on that.

Directly link to Firefox and IE versions of TrackerBlock

These: http://t.co/MhGoeXA http://t.co/mXtXyUK