Merging the brushing-filtering function with simple viewer layouts using loaded data file about dc.graph.js

[gordonwoodhull]

Hi @happylilem, thanks for your interest in dc.graph.js!

I am taking a quick look at the source, hope to get to this soon.

One limitation is that the brushing-filtering code expects certain attributes in the graph, specifically node.color

dc.graph.js/web/js/brushing-filtering.js

Lines 76 to 78 in 1466cef

	colorDimension = data.nodef.crossfilter.dimension(function(n) {
	return n.color;
	}),

and edge.dash

dc.graph.js/web/js/brushing-filtering.js

Lines 80 to 82 in 1466cef

	dashDimension = data.edgef.crossfilter.dimension(function(e) {
	return e.dash;
	}),

Maybe some more too. So it's likely that you will still have to modify the demo for your purposes, adding and removing charts too.

When I started this project, I imagined it as a programmers toolkit for building network visualizations, much as dc.js is for other charts. Of course, most people don't want to learn JavaScript in order to see their graphs, so this may have been a step in the wrong direction.

Later on, I added "self serve" features where you can import your own data into the browser, and these proved very popular.

Not sure where the sweet spot is between having all the interactions and all the layouts and making it easy to use.

[happylilem]

Thanks Gordon for the reply! Yes I also noticed the color part of the brushing-filtering code, if we were to import our own data, the color part would require more setup regarding the actual data instead of the fixed ones in the demo. Since my biggest problem currently is still how to get my own data file to work in the brushing-filtering demo, maybe color would be of less priority in my case. I would really appreciate it if you could make the "import data" function available for the brushing-filtering demo, and then I'll see how I can get the two demos merged and consider the color part later on :)

[gordonwoodhull]

Okay, I have added the feature to the demo.

I ran into a couple of bugs which I don't have time to track down atm:

1. sometimes layout will fail with the following error:

Uncaught TypeError: Cannot read property 'dcg_shape' of undefined
  at project_port (place_ports.js:29)
  at diagram.js:2103
  ...

2. sometimes the auto-zoom is incorrect and parts of the graph are not displayed.

These don't seem to have anything to do with the feature in question, just, well, it's a complicated system and it evolved somewhat chaotically.

Also, it seems that graphs which don't have the attributes mentioned above will cause nothing to be displayed.

I won't have time to work on this for a few days, so I thought it would be better to publish the updates and hopefully return to this soon. Hope it helps!

[happylilem]

Thank you so much for adding that feature! I just checked it out and tried with my data file, I think it is just like what you mentioned, nothing is displayed because my data file lack the features required in this demo. But having this feature does help a lot with my merging, I'll look more closely at the code and see how I could further modify it. I'll let you know if I got other questions or, hopefully, if it worked!

[gordonwoodhull]

Cool, thanks for confirming!

It should be pretty easy to make it resilient to data that doesn’t fit (and just let the other charts fail). Will give that a try soon.

The other bugs are more difficult.

[happylilem]

Hi Gordon, I'm having trouble finding the function where I can set an attribute in my data file to be a distinguisher which the color groups can depend on. For example I want the colors groups to be sorted base on "gender" group in my data, so all males get the same color, all females get another color, and non-binaries get another color. Is there a function in this repository that can do this?

And I would also like to know where function(n) is defined.

dc.graph.js/web/js/brushing-filtering.js

Lines 69 to 78 in 1466cef

	var populate = function(n) {
	var random = dc_graph.random_graph({
	ncolors: 3
	});
	random.generate(n);
	var data = build_data(random.nodes(), random.edges()),
	colorDimension = data.nodef.crossfilter.dimension(function(n) {
	return n.color;
	}),

I was navigated to jquery-ui.js and d3-tip.css for the definition, but they don't seem quite right to me. May I know what exactly does function(n) do? Thank you!

[gordonwoodhull]

The node and edge colors, shapes, dashes, etc., are parameterized on the diagram by data in a pattern similar to how dc.js charts are.

In particular, the node color is initialized here:

dc.graph.js/web/js/brushing-filtering.js

Lines 169 to 171 in 3c76b26

	.nodeFill(function(kv) {
	return kv.value.color;
	})

[gordonwoodhull]

Also note nodeFillScale a few lines down… nodeFill provides a value , its scale maps the value to a color.

[happylilem]

Hi Gordon, sorry for not being able to get back to you since I've been very busy these days. I finally got time to take a closer look at nodeFill and nodeFillScale and got them like halfly understood.
I'm wondering what function kv does here,

dc.graph.js/web/js/brushing-filtering.js

Lines 169 to 171 in 3c76b26

	.nodeFill(function(kv) {
	return kv.value.color;
	})

and I looked up where kv is defined, but I couldn't quite interpret line 7270:

dc.graph.js/web/js/yoga-layout.js

Lines 7269 to 7270 in 3c76b26

	}function kv(a, b, c) {
	a = a | 0;b = b | 0;c = c | 0;a = ai(b) | 0;b = lv(c) | 0;c = mv(c, 0) | 0;Zv(a, b, c, nv() | 0, 0);return;

Can you please kindly tell me how kv works, what a,b,c stand for and how they interact in this function? Does this function has anything to do with the "colors" variable and "domain" in nodeFillScale?

dc.graph.js/web/js/brushing-filtering.js

Line 141 in 3c76b26

var colors = ['#1b9e77', '#d95f02', '#7570b3'];

dc.graph.js/web/js/brushing-filtering.js

Line 175 in 3c76b26

.nodeFillScale(d3.scale.ordinal().domain([0,1,2]).range(colors))

Because when I tried to switch node colors shown on the graph by changing the "domain", I discovered that it doesn't work as I thought. I thought domain[0], domain[1], and domain[2] should each correspond to a hex code in the colors variable, but in fact, either of those corresponds to the first color hex code, and if I want to switch to the second color, I had to do domain[0,1]. And domain[0,1,2] gives the color represented by the third hex code. Are these defined by the kv function?

[happylilem]

BTW, right now I have a thought that although we have the API which explains many of the major functions used in the js files of the graph demos, there are many other underlying functions (such as function kv) that lack a documentation. I'm thinking it might be a good idea if I start documenting those functions, e.g. the components that made up the functions, how the formula in the function works etc. as I try to understand them, so that it would be clearer to people who wants to modify the codes for their own use. What do you think about this idea on documenting those?

[gordonwoodhull]

kv here is just an ordinary function parameter, not a function.

I use the name kv to indicate a parameter which is a key/value pair in the form of an object. This is a convention that comes from crossfilter groups.

So nodeFill is a method which gets/sets a function which takes a {key,value} pair, and calculates some "domain value".

And nodeFillScale is a method which gets/sets a D3 scale, which is a function that takes a "domain value" and returns a "range value", in this case a color.

Functions which take functions as parameters can be confusing, but they are very powerful. In this case we are telling the library not just a value, but how to retrieve or calculate a domain value from the data supplied to the library. This make the library oblivious to the underlying data format.

d3.scale.ordinal() maps from ordinal "domain values" to "range values". So [0,1,2] are the possible inputs, and colors are the corresponding outputs.

[gordonwoodhull]

I agree the documentation is sparse. Any help would be appreciated.

[happylilem]

Hi Gordon, I guess now I’m kinda able to group the nodes array-wise, but not completely have them grouped when I try to reflect the groups by different colors on the graph.
I asked previously about how the colors were assigned to the nodes, but colorDimension just gives a range of colors that each group refers to, and the nodes should be grouped first before getting different colors. So I guess my fundamental problem should be: how exactly does the brushing-filtering demo decide the groups?
I put breakpoints at almost every line and could see that the randomly generated data arrays have key and color elements, and the color element has values such as 0,1,2, which corresponds to the color domains.
I thought the color element does the grouping, but turned out that it's not when I tried to add this element to the data array in my own data file. I couldn’t find any other line which set the nodes into groups and how these groups and the colors are connected. Does the grouping come with the generated data? Because when I tried with my own data file, hoping to see the nodes show different colors according to their groups that I set (but maybe groups aren’t formed at all, I don't know how the system recognizes the groups that I try to make), but all the nodes just still show the same color.
Can you possibly let me know which line(s) group the nodes? Thank you!

[happylilem]

Hi Gordon, never mind I just got the whole merging figured out! And I created a data structure format that can apply to the brushing and filtering demo, so as long as the data file imported is in that format, the demo will be able to do its job without throwing any errors :) If you're interested in having a look at that and creating some documentation to explain how the demo works, please let me know, I'd be happy to discuss with you!