GithubHelp home page GithubHelp logo

d3-collection's Introduction

d3-collection

Deprecation notice: Use JavaScript’s built-in Map, Set and Object classes instead of d3-collection’s corresponding methods. Use d3-array’s group and rollup instead of d3-collection’s nest.


Handy data structures for elements keyed by string.

Installing

If you use NPM, npm install d3-collection. Otherwise, download the latest release. You can also load directly from d3js.org, either as a standalone library or as part of D3 4.0. AMD, CommonJS, and vanilla environments are supported. In vanilla, a d3 global is exported:

<script src="https://d3js.org/d3-collection.v1.min.js"></script>
<script>

var map = d3.map()
    .set("foo", 1)
    .set("bar", 2);

</script>

API Reference

Objects

A common data type in JavaScript is the associative array, or more simply the object, which has a set of named properties. The standard mechanism for iterating over the keys (or property names) in an associative array is the for…in loop. However, note that the iteration order is undefined. D3 provides several methods for converting associative arrays to standard arrays with numeric indexes.

A word of caution: it is tempting to use plain objects as maps, but this causes unexpected behavior when built-in property names are used as keys, such as object["__proto__"] = 42 and "hasOwnProperty" in object. If you cannot guarantee that map keys and set values will be safe, use maps and sets (or their ES6 equivalents) instead of plain objects.

# d3.keys(object) <>

Returns an array containing the property names of the specified object (an associative array). The order of the returned array is undefined.

# d3.values(object) <>

Returns an array containing the property values of the specified object (an associative array). The order of the returned array is undefined.

# d3.entries(object) <>

Returns an array containing the property keys and values of the specified object (an associative array). Each entry is an object with a key and value attribute, such as {key: "foo", value: 42}. The order of the returned array is undefined.

d3.entries({foo: 42, bar: true}); // [{key: "foo", value: 42}, {key: "bar", value: true}]

Maps

Like ES6 Maps, but with a few differences:

# d3.map([object[, key]]) <>

Constructs a new map. If object is specified, copies all enumerable properties from the specified object into this map. The specified object may also be an array or another map. An optional key function may be specified to compute the key for each value in the array. For example:

var map = d3.map([{name: "foo"}, {name: "bar"}], function(d) { return d.name; });
map.get("foo"); // {"name": "foo"}
map.get("bar"); // {"name": "bar"}
map.get("baz"); // undefined

See also nests.

# map.has(key) <>

Returns true if and only if this map has an entry for the specified key string. Note: the value may be null or undefined.

# map.get(key) <>

Returns the value for the specified key string. If the map does not have an entry for the specified key, returns undefined.

# map.set(key, value) <>

Sets the value for the specified key string. If the map previously had an entry for the same key string, the old entry is replaced with the new value. Returns the map, allowing chaining. For example:

var map = d3.map()
    .set("foo", 1)
    .set("bar", 2)
    .set("baz", 3);

map.get("foo"); // 1

# map.remove(key) <>

If the map has an entry for the specified key string, removes the entry and returns true. Otherwise, this method does nothing and returns false.

# map.clear() <>

Removes all entries from this map.

# map.keys() <>

Returns an array of string keys for every entry in this map. The order of the returned keys is arbitrary.

# map.values() <>

Returns an array of values for every entry in this map. The order of the returned values is arbitrary.

# map.entries() <>

Returns an array of key-value objects for each entry in this map. The order of the returned entries is arbitrary. Each entry’s key is a string, but the value has arbitrary type.

# map.each(function) <>

Calls the specified function for each entry in this map, passing the entry’s value and key as arguments, followed by the map itself. Returns undefined. The iteration order is arbitrary.

# map.empty() <>

Returns true if and only if this map has zero entries.

# map.size() <>

Returns the number of entries in this map.

Sets

Like ES6 Sets, but with a few differences:

# d3.set([array[, accessor]]) <>

Constructs a new set. If array is specified, adds the given array of string values to the returned set. The specified array may also be another set. An optional accessor function may be specified, which is equivalent to calling array.map(accessor) before constructing the set.

# set.has(value) <>

Returns true if and only if this set has an entry for the specified value string.

# set.add(value) <>

Adds the specified value string to this set. Returns the set, allowing chaining. For example:

var set = d3.set()
    .add("foo")
    .add("bar")
    .add("baz");

set.has("foo"); // true

# set.remove(value) <>

If the set contains the specified value string, removes it and returns true. Otherwise, this method does nothing and returns false.

# set.clear() <>

Removes all values from this set.

# set.values() <>

Returns an array of the string values in this set. The order of the returned values is arbitrary. Can be used as a convenient way of computing the unique values for a set of strings. For example:

d3.set(["foo", "bar", "foo", "baz"]).values(); // "foo", "bar", "baz"

# set.each(function) <>

Calls the specified function for each value in this set, passing the value as the first two arguments (for symmetry with map.each), followed by the set itself. Returns undefined. The iteration order is arbitrary.

# set.empty() <>

Returns true if and only if this set has zero values.

# set.size() <>

Returns the number of values in this set.

Nests

Nesting allows elements in an array to be grouped into a hierarchical tree structure; think of it like the GROUP BY operator in SQL, except you can have multiple levels of grouping, and the resulting output is a tree rather than a flat table. The levels in the tree are specified by key functions. The leaf nodes of the tree can be sorted by value, while the internal nodes can be sorted by key. An optional rollup function will collapse the elements in each leaf node using a summary function. The nest operator (the object returned by nest) is reusable, and does not retain any references to the data that is nested.

For example, consider the following tabular data structure of Barley yields, from various sites in Minnesota during 1931-2:

var yields = [
  {yield: 27.00, variety: "Manchuria", year: 1931, site: "University Farm"},
  {yield: 48.87, variety: "Manchuria", year: 1931, site: "Waseca"},
  {yield: 27.43, variety: "Manchuria", year: 1931, site: "Morris"},
  ...
];

To facilitate visualization, it may be useful to nest the elements first by year, and then by variety, as follows:

var entries = d3.nest()
    .key(function(d) { return d.year; })
    .key(function(d) { return d.variety; })
    .entries(yields);

This returns a nested array. Each element of the outer array is a key-values pair, listing the values for each distinct key:

[{key: "1931", values: [
   {key: "Manchuria", values: [
     {yield: 27.00, variety: "Manchuria", year: 1931, site: "University Farm"},
     {yield: 48.87, variety: "Manchuria", year: 1931, site: "Waseca"},
     {yield: 27.43, variety: "Manchuria", year: 1931, site: "Morris"}, ...]},
   {key: "Glabron", values: [
     {yield: 43.07, variety: "Glabron", year: 1931, site: "University Farm"},
     {yield: 55.20, variety: "Glabron", year: 1931, site: "Waseca"}, ...]}, ...]},
 {key: "1932", values: ...}]

The nested form allows easy iteration and generation of hierarchical structures in SVG or HTML.

For a longer introduction to nesting, see:

# d3.nest() <>

Creates a new nest operator. The set of keys is initially empty.

# nest.key(key) <>

Registers a new key function. The key function will be invoked for each element in the input array and must return a string identifier to assign the element to its group. Most often, the function is a simple accessor, such as the year and variety accessors above. (Keys functions are not passed the input array index.) Each time a key is registered, it is pushed onto the end of the internal array of keys, and the nest operator applies an additional level of nesting.

# nest.sortKeys(comparator) <>

Sorts key values for the current key using the specified comparator function, such as d3.ascending or d3.descending. If no comparator is specified for the current key, the order in which keys will be returned is undefined. For example, to sort years in ascending order and varieties in descending order:

var entries = d3.nest()
    .key(function(d) { return d.year; }).sortKeys(d3.ascending)
    .key(function(d) { return d.variety; }).sortKeys(d3.descending)
    .entries(yields);

Note that this only affects the result of nest.entries; the order of keys returned by nest.map and nest.object is always undefined, regardless of comparator.

# nest.sortValues(comparator) <>

Sorts leaf elements using the specified comparator function, such as d3.ascending or d3.descending. This is roughly equivalent to sorting the input array before applying the nest operator; however it is typically more efficient as the size of each group is smaller. If no value comparator is specified, elements will be returned in the order they appeared in the input array. This applies to nest.map, nest.entries and nest.object.

# nest.rollup(function) <>

Specifies a rollup function to be applied on each group of leaf elements. The return value of the rollup function will replace the array of leaf values in either the associative array returned by nest.map or nest.object; for nest.entries, it replaces the leaf entry.values with entry.value. If a leaf comparator is specified, the leaf elements are sorted prior to invoking the rollup function.

# nest.map(array) <>

Applies the nest operator to the specified array, returning a nested map. Each entry in the returned map corresponds to a distinct key value returned by the first key function. The entry value depends on the number of registered key functions: if there is an additional key, the value is another map; otherwise, the value is the array of elements filtered from the input array that have the given key value. If no keys are defined, returns the input array.

# nest.object(array) <>

Applies the nest operator to the specified array, returning a nested object. Each entry in the returned associative array corresponds to a distinct key value returned by the first key function. The entry value depends on the number of registered key functions: if there is an additional key, the value is another associative array; otherwise, the value is the array of elements filtered from the input array that have the given key value.

Note: this method is unsafe if any of the keys conflict with built-in JavaScript properties, such as __proto__. If you cannot guarantee that the keys will be safe, you should use nest.map instead.

# nest.entries(array) <>

Applies the nest operator to the specified array, returning an array of key-values entries. Conceptually, this is similar to applying map.entries to the associative array returned by nest.map, but it applies to every level of the hierarchy rather than just the first (outermost) level. Each entry in the returned array corresponds to a distinct key value returned by the first key function. The entry value depends on the number of registered key functions: if there is an additional key, the value is another nested array of entries; otherwise, the value is the array of elements filtered from the input array that have the given key value.

d3-collection's People

Contributors

mbostock avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

d3-collection's Issues

es6 import syntax example?

I think that there's should be an example on the Readme for how to import the library using the es6 import syntax. E.g., neither of these syntaxes works:

import d3 from "d3-collection";  // Doesn't work
import { d3 }  from "d3-collection";  //  Doesn't work either

There's no error given but the d3 variable is always undefined.

After some hunting around, I finally found a syntax that does work :

import * as d3 from "d3-collection"; // does work

but this was far from obvious, for me at least!

d3.nest retains node properties after rollup

Its difficult to understand why rollup should strip leaf node properties, when there are all kinds of reasons those properties might be desired for viz/labelling. etc once the nest is fed to d3.hierarchy. I understand that I can have rollup deliver back an object rather than a single value but 1) that is laborous and idiosyncratic to the data set and 2) d3.hierarchy then finds 0/undefined in the "value" property for non leaf nodes.
Maybe stratify handles this better, but in my environment its not feasible to ask my non-viz counterparts to keep a bunch of dead/non-functional rows in their data so stratify can find them.

Can there be a way to for rollup to retain leaf properties in the nest?

d3.nest reformats a Date object to a string

Using d3.nest to aggregate a dataset by date. My code:

vis.nestData = d3.nest()
	.key(function(d){ return d.survey; })
	.rollup(function(leaves){ return leaves.length; })
	.entries(vis.data);

vis.nestData.forEach(function(d){
	d.key = new Date(d.key);
})

vis.data has field survey as JS Date objects. Upon return, the keys become strings, NOT Date objects. The second call is needed to reformat them into dates. Somewhere in the nest call they are converted to strings.

Support string as accessor?

Since plucking off a single property is likely a very common use case, it would be nice to support simply passing a string so that d3.set(stuff, 'name') would be equivalent to d3.set(stuff, x => x.name).

I'd be happy to submit a PR if this is a feature you'd accept.

ES6 Collections.

What would d3-collection look like if we dropped the d3.map and d3.set implementations? You might still want a d3.map with a key accessor for constructing a Map from an array of data (the key would default to index if not specified, and perhaps #10 you’d also allow an optional value accessor), and a d3.set with an optional accessor.

sortValues after rollup

I tried to call sortValues after rollup and it didn't work. Looking at the code it seems to me that is not possible in the way it was implemented, i think that it would be good, so we could group some objects and then summarize some values with rollup by group and order the groups by this value. I know it could be done with Array.sort, but anyway, i think that would be nice to do it with sortValues.

d3.group convenience function?

Sometimes it’s convenient to group an array by a single key. You can do that using d3.nest like so:

var entries = d3.nest()
    .key(d => d.foo)
    .entries(data);

Is it worth making a convenience function for this?

d3.group = function(array, key) {
  return d3.nest().key(key).entries(array);
};

I guess it would violate the parsimony principle.

Mutable nests?

Copied from d3/d3-array#31:

What if d3.nest returned an empty nest object, on which you can define some keys (which would nest existing values, if any), and then you added objects to the nest object and they were automatically slotted into the correct position? Perhaps you could remove objects, too. It’s not clear how nest.rollup would work in this context, though.

Add nest.count or nest.visit?

Copied from d3/d3-array#10:

Related d3/d3#1091, it might be nice if there were an easy way to count the number of elements matched at each level of the hierarchy. Yes, things like the cluster layout do that for you already, but it’d be nice to do that with a simple nest operator, too.

This feels slightly related to the nest.rollup method, too. Like, instead of replacing the set of nested values with the return value of the rollup function, I just want to decorate the object (say by assigning a count value). But another big difference is that rollup only operates on arrays of siblings, and nest.count should be a recursive operation on the entire tree.

So… it’s almost like you want tree visit methods on the returned nest object. Which makes me wonder if there should be a nest.tree method instead of nest.map, and then have some useful methods on the returned tree instance.

Feature Request - same level nesting by multiple key

See this interactive notebook to display problem

Suppose we have dataset like this

var arr = [
 {name: "jim",   amount: "34.0",   date: "11/12/2015"},
 {name: "carl",  amount: "120.11", date: "11/12/2015"},
 {name: "stacy", amount: "12.01",  date: "01/04/2016"},
 {name: "stacy", amount: "34.05",  date: "01/04/2016"}
]

Would be good if we have possibility to nest by multiple key at one level

So, if we would try

expensesByNameAndDate = d3.nest()
  .keys(["name","date"])  // keys , not  key
  .entries(expenses)

We would get

[
{
  "key": { "name": "jim","date": "11/12/2015"},
  "values": [{"name": "jim","amount": "34.0","date": "11/12/2015"}]
},
{ "key": {"name": "carl","date": "11/12/2015"},
   "values": [{"name": "carl","amount": "120.11","date": "11/12/2015"}]
},
{  "key": {"name": "stacy","date": "01/04/2016"},
  "values": [{"name": "stacy","amount": "12.01","date": "01/04/2016"},{"name": "stacy","amount": "34.05","date": "01/04/2016"}]
  }
]

Possible overloads for keys func

  1. .keys(["name","date"])
  2. .keys([d=>d.name,d=>d.date])

Custom key, values field names for the nest()

nest hardcodes key and values. If not too difficult, please support custom field names, as this would make this Vega's issue much easier to solve. Thanks!

BTW, I think this has benefit to other users because it simplifies result examination in case when original data also contains key or values fields, and the caller needs to find which objects were generated.

Nest: individual key sorting

Sometimes it's necessary to sort keys differently. Currently, only one sort is applied to all keys. I'm thinking the key function could accept an optional sort function:

d3.nest()
  .key(d => d.region, d3.ascending)
  .key(d => d.country, d3.descending)
  .entries(data)

Would be very useful for something like a pivot table.

Why does d3.nest.map prepend a prefix?

Hi there 👋

We're long time users of d3 and recently looked at upgrading from d3 v3.5.8.

C3 have bumped their dependency on this project so we're wanting to make use of the latest d3 if we're going to update.

One thing I noticed right away is that since quite a while ago you introduced a prefix to keys when using d3.map;

https://github.com/d3/d3-collection/blame/5aad1d67f264c0b3b6a9f3e9833021497f03beeb/src/map.js#L1

Some very common code from us might look like the following;

let nestedData = d3.nest()
  .key((b) => b["Name"])
  .rollup((b) => {
    // return some aggregation on buckets of data by name
  })
  .map(data);

let names = Object.keys(nestedData);

So now in getting all of the unique "Name" elements we now get, for example ["$David", "$Jason"] if that makes sense, instead of ["David", "Jason"].

Now I'm not saying there's anything wrong with this, I imagine this update was part of a breaking change version bump, I'm just curious as to the reasoning?

Also I noticed that you export the prefix, did you have a snippet of code as an example of setting the prefix to nothing, if that's possible?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.