fastrecord about clj-fast

[xificurC]

@joinr if the record improvements are backward compatible with current defrecord wouldn't it make sense to get them into clojure proper?

[joinr]

It would, but that's a tall order IME :) Much easier to sort things out in a library IMO. That would require patching and going through JIRA, which is now a bit more closed off. There's typically a fairly high burden of proof standard as well; for the moment this is a pretty easy way to explore possible performance opts without formal patching and review and waiting etc. The other problem is that the implementation enables some changes to existing semantics; allowing pure static fields is new, the operations you'd expect on a value that passes record? would fail with a fastrecord with static (e.g. struct-like) fields (no more dissocing freely and having the record type possibly change to a map). It also provides an IFn implementation to be consistent with maps, which records do not (users can define their own IFn, and often use records for this specifically).

There's also the general disinterest in the nanos and microbenchmarks and a general dubiousness about these kinds of differences IME. Some of that is valid, where microscopic performance does not hold at large (or as in the case with zach tellman's implementation of faster small vectors called tuples, they led to slow downs in other unforeseen use cases due to the jvm being confused by the proliferation of types and de-optimizing due to megamorphic call sites).

Maybe if the thing is demonstrably better at no risk, it'd make sense to go through the motions of submitting a patch (probably a bit different than the rewriting I did here though. I think the ideas are there though (like improving typical lookups) to provide a simpler focused optimization of the existing defrecord.

[bsless]

Another case I came across here is fast hash calculations for records
If you close the extmap you can unroll the hash calculations

[joinr]

That makes sense. Would be particularly useful if they are use as keys in collections. That was actually an initial optimization I ran up against; the ICFPC code was using a pre-computed lookup table of record coordinate pairs like {:x :y y} to index into precomputed neighborhoods (a memoized mapping to a vector). The hashing was way slow. That led to other observations...

I think that it would be useful to re-implement the clojure.core stuff directly and extend this (get more control e.g. over hashing and the like); my implementation is an intentional piggy-back hack on top of that infrastructure.

[bsless]

Seems reasonable.
Easiest would just be to just use Java records but that's some way off.
Implementing the hashing correctly seems to be the hardest part imo. Just hash it like a map or some other way?
Everything else can probably be parametrized

[joinr]

Map hashing, yeah. I ran into some corner cases implementing that stuff correctly. You can probably just rip out the hash computation from existing defrecord and elide the bit where it includes the unordered-hash of the extmap. I think it would be unrolling hashUnorderedColl, not too bad. I believe I did that for some stuff in fastmath patches and other things.

[bsless]

I think the more unpleasant part of it would be implementing the hashing of MapEntry which extends APersistentVector
It needs to be done only once, but it would be slightly unpleasant
https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/APersistentVector.java#L156

[joinr]

Example from fast-math

[joinr]

You can cover up the nasty bits like he did by using a utility fn (or inline all the things if you want).

[joinr]

Probably just re-use clojure's generic hash function (this example is optimized for doubles specifically).