SWIG Java uint64_t binding on Windows about blst

[dot-asm]

Unfortunately our SWIG experience is rather limited and is kind of learn-as-we-go. As well as learn enough to evaluate if it actually meets an objective. As an example, SWIG was briefly considered as option for Go support, but it didn't quite meet the expectations, and we went straight to raw cgo. Either way, let me tell a little behind-the-scenes story. My initial suggestion for limb_t in blst.h was ... double. With rationale that there wouldn't be any ambiguity related to platform, compiler or even host environment. It might appear objectionable, but application is not actually expected to examine the values anyway, only allocate a properly sized and aligned buffer (or have wrapper allocate one) and pass the pointer around between methods. And of course [adhere to] deserialization to instantiate the objects with specific values, and serialization to extract usable/presentable data. Internal representation is of no application concern whatsoever. So with this in mind, would typedef double limb_t work for you? If so, it might be appropriate to slip it in between #ifdef SWIGJAVA/#else...

On more general note. If one lets wrapper manage memory (as opposite to having the caller language do it like in Go and Rust), then one can wonder why would application code have to be concerned with limbs and arrays at all. What I'm trying to say is that if we're OK with memory being managed in SWIG wrapper, then we might as well look for a way to make blst_* types opaque and expose only constructors (and destructors of course), and avoid a whole class of subtle problems... What I'm trying to say is that don't view current blst.swg as the only way to interoperate with library, let's think even outside the box...

[Nashatyrev]

Hey @dot-asm!
Thanks for the detailed response! That all totally makes sense in terms of making blst_* types opaque and I probably missed that concept cause of lacking serialize/deserialize functions. I mean functions like blst_*_from_?endian and blst_?endian_from_*.
BTW why they all were excluded from SWIG generation?
Should I probably need making another SWIG effort with those functions included?

I'm now experimenting with JNA atop of shared library variant and it seems to work for me with some extra boilerplate. However it would be better of course to stick with some universal solution (SWIG in your case) to avoid doing double work on library support.

[dot-asm]

There will be C++ bindings, and it would be arguably more natural choice for SWIG-ing, especially to object-oriented languages. This implies that types would be opaque, at least that's what we'll aim for.

As for blst_*_from_*, or even more specifically field operations, not being available through SWIG. It's argued that scripting languages, and by extension majority of applications have no need for them. It's serialized EC points that are passed around over network, and there is no real need for interface lower than that. So that when I mentioned deserialization I referred rather to point objects being instantiated from wire data, and not field elements. Do you actually have need for field operations? If so, then I for one would like to hear about it.

Though there is one problem as it is now. Of course, blst_scalar [de-]serialization would have to be available through SWIG. So that #ifdef SWIG needs to be moved few lines down, but not past "Fr operations"... Done.

As for giving it another shot now or later. Well, it's up to you. As already said, there will be changes, so it might make more sense to wait. But do also tell how does JNA work for you...

[dot-asm]

It's argued that scripting languages, and by extension majority of applications have no need for [field operations].

Just in case, additional dimension to this is that it makes no sense from performance viewpoint, because SWIG call overhead is larger than time it takes to perform the operation itself. Of course one can naturally argue [that] it would still be huge win in comparison to corresponding implementation in scripting language or Java... Well, yes... Then I suppose if one puts together a compelling argument in favour...

[Nashatyrev]

Oh, it seems I just didn't find any serialize/deserialize functions for scalar type, gave up and missed ones for EC points :)
So I likely need to give another try for SWIG. Thanks!

[Nashatyrev]

@dot-asm thanks a lot for your help! SWIG binding works for me now 👍
I have some Java specific changes though and I will make a PR(s) closer to the end of our integration

[Nashatyrev]

I think this could be closed.