Comments (5)
Thanks for finding and checking out this bug. Can you point out a torture-test that reliably reproduces it (sooner or later)? For example, have you observed BenchmarkPointEncode alone to reproduce the problem if run long enough? I tried:
go test -v -bench PointEncode -benchtime 60s
...but I wasn't immediately able to reproduce it at least in that way.
One thing to try immediately when any likely allocation-related bug crops up using the OpenSSL library is try NULLing out the BN_CTX parameters in the relevant OpenSSL library calls, to see if we might be misusing or corrupting a BN_CTX object in some way. Not using a BN_CTX will be slower but might be slightly more robust.
Ah, wait - are you using a single cipher.Suite object across multiple threads, i.e., shared by multiple virtual "nodes" in your intra-process test framework? That could definitely cause the problem, because BN_CTX is not safe for concurrent use by multiple threads (and isn't supposed to be), and as a result an openssl Suite object probably isn't either. Can you try just creating a Suite object per virtual node (i.e., per thread) and see if that makes the problem go away?
If it does, this brings up the obvious question of whether Suite objects "should" be thread-safe. I can see arguments both ways.
from kyber.
Dylan, have you had a chance to verify definitively whether or not this is a thread-safety issue, as I suggested above? It should be a 10-minute test at most, and would be nice to know for certain even if we decide to punt on the "right" solution for now (as I'm inclined to). Thanks.
from kyber.
I'm of the understanding that OpenSSL isn't thread-safe by default, but
maybe in some situations it is. It may be necessary to implement the
following hooks:
https://www.openssl.org/docs/crypto/CRYPTO_lock.html
On Wed, Jan 28, 2015 at 1:52 PM, Bryan Ford [email protected]
wrote:
Dylan, have you had a chance to verify definitively whether or not this is
a thread-safety issue, as I suggested above? It should be a 10-minute test
at most, and would be nice to know for certain even if we decide to punt on
the "right" solution for now (as I'm inclined to). Thanks.—
Reply to this email directly or view it on GitHub
#18 (comment).
from kyber.
Point and Secret arithmetic operations aren't supposed to be thread-safe, in that you should be using a given Point or Secret object in one thread at a time anyway, as with most purely computational code. Adding thread-safety to Point/Secret arithmetic would be useless to just about everyone and would just slow things down.
If this is the particular thread-safety issue I suspect it is, however, then the problem is very specific to the fact that OpenSSL's bignum library uses this allocation-pooling optimization that tries to reuse bignum objects to reduce allocations. Reasonable, except the bignum CTX object that's intended to be that cache of bignum objects is probably not designed to be thread-safe, intentionally for performance reasons: you're supposed to be using one per thread. So there are three obvious solutions:
- Just don't use CTX objects at all; set all those parameters to null, which is allowed, but will probably slow things down by preventing the caching optimizations.
- Have one big CTX object and make access to that thread-safe, e.g., by taking and releasing a master lock around any OpenSSL-based arithmetic operation that needs to use that ctx object. This would work but would obviously be bad for parallelism.
- Implement a thread-safe pool of CTX objects, where any thread that comes into the OpenSSL code grabs one temporarily (via thread-safe primitives), does the operation it needs to do, then dumps the CTX object back on the pool (again using thread-safe primitives). This would at least permit some parallelism, but contention on the CTX object lock might eventually be a problem - if we end up using the OpenSSL-based implementation of this stuff for anything important, which we might not.
- The "proper" approach would be to use per-thread state: i.e., have a per-thread CTX object that gets used automatically whenever we invoke an OpenSSL-based bignum/EC routine. But Go still doesn't have native support for per-thread state as far as I can tell, and I don't know if the pthread-based threadpsecific state functions will work in the context of a Go program (might be worth trying).
But in any case, it's not worth investing much time in this, at least not now, because we're not sure whether we really want to use the OpenSSL-based cipher suite for anything but testing/benchmarking against. So perhaps it's OK (for now) if it's just not thread-safe at all.
from kyber.
wontfix
from kyber.
Related Issues (20)
- How to transmit shares over the network? HOT 4
- Use another suite for sign and verify in DKG
- How to run tbls over network? HOT 4
- Testing HOT 1
- Infinite for loop while embedding data in an Ed25519 point HOT 1
- Old dkg nodes doesn't certify after re-sharing HOT 1
- found a bug in dkg_test HOT 1
- Bug: share/dkg/rabin: All N participants must be available HOT 5
- Bug: share/vss/rabin MinimumT(6)=3 instead of 4 HOT 12
- Build fails on MacOS HOT 4
- github-workflow: go mod tidy fails HOT 1
- Inconsistent edwards25519's Scalar (un)marshaling for some inputs. HOT 4
- DKG Protocol - Public Key Change After Reshare HOT 9
- Use fixed-length integers to improve cross-platform compatibility HOT 3
- Building on 32-bit systems fail HOT 10
- v3/sign/bls shows up as depreciated when i try to import it HOT 10
- Use specific go versions in workflows and go.mo HOT 1
- The Scalar and Point interface do not allow error handling HOT 1
- Refactor some package names
- Add a Montgomery representation of Curve25519, called "Curve25519"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kyber.