Atomic incrementers unnecessary about gosnmp

[soniah]

See my comment in #71

[phemmer]

Further use of the plugin has come up with the need to actually make the plugin parallel safe. Basically to speed up a large number of SNMP operations, I need to issue them in parallel, instead of waiting for one to finish before starting another. Currently the only way to do this is to create multiple GoSNMP objects. And in my use case, this could potentially be hundreds of them.

Instead I propose that the sendOneRequest be redesigned such that it utilizes a single receiver goroutine, handling messages for any number of requesters.
For example, when a request is made, the plugin registers the request ID with a receiver goroutine. The request is then sent out to the network. When the receiver goroutine receives a packet, it looks at the request ID to see if anyone has registered it. If there is a registration, the response is sent back to the requester, and the registration is cleared. If there is no registration, the response is simply discarded.

Since I have the need for this, if the idea is acceptable, I would be willing to work on it. It should be pretty easy to implement.

[wdreeveii]

I do create several hundred GoSNMP connections/structs to query in parallel
and have only seen minimal resource consumption. Have you tried it and
found resource contention
to be a problem?

On Sep 3, 2016 8:57 PM, "Patrick Hemmer" [email protected] wrote:

Further use of the plugin has come up with the need to actually make the
plugin parallel safe. Basically to speed up a large number of SNMP
operations, I need to issue them in parallel, instead of waiting for one to
finish before starting another. Currently the only way to do this is to
create multiple GoSNMP objects. And in my use case, this could
potentially be hundreds of them.

Instead I propose that the sendOneRequest be redesigned such that it
utilizes a single receiver goroutine, handling messages for any number of
requesters.
For example, when a request is made, the plugin registers the request ID
with a receiver goroutine that is already running. The request is then sent
out to the network. When the receiver goroutine receives a packet, it looks
at the request ID to see if anyone has registered it. If there is a
registration, the response is sent back to the requester, and the
registration is cleared. If there is no registration, the response is
simply discarded.

Since I have the need for this, if the idea is acceptable, I would be
willing to work on it. It should be pretty easy to implement.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#70 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AApLwTqdfgrKQjLmDC4I4X-onYcU8hvnks5qmk-ngaJpZM4JJCKu
.

[phemmer]

No, but even if my personal systems could support it, the project that uses this plugin is shared by others, and they might not have the same capabilities, or might have higher requirements than I. Additionally, the number of "connections" needed is dynamic. Without pre-creating the maximum number that might be needed, I would have to do something like a connection pool where I check them in and out, and also create as needed. This is just overly complex & messy for something that can be handled gracefully within the gosnmp package.

Edit: Went and created an implementation of the idea. Works, and the code is smaller too. Will polish it up later and submit a PR.

[wdreeveii]

V3 tests are passing in master - of course that only verifies functionality
against a NET-SNMP agent. What kind of problem are you seeing?

On Sun, Sep 4, 2016 at 9:04 PM, Patrick Hemmer [email protected]
wrote:

I've got the implementation for this finished, all the tests and
everything are done. Works fine in my tests so far, but I can't test it
with SNMP v3 as v3 support is currently broken (see #79
#79). Will wait until it gets
fixed before putting up PR.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#70 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AApLwclFWi6uqn4sl-kAOZpx8eS5CXyBks5qm6LUgaJpZM4JJCKu
.

Whitham D. Reeve II
907-764-5263

[phemmer]

I'm giving up on this. The code & design doesn't work with SNMP v3 due to the way gosnmp is written. For this to work, gosnmp needs to cache the security parameters (AuthEngineID, UserName, AuthKey, etc) for the connection. However it doesn't do this. It instead caches them per-request. Additionally gosnmp doesn't have the ability to unmarshal without performing authentication (which would be performed at a later step). These 2 things combined make it impossible for the proposal to work as the receiver is shared and receives responses for all requests, and thus doesn't have access to the per-request cached information.

Fixing the above issues would likely be fairly trivial for someone familiar with the code, but the code is extremely hard to follow. The large majority of the SNMP v3 implementation goes against what RFC3414 recommends (it also appears there's several auth related bugs because of this, but I haven't tested them). If the recommendations were followed, this likely wouldn't be an issue. I've now spent probably over 8 hours on this, and I'm tired of it. I'm putting this issue on the shelf, but will come back to it if someone else wants to tackle the v3 issues.

Edit: the general requirement for a shared receiver is that these lines be deleted and it still work

[wdreeveii]

GoSNMP does cache security parameters in the connection struct. if you
desire to separate unmarshaling and authentication - be my guest - the code
base is small and should easily afford this.

You are going to have to expand on the rest of this. If you see bugs,
please provide details.

On Mon, Sep 5, 2016 at 4:43 PM, Patrick Hemmer [email protected]
wrote:

I'm giving up on this. The code & design doesn't work with SNMP v3 due to
the way gosnmp is written. For this to work, gosnmp needs to cache the
security parameters (AuthEngineID, UserName, AuthKey, etc) for the
connection. However it doesn't do this. It instead caches them per-request.
Additionally gosnmp doesn't have the ability to unmarshal without
performing authentication (which would be performed at a later step). These
2 things combined make it impossible for the proposal to work as the
receiver is shared and receives responses for all requests, and thus
doesn't have access to the per-request cached information.

Fixing the above issues would likely be fairly trivial for someone
familiar with the code, but the code is extremely hard to follow. The large
majority of the SNMP v3 implementation goes against what RFC3414 recommends
(it also appears there's several auth related bugs because of this, but I
haven't tested them). If the recommendations were followed, this likely
wouldn't be an issue. I've now spent probably over 8 hours on this, and I'm
tired of it. I'm putting this issue on the shelf, but will come back to it
if someone else wants to tackle the v3 issues.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#70 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AApLwYvzm_x9UJyKtoZhJ-ctPTgWO1q9ks5qnLcagaJpZM4JJCKu
.

Whitham D. Reeve II
907-764-5263