Discussion: Options for Authenticator Decoupling about conjur

[infamousjoeg]

I'm not going to reply from Ben's perspective, but from the perspective of the customers I encounter in the Southeast region of the US.

They barely understand it as it stands today. All of Ben's "Fully Decoupling - General Feedback" section is completely spot on from my user's perspective.

If this was started from feedback from DevOps SMEs, I'd like it to be discussed with them prior to any further action on this to confirm the experience you're accepting from an R&D perspective is the same as their customers.

[jonahx]

These are comments @rafis3 sent me offline. I am responding to them here for visibility.

I am missing your opinion, which is very important to me. What is your thought and recommendation and why?

I think there are definitely tradeoffs, and agree with some of your concerns, but as I said fully decoupled strikes me as the superior Long-term solution. I will elaborate below...

Originally, we discussed that we need to understand if we want to go the route of making the authenticators pluggable or if the effort, code complexity, the decoupling process is too hard, to make it worth it.

Imo it needs to be done, if for no other reason than that it is a requirement to reduce our CI times to something reasonable. Our current CI is ~1hr. It's an absolute productivity killer.

If we created partial plugins based on an engine, rather than a gem, we could have had the routes in the plugin as well, right?

Yes, but I would strongly prefer to avoid engines. They bring in all the complexity of a complete rails app, and couple our authenticator code to rails, for no reason. The authenticator logic has nothing to do with rails, and taking on that dependency means that: It is yet another tool for developers learn (engines have subtle differences from rails); yet another thing that will need updates and might have CVE; the code will be more bloated than it needs to be. Also, as compared with vanilla middleware, they don't change anything from the POV the problems I raised.

The full decoupling might have performance implications, because there will be more networking between services. Kind of a step back to how Conjur v4 worked. Might not be significant, but still a risk I think The footprint will probably be larger with the full decoupling, if running more processes, more web services, because each brings its own stack which is an overhead

This is genuine concern I agree with. Unless we're very careful, it is also likely to add to complexity. Imo these are serious concerns that we need to discuss and find good solutions for. I think it is probably possible to do so.

Long-term, imo this is probably the right solution. this is a subjective “pro” as opposed to the others which are technical. So I think it’s worth adding why it’s the right solution, someone else could think differently if no explanation is provided.

My reasons are:

• It enables 3rd party development, which the field and sales engineers have been requresting for a long time. I understand the security concerns around this but believe they can be addressed with appropriate UNSAFE flags and warnings.
• Allows authenticators to be written in any language.
• Allows authenticators to scale independently of Conjur.
• It's conceptually simpler (though, as you noted, operationally more complex).

[rafis3]

Imo it needs to be done, if for no other reason than that it is a requirement to reduce our CI times to something reasonable. Our current CI is ~1hr. It's an absolute productivity killer.

There is more than one way to improve the CI. We can trigger tests that run as part of a PR and commit, based on the area that you touched. The authenticators tests don't have to run always, but can give an asynchronous feedback later. Since they also run in parallel, adding more authenticators doesn't necessarily mean more time with the important assumption that our tests are stable. We must have stable tests, so that more tests, doesn't increase the chance of random failures.

So what I would love to focus our conversation on is ROI. What do we invest, and how we are going to get the value. If the ROI is too low, then while the direction is good, it might not be justified to spend a few months of work. Because once we make a decision to go in that direction, we need to have a clear and realistic plan of how we make it happen.

[jonahx]

@rafis3 Fwiw, after a very long discussion with Ben, he's convinced me that the impact of option 1 (at least the version described above) is not feasible for field operations, and so it's off the table for now. We only want to find a solutions that work for everyone.

[rafis3]

Ok, let's discuss, align and capture what we learned.