Package EvaP (with nix?) about evap

[richardebeling]

I'm a bit pessimistic.

Yes, there are multiple steps, and they might have different hackiness levels, but I think they're all real complexity, so we won't be able to reduce it. We can distribute it across files, but I'm not sure if the indirection would actually help. I don't usually feel cognitively overwhelmed by the provision_vagrant script, so indirection might just make it more complicated, idk.

"Undo"-Operations in the current script: Optimally, most of our setup steps would be idempotent. Most of them are (since most linux programs behave idempotently, e.g., apt-install, service start, ...)

if we would have a more contained package, we could more easily recommend just rolling [an?] alternative development environment.

You mean switching between VM providers? For anything more "alternative", we would need to have the alternative already prepared and defined, right?

---

I'm a bit afraid of using exotic tools. I'd say we're most likely to make it easy for new contributors if we use the most widespread tools (with all error messages having the corresponding solutions written down on StackOverflow). To me, it seems podman and nix are a bit niche. I'd be fine with WSL2 for the reasons you gave.

For similar reasons, I favor simple setups: The fewer installations required the better. "git+vagrant+provider" is pretty minimalistic, the setup instructions on your branch seem more complicated to me at first glance.

sharing the environment with CI and production

the most apparent advantage still is that we can bundle all configuration and setup into a single source of truth though.

I'm not so sure about this. A full production grade apache2 config is huge, it hassles with SSL certificates, cipher choices, rate limiting, setting up mod_wsgi, caching, logging, etc. This is sysadmin stuff. EvaP imho can/should not provide this stuff, an administrator setting up the software will have to jump through the hoops to do it correctly. I'd argue that a CI setup should be minimal for fast startup time. I wouldn't want every CI job to waste time and energy reloading redis when we're not using redis while running tests.

[niklasmohrin]

Okay, I am going to try to tackle it from another angle: Docker with vagrant, for me, and with the current provision script, has failed. The idea with vagrant is that we can easily spin up a satisfactory environment that behaves as we expect. Now, with different docker behaviors on the different platforms (and different installation methods), vagrant is not able to do this.

However, I am a big fan of using docker for evap development, and on my computer, the setup usually works. It is a lot faster on my machine to start and while running than with the virtualbox provider (I would assume mainly because there are no hardware limits configured). However, the idea that we could use vagrant to share this setup with everyone, did not work out.

Using Virtualbox, for me (and I believe others) is inappropriate when I just want to develop userspace software, not an entire stack. One might say that some of us are in fact sometimes developing an entire stack, but I think this does not outweigh the desire to just work on the EvaP process while not caring about the rest.

(Arguably, the vagrant + virtualbox setup has thus failed in the same way, because it does not universally provide a satisfactory development environment to everyone, because it is too slow. However, for the people that haven't moved to docker and are happily using virtualbox, there is no reason to force them to abandon that either - in the end, we should still be able to use the packaged evap in a VM as before, just with a different stack)

---

Yes, there are multiple steps, and they might have different hackiness levels, but I think they're all real complexity, so we won't be able to reduce it.

I think there is a incidental complexity in there. It's small things like DEBIAN_FRONTEND=noninteractive, but also setting up bindfs, copying ssh keys, and updating pip - this has nothing to do with evap. Between those there is crucial stuff like generating a fresh secret key!

We can distribute it across files, but I'm not sure if the indirection would actually help. I don't usually feel cognitively overwhelmed by the provision_vagrant script, so indirection might just make it more complicated, idk.

I wouldn't say that I am overwhelmed, but I do not enjoy reading or editing this script - and without set -e, I also always hate finding what broke after noticing that ./manage.py run does not work in the end. In my opinion, the script is too long and does too many things.

"Undo"-Operations in the current script: Optimally, most of our setup steps would be idempotent. Most of them are (since most linux programs behave idempotently, e.g., apt-install, service start, ...)

Yes agree - I have to add that it would be even better if things like apt would also be idempotent regardless of time, something that a locked nix config would solve.

You mean switching between VM providers? For anything more "alternative", we would need to have the alternative already prepared and defined, right?

I meant that with a more contained package, developers could more easily use setups that we have not thought about yet, exactly because our package is agnostic of the environment. The goal would be that when someone comes thinking "I want to use qemu", then they can make sure that they get postgres and redis running and then just follow our normal instructions.

I'm a bit afraid of using exotic tools. I'd say we're most likely to make it easy for new contributors if we use the most widespread tools (with all error messages having the corresponding solutions written down on StackOverflow). To me, it seems podman and nix are a bit niche. I'd be fine with WSL2 for the reasons you gave.

For similar reasons, I favor simple setups: The fewer installations required the better. "git+vagrant+provider" is pretty minimalistic, the setup instructions on your branch seem more complicated to me at first glance.

For me, vagrant is more exotic than podman to be honest, nowadays containers seem to have replaced VMs for most purposes. In fact, I can count on one hand the number of projects I know that have VM configurations for development. Now, this does not mean that having such a config is bad, quite the opposite, it is forthcoming to new contributors, but I think if the goal is to aid people who are new to this stuff, we might as well pick them up with tools that fit today's standard practices. We don't need to settle on podman by the way, it is just what I found to be fitting and would suggest as our "official" recommendation.

So what about nix? For me, clearly nix > apt+pip+npm¹, it has worked out really well over the last months and I honestly believe that there is much to gain from this setup. I have admitted that the suggested setup is more complicated, because it is. I think that exposing the setup of the VM more directly is a good move though, because it empowers contributors to adapt the setup to their situation - and if at the end they can run nix develop, they are still good to go.

On the host machine side, I would also argue that "git + (nix on host OR nix in whatever virtualization layer you like)" is more minimalistic and less "intrusive" than forcing vagrant. While nix is admittedly more "intrusive" than vagrant on its own, the new setup allows you to choose where in your virtualization stack you place nix - and to omit virtualization altogether (unless you are on Windows).

About being afraid: At the end of the day, I am not 100% sure that this is going to work either - after all, there have been numerous times where we thought the vagrant configs are now really ready for everyone. This is why I would this kind of "experimental" phase during which we have both setups in the repository and users can try both out. In this time, we can also nail out common pain points and work on simplifying the first time setup.

¹ "npm" not in "nix" should sound very enticing to everyone who was worked on the vagrant setup :D

I'm not so sure about this. A full production grade apache2 config is huge, it hassles with SSL certificates, cipher choices, rate limiting, setting up mod_wsgi, caching, logging, etc. This is sysadmin stuff. EvaP imho can/should not provide this stuff, an administrator setting up the software will have to jump through the hoops to do it correctly.

Okay, admittedly I only know very little about apache, so that was maybe too ambitious. I was hoping that we could use the occasion to share more of the apache config in the repo (also to close my knowledge gap), but that seems out of scope then. My point that apache should not be embedded in the evap package still stands though - I think we should separate it as much as we can from the rest of the code, optimally, evap would be equivalently difficult to set up with nginx as it is with apache after we have packaged it - maybe this is unpractical though, like I said, no idea about this admin stuff.

I'd argue that a CI setup should be minimal for fast startup time. I wouldn't want every CI job to waste time and energy reloading redis when we're not using redis while running tests.

That is easy to do though, we can have multiple configurations (with different features turned on or off) all revolving around the central piece which is evap.

---

@Kakadus, @janno42 I would also love to hear what you think about this :)

[richardebeling]

Hmm. I'd definitely be open to try a nix setup. To judge whether I think it's a simpler setup declaration/definition, I'd have to the see comparable set of files for nix.

(Minor detail: I don't like "running two things in parallel" because I think that's what caused us most of the issues with docker vs virtualbox, but there's probably no way around it)