Status of this repo / docker image about mailserver

[padrino121]

Time is a fleeting resource however given the popularity would you support some fundraising to help fund your continued time to maintain or fund maintenance utilizing other resources?

[SaraSmiseth]

@sknight80 I tried to reach out to you above, basically asking the same thing.

The new organisation created by @SaraSmiseth looks like a perfect place to accumulate all the changes but they have not responded so far. Ideally we would need rights to merge there or have PRs merged relatively quickly, which is a big ask.

I will invite you and the others and give you rights to merge PRs.

I am personally not volunteering to do that (lead maintaining a fork), is because there are a few areas that require some time investment. For example I think the tests are crucial, yet I'm not familiar with the testing framework, and I don't feel I can familiarise myself with it in reasonable time.

Well I think thats the main problem. I don't really want to be the lead maintainer either. Maybe we can split this if we have enough maintainers and whoever has time can review and merge PRs but I'm not really sure if thats a good idea.

I have experience with the tests as I have updated them. I have protected the master branch and made it so that changes are only possible in form of PRs which pass all tests. We should also require new tests for new features.

We should probably also require that at least 2 maintainers review PRs before they get merged.

[SaraSmiseth]

I started updating the main docker image for the mail server in my forked branch. I am planning to be ready for that image by the end of this week. After that, I am going to update this image with the latest versions. I am not sure about how to check the changes are still okay, but I will try to set up Travis for my forked branch. If all goes good, I am planning to update the setup to Traefik v2.x

I also forked this and updated most of it: https://github.com/mailserver2

I updated everything in the debian-mail-overlay image.

In the mailserver image I fixed all tests. Merged the dkim pr and the typo pr. I also updated zeyple and enabled the zeyple tests.

I have been using it for about a month.

The example configuration is not updated for Traefik v2.

Feel free to use my fork as a reference.

[AndrewSav]

@sknight80 so do you prefer to use your fork going forward or @SaraSmiseth 's?

I would like to also include rainloop updates (which I've done), under which repo do you guys think it's best to do?

It's kind of funny, three people duplicated the same work independently: same as you guys I upgraded postfixadmin, debian-mail-overlay, rspamd, etc to the latest. Did not go as far as including pull requests though and did not try to run the test.

Oh and I also updated traefik config to v2. I'll try and publish rainloop and traefik changes shortly, but it would be nice to decide in which repo the new home will be.

[prometheanfire]

I believe the traefik-2 change was merged a week or two ago.
I could look at the rspamd-1 -> rspamd-2 upgrade. Though that'd fall behind the traefik-2 and debian-10 changes as they are already merged (just time wise).
For Debian 10, this means the testing needs to hit the scenarios to be approved for stable.

I wonder if some CI testing like we've done in openstack would help (we have a test unit called 'grenade' that tests upgrades, along with various scenarios).

After each change, you need to check the building process, the logs
and behavior of all 6 images (mailserver_default, mailserver_reverse,
mailserver_ecdsa, mailserver_ldap, mailserver_ldap2,
mailserver_traefik_acmev1, mailserver_traefik_acmev2)

I personally do not know what goes into each of these configurations and tests, but it can likely be automated.

acmev1 and v2 can likely be merged as they are hitting the same code path (just verify that it gets the right data from the datastructure and puts it in the right place for both acmev1 and acmev2). Once the datastructure munging part is tested, whatever goes for standard testing would occur.

The ecdsa testing sounds like it's testing if random_curve can be made to work with postfix? The readme looks like it handles that already documentation wise. It may also be easier to state what is supported / tested and leave going beyond that to the end user. Need to know when and what to test and all that. (hopefully this would drop that test).

I do not know why there are two ldap tests either, nor do I know the reason for the default and reverse tests, but perhaps they could be pruned down or focused.

Should be able to drop to 5 tests in my view.

1. mailserver_default
2. mailserver_reverse
3. mailserver_ecdsa
   a. dropped
4. mailserver_ldap
5. mailserver_ldap2
6. mailserver_traefik_acmev1
7. mailserver_traefik_acmev2
   a. merged with other acme test

I'm not sure why the images need to be deployed in the real world for so long and so many configurations. Are you looking to test something specific or has it caught something unexpected in the past? If not I'd reduce this test to be what's needed, not what feels good (sounds like over-caution may be a thing here).

[alexanderadam]

This is a great project, the best of its kind I've found

May I ask what exactly makes it better than projects like Mailu, mailcow, tomav's mailserver, jeboehm's mailserver, kopano and other projects?

I mean, the only thing I'm personally missing on the other projects is the Zeyple integration (1, 2, 3) and security features like MailCrypt aren't even in this project (i.e. Mailcow has at least MailCrypt support in global mode and tomav's solution and Mailu might get it someday).

Also LDAP is already build in in tomavs mailserver, somehow working in mailcow (not yet in Mailu).

Mailserver administration is not an easy task and maybe it would be a good thing to merge efforts?
Or are those projects too different on their own? 🤔

[neuffer]

@hardware @AndrewSav @SaraSmiseth @sknight80 If the four of you would come to some kind of understanding of working together, maybe this project can be resurrected and people have once again a central converging point?
That would be awsome. :-)

[hardware]

Unmaintained status is not final if someone wants to take care of it. For now I have not found anyone to maintain this repo in an active and regular way, most people just want to make pull request (and that's fine), but that's not enough to maintain a project properly. And I do not want to leave any status, so I preferred to be clear about my position about the images set to find someone asap.

I have probably spent more than 1500+ hours on this docker image since 2016 and I need a maintainer who can take care of it on a monthly basis, because security and auditing are important, not a bit from time to time. But if several people can take care of it from time to time, it's good too.

This image was downloaded 1 million times last 6 months (jeez it's crazy), so that means a lot of people are using it, even small companies to my knowledge, so it's a responsibility.

[navossoc]

At the time, this was one of the few projects that use rspamd.

[prometheanfire]

I'd be willing to help out, I already maintain stuff for a distro downstream, though not mail (mainly ldap as related to this repo). I think the problem may be that it is more than one person can handle. What about this is taking the most time? Where would help be best applied?

[AndrewSav]

@neuffer from the state of this thread it does not seem likely

[AndrewSav]

In case anyone is interested I published my traefik v2 configs here: https://github.com/AndrewSav/mailserver/tree/master/docker-compose

Rainloop update is here: https://github.com/AndrewSav/rainloop

[sknight80]

Wow! Great @AndrewSav ! Thanks for sharing! If you don't mind, I would love to merge into my branch to test out in my setup.

@neuffer, I would be more than happy to start a collaboration in this project with the others. @AndrewSav , @SaraSmiseth what do you think?

[AndrewSav]

@sknight80 can you confirm that you got invites for the new org? Is there anything in your fork that is worth porting to that repo, now when you can merge there?

I probably need to make a PR for traefik and merge it. I put the new configs separately in my repo, but I think they need to replace the v1 in the common repo.

Also @SaraSmiseth @sknight80 what do we do with the docker images? We cannot update the old ones in place for obvious reasons, should we also create a new project on DockerHub?

[pollux]

@AndrewSav there already is a new project on dockerhub, its linked in the mailserver2/mailserver repo.

[SaraSmiseth]

I probably need to make a PR for traefik and merge it. I put the new configs separately in my repo, but I think they need to replace the v1 in the common repo.

Yes please create a PR. Yeah I think its better for the average user to put it all in one docker-compose.yml file. Makes it easier to get started if you only run mailserver.

Sure, but how do I, say, get the rainloop image pushed to it?

I just created a brand new discord server here https://discord.gg/nb8tRGs in case people want to discuss this in a chat rather than a thread.

I have added you to the dockerhub organization and created a new repository for rainloop and setup automated builds.

@sknight80 Tell me your dockerhub account and I'll add you as well.

[ksylvan]

My docker hub user is kayvan

I’d love to check this out as it cones back alive and update my ansible playbooks and contribute them to the repo.

[AndrewSav]

I got it! Will check it today.

@sknight80 no rush ;)

[AndrewSav]

@SaraSmiseth Understood. I'm not going to use matrix org as it's most clunky and slow piece of software IMO, I've ever used. It boggles my mind why people put up with using it. I just created another account on discord and joined the server all I was asked is my email and a new password - no other information, personal or not. Where did you get the phone number thing from? https://discord.com/register

[AndrewSav]

I opened a new thread here: mailserver2#9 since it could be more appropriate.

[shawniverson]

Following...

[hardware]

What about this is taking the most time ?

Testing / building / upgrade / project monitoring.

When you maintain an image for yourself it's easy, you can try and fail, then retry. But when people are using it with different purpose and usage, a good testing process, stability and backward compatibility are important, but that takes time.

Where would help be best applied ?

The next steps for this image are :

• Upgrade to Rspamd 2
• Upgrade to Treafik 2
• Upgrade to Debian 10 in stable branch (already working on the latest branch)

When you upgrade something, you need to test each piece of software, unit tests are very useful in this case, but it is not enough. After each change, you need to check the building process, the logs and behavior of all 6 images (mailserver_default, mailserver_reverse, mailserver_ecdsa, mailserver_ldap, mailserver_ldap2, mailserver_traefik_acmev1, mailserver_traefik_acmev2), then test in real conditions for weeks before pushing in stable branch.

[sknight80]

@hardware if we start adding testing infrastructure to this:

When you upgrade something, you need to test each piece of software, unit tests are very useful in this case, but it is not enough. After each change, you need to check the building process, the logs and behavior of all 6 images (mailserver_default, mailserver_reverse, mailserver_ecdsa, mailserver_ldap, mailserver_ldap2, mailserver_traefik_acmev1, mailserver_traefik_acmev2), then test in real conditions for weeks before pushing in stable branch.

Would be easier for maintaining the images? I am using this image and others from you, and I am willing to allocate my time to review, improve the repo.

[marcdeop]

I would be willing to help maintaining this software :-)

I have created a room at matrix.org: #mailserver:matrix.org in case anybody wants to join and have a more interactive talk/discussion.

Best,

Marc

[sknight80]

Hi Marcdeop! Sounds good. I joined to the chat.

[prometheanfire]

joined as well

[chuckmckinnon]

I'll join the chat. This is a great project, the best of its kind I've found, and it would be a shame for it to go unmaintained.

[shawniverson]

Count me in. I'll join.

[albjeremias]

is there a donation button somewhere? thank you so much for this project..

[albjeremias]

I'm not sure why the images need to be deployed in the real world for so long and so many configurations. Are you looking to test something specific or has it caught something unexpected in the past? If not I'd reduce this test to be what's needed, not what feels good (sounds like over-caution may be a thing here).

because email is complicated.. and if you want to test acme and SSL certificates you will need a real server for the tests!

mailserver_default

mailserver_reverse
mailserver_ecdsa
a. dropped
mailserver_ldap
mailserver_ldap2
mailserver_traefik_acmev1
mailserver_traefik_acmev2
a. merged with other acme test

I would suggest some other tests:

• test LDAP authentication
• test send an email
• test receive an email
  • test imap if email was received
  • test pop3 if email was received
• test spam filters somehow (SPF, DKIM, DMARC, ARC, ratelimit and greylisting capabilities)
• test automatic GPG
• test auto-responder, auto-forward
• test fetchmail
• test the webmail if logging in works
• test that you can login to postfixadmin
• find a test for unbound/NSD/traefik/SSL..?
• test LDAP (if login works)

[prometheanfire]

You can get certificates not via public IPs and not exposed to the internet. I'm using traefik and rfc2136 (send updates to bind), there are other plugins that can be used as well (cloudflare, etc). I'd say for testing self signed could even work, though I see no reason why that's needed.

[prometheanfire]

For me it was basically identical to the setup I was migrating from. There's also a cost to migration to consider. This project did a lot of things and was reasonably popular.

[albjeremias]

thank you @alexanderadam awesome post! :)

@prometheanfire let's think about that together? I also need to do it...

[msander]

Is there a convenient way to migrate to another project?

[KochFolie]

I would also like to thank you for this great project, which I've been using for some years.

That said I migrated to tomav's mailserver and, after realizing it doesn't support per-mailbox-quota, moved on to mailcow a few days ago. I basically just created my domains, mailboxes and aliases, sent test mails to them, so the folders would get created, copied over all mail-files and set correct owner/permissions. I would guess that this process is similar for most other projects.

[denji]

@KochFolie #48 (comment)

[AndrewSav]

@alexanderadam there are little differences everywhere. For example, I'm fond of Rainloop and in particular about it's feature that allows you to specify raw Sieve script for filtering. I'm using this feature extensively. From the list you've given, it appears that only mailu comes with Rainloop out of the box and then it has Sieve scripts disabled. It also has Rainloop admin panel disabled so you cannot re-enable it easily. Some people was able to do this via a config change, but that gets regenerated on every restart of the container.

Of course one could install rainloop separately, but the point of a project like this, is that you do not have to.

In conclusion, I do not believe that there is one size fits all.

[alexanderadam]

there are little differences everywhere.

Of course there are. It's also a good thing obviously. 😉

In conclusion, I do not believe that there is one size fits all.

I didn't claim that it have to. But a project should at least be maintained. And maybe even maintained by more than one person.
IMHO it would even better if there would be less popular projects at all. So that there are more maintainers/contributors per project.

This could increase development speed and increase the probability of implementing even "less popular" features (i.e. specifying Sieve scripts from Rainloop).

Having said that, the maintainers of mailu (or any other project) would probably even be happy if you would add a PR that adds this feature.

[neuffer]

The questions do we have a chance to get the project back into the status maintained?

[neuffer]

Which is a shame
The one alternative that I have found is sovereign, but it refused postfixadmin when Adrian Goins contributed the necessary patches oscapt/sovereign and has some very strange notions about users and their data.
An upgrade/install always includes the deletion of all accounts & user data. The users are predefined in the Ansible config files.

[AndrewSav]

Having said that, the maintainers of mailu (or any other project) would probably even be happy if you would add a PR that adds this feature.

Having a bit of experience with open source, it's not that simple. There are 30 open PR's at mailu right now, some of which months old. Some open project maintainers (I'm generalising I have no information about mailu in particular) are very strict with what they would merge. At the same, they are usually busy, and not available for a proposed PR discussion.

Not everyone has skills to create good PR too. Assuming I do, I would not waste my time working on a PR, that does not have a chance to get merged, so to me an approval in principal form the maintainers is a prerequisite for working on PR.

From my analysis of previous similar issues on mailu GitHub, the maintainer has a philosophy of "keeping it simple", this was the reason, why exposing Rainloop admin UI was rejected. I did open an issue re:sieve, just to find out how they go about such things, but I'm not very optimistic. Unfortunately, the way containers are stand up in mailu affect configuration management in Rainloop, in particular, main configuration files are assembled from scratch from minimalist templates, overwriting any changes that an admin could do should be thre Rainloop admin UI enabled. I'm not clear about the reason, why it's done this way, but it seems to have to do how discovery works for "neighbouring" containers which can differ, say between docker and kubernetes.

In areas of that complexity, from my experience maintainers sometimes are very reluctant to accept changes, since a person outside of the project has no chance to guess what is "the right" way from the maintainer perspective in such complex situations. At the same time they are understandably busy to be able to explain those to everyone who asks.

So in my opinion, it is possible but not that probable ;)

[alexanderadam]

The one alternative that I have found is sovereign

It's not using Docker though. Which might a requirement for some.

There are 30 open PR's at mailu right now, some of which months old.

There are also many closed PRs that were merged very recently.
It's not automatically a good sign to merge every PR if doesn't full-fill certain requirements.

Not everyone has skills to create good PR too.

I didn't claim so. 😉

I would not waste my time working on a PR, that does not have a chance to get merged

That's why communication is important. Simply ask whether that would be okay and what requirements have to be full-filled.

[sknight80]

I started updating the main docker image for the mail server in my forked branch. I am planning to be ready for that image by the end of this week. After that, I am going to update this image with the latest versions. I am not sure about how to check the changes are still okay, but I will try to set up Travis for my forked branch. If all goes good, I am planning to update the setup to Traefik v2.x

[neuffer]

@sknight80 where is your fork? I didn't find it in the list of forks here.

[sknight80]

The main image is here: https://github.com/sknight80/debian-mail-overlay
The mailserver is here: https://github.com/sknight80/mailserver

I am also planning to move over some PR that is in the queue here.

[sknight80]

@SaraSmiseth wow! That is awesome. I just updated the main image yesterday, I am going to push my changes to my branch, but now, you have done most of the work, I will use your fork going forward. Is it okay?

[SaraSmiseth]

@sknight80 Yes sure. If you are going to merge it back into this you have to change the links for the badges back to the original links.

The Dockerfile also still uses mailserver2/debian-mail-overlay. I'm not sure if you can unarchive hardware's debian-mail-overlay, merge my fork and then change it back.

[navossoc]

@AndrewSav going to traefik v2 is on my TODO list ;p

Have you tested the renew? Is it working properly?
Last time I checked there was some work being done to auto reload the certificates when they are renewed, not sure how it is right now...

[AndrewSav]

@navossoc I've been running traefik v2 elsewhere for months with no problems with renewals.

[neuffer]

@ksylvan How up-to-date are your mailserver Ansible configs/scripts? Maybe this could be merged in as well?

[AndrewSav]

@sknight80 I tried to reach out to you above, basically asking the same thing.

I did NOT do the tests and did NOT do the PRs. However I did traefik and rainloop.

The new organisation created by @SaraSmiseth looks like a perfect place to accumulate all the changes but they have not responded so far. Ideally we would need rights to merge there or have PRs merged relatively quickly, which is a big ask.

I am personally not volunteering to do that (lead maintaining a fork), is because there are a few areas that require some time investment. For example I think the tests are crucial, yet I'm not familiar with the testing framework, and I don't feel I can familiarise myself with it in reasonable time.

Similarly I have no experience with Clamav, Zeyple, Unbound, NSD, or OpenLDAP, and I'm personally not using these.

If you are happy to lead and maintain a fork, then you are of course welcome, I will help within constraints of abilities and time available. If not, then it will remain as it is now.

If you have any issues with integrating traefik 2, I'll do my best to work with you to resolve them, I'm quite experienced with traefik, and I do not anticipate any hard obstacles there.

[sknight80]

@neuffer : Currently, my fork is up-to-date. However, I need to test if the image is actually is capable to start. Last time when I checked there was some problem so, I have to check it.

[AndrewSav]

Oh and clamav now complains that it's out of date, even though it's the latest version available in debian apt

[sknight80]

@AndrewSav same here, however, I checked a couple of existing test but I need time to get familiar with it.

I added a little roadmap into my fork what I planned to do with the project, but I am more than happy to start working in a centralized place.

[AndrewSav]

I don't think there will be a lot of PRs to be honest so it could be workable. We'll see. Thank you for the invites.

I added rainloop here: https://github.com/mailserver2/rainloop

[AndrewSav]

Sure, but how do I, say, get the rainloop image pushed to it?

I just created a brand new discord server here https://discord.gg/nb8tRGs in case people want to discuss this in a chat rather than a thread.

[sknight80]

@sknight80 can you confirm that you got invites for the new org? Is there anything in your fork that is worth porting to that repo, now when you can merge there?

I probably need to make a PR for traefik and merge it. I put the new configs separately in my repo, but I think they need to replace the v1 in the common repo.

Also @SaraSmiseth @sknight80 what do we do with the docker images? We cannot update the old ones in place for obvious reasons, should we also create a new project on DockerHub?

I got it! Will check it today.

[sknight80]

I probably need to make a PR for traefik and merge it. I put the new configs separately in my repo, but I think they need to replace the v1 in the common repo.

Yes please create a PR. Yeah I think its better for the average user to put it all in one docker-compose.yml file. Makes it easier to get started if you only run mailserver.

Sure, but how do I, say, get the rainloop image pushed to it?

I just created a brand new discord server here https://discord.gg/nb8tRGs in case people want to discuss this in a chat rather than a thread.

I have added you to the dockerhub organization and created a new repository for rainloop and setup automated builds.

@sknight80 Tell me your dockerhub account and I'll add you as well.

My dockerhub account: knightsecret

[AndrewSav]

@SaraSmiseth do you happen to know if one can run tests locally?

[AndrewSav]

@SaraSmiseth sorry I recreated rainloop repo to fork from the origin, not from my fork, I saw you accessed it, and your access key may be lost now. I invite you to discord (link above) because I feel that discussing these details here is no longer appropriate.

[SaraSmiseth]

@SaraSmiseth do you happen to know if one can run tests locally?

Yes. Make sure you also initialize and clone the submodules. Just run make. This will build the image and run the tests.

@SaraSmiseth sorry I recreated rainloop repo to fork from the origin, not from my fork, I saw you accessed it, and your access key may be lost now. I invite you to discord (link above) because I feel that discussing these details here is no longer appropriate.

I'll join later.

EDIT: Well I joined #mailserver:matrix.org. Discord wants too much personal information. I'm not gonna give them my phone number.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.