iredmail / iredapd

A simple Postfix policy server.

Introduction

Don't forget to check out our lightweight email archiving software: https://spiderd.io/

• iRedAPD is a simple Postfix policy server, written in Python and runs as a low-privileged user (iredapd by default), with plugin support.
• The latest iRedAPD works with OpenLDAP, MySQL/MariaDB and PostgreSQL backends.
• License: GPL v3, except few files are in different licenses, shipped in iRedAPD for easier setup, but not license change.
  • file libs/daemon.py is BSD license.
  • file libs/srslib.py is Apache License, Version 2.0.
• Author: Zhang Huangbin .

NOTES:

• iRedAPD is a sub-project of iRedMail project.
• iRedAPD is installed and enabled in iRedMail by default, you don’t need this tutorial if you already have iRedMail running. For standard installation please check document INSTALL.md.
• iRedAPD listens on 3 ports by default:
  • 7777: normal smtp policy service
  • 7778: SRS (Sender Rewriting Scheme) for sender address rewriting
  • 7779: SRS (Sender Rewriting Scheme) for recipient address rewriting
• You can manage iRedAPD with iRedMail web admin panel - iRedAdmin-Pro.

Requirements

• Python 3.5+

Manage iRedAPD with command line tools

iRedMail project has a detailed tutorial to show you how to manage iRedAPD with command line tools: Manage iRedAPD

Available plugins

Plugins are files placed under plugins/ directory, plugin name is file name without file extension .py. It's recommended to read comment lines in plugin source files to understand what it does and how it works.

Plugins for all backends

• reject_to_hostname: reject emails sent to xxx@<server hostname> from external network.

• reject_sender_login_mismatch: Reject sender login mismatch (addresses in From: and SASL username). It will verify user alias addresses against SQL/LDAP database.

  This plugin also verifies forged sender address, e.g. sending email as a local domain to local domain.

• reject_null_sender: Reject message submitted by sasl authenticated user but use null sender in From: header (from=<> in Postfix log). RECOMMENDED to enable this plugin. It doesn't require SQL/LDAP query.

  If your user's password was cracked by spammer, spammer can use this account to bypass smtp authentication, but with a null sender in From: header, throttling won't be triggered.

• amavisd_wblist: Whitelist/blacklist for both inbound and outbound messages.

  The white/blacklists are used by both iRedAPD (before-queue) and Amavisd (after-queue).

• greylisting: for greylisting service.

• throttle: Throttling based on:

  • max number of mail messages sent/received in specified period of time
  • total mail size sent in specified period of time
  • size of single message

• whitelist_outbound_recipient: automatically whitelist recipient addresses of outgoing emails sent by sasl authenticated (local) users. It's able to whitelist single recipient address or domain for greylisting and normal white/blacklist.

Plugins for OpenLDAP backend

• ldap_maillist_access_policy: restrict who can send email to mail list.
• ldap_force_change_password_in_days: force users to change password in days (default 90 days). User cannot send email before resetting password.

Plugins for MySQL/MariaDB and PostgreSQL backends

• sql_alias_access_policy: restrict who can send email to mail alias.
• sql_force_change_password_in_days: force users to change password in days (default 90 days). User cannot send email before resetting password.

iredapd's Issues

Blocking of valid incoming emails with iRedAPD policy server

MTAs have a timeout limit for external server processes (e.g., policy servers or milters). For example, using a policy server like iRedAPD, the postfix MTA will wait for the completion of a process until it reaches its timeout limit, which is set by default to 100 seconds (policy_service_timeout). Since iRedAPD processes SMTP connections in a sequential manner, attackers can exploit this timeout mechanism: they do this by designing the SPF record of the sending domain to return with deliberate delays, thereby disrupting valid incoming email delivery within a specific timeframe.

Here's the scenario: attackers require a domain and a custom authoritative server for that domain, both of which can be relatively easily obtained and controlled. Consider an attacker owns a domain named a.com. Given iRedAPD's SPF lookup limit of 20, the attacker can configure a.com's SPF record requiring 20 lookups. By controlling the domain with a custom authoritative server, the attacker can introduce a delay of up to 3 seconds for each lookup (since iRedAPD allows up to 3 seconds for each DNS query). This setup extends the SPF validation time to 60 seconds in total (20 lookups * 3 seconds each).

One might argue that the DNS resolver's timeout could be shorter than 3 seconds, but this limitation can be bypassed: with CNAME record expansion. Attackers can use CNAMEs to create chains for domains included in the SPF record, and this expansion is handled by the resolver, not the SPF validator. This strategy increases the number of queries to the authoritative server without affecting iRedAPD's maximum lookup threshold.

With this scenario, if two emails are sent at t0, SPF validation for these couple emails will complete at time t120. This means, any benign emails being sent within t0 and t20 is going to be rejected since policy_service_timeout will be invoked. This is illustrated in the attached figure. Here t_lim is individual DNS query resolution time and T_lim is total SPF resolution time.

Can't list blacklist with wblist in 4.4 + 1 commit

Hi,

I juste want to upgrade iRedApd-4.4 to use wblist.py, but when I download the last package from https://github.com/iredmail/iRedAPD/releases/4.4.tar.gz or the ZIP file ; none of them include the change in the commit 4.4...master.

To be sure I apply manually the commit, and now I can add into blacklist /usr/bin/python /opt/iRedAPD-4.4/tools/wblist_admin.py --add --blacklist @example.com.

In the lastes devloppement (4.4 + 1 commit) I continue to see some .decode(), and for example, I can't list BL :

# /usr/bin/python /opt/iRedAPD-4.4/tools/wblist_admin.py --list --blacklist
* Establishing SQL connection.
* List all inbound blacklist for account: @.
AttributeError("'buffer' object has no attribute 'decode'",)

I'm available for testing if you need.

--
Thanks for all your work.

iRedAPD logrotate script generates Cron error messages

Hi,

This is a very minor bug which unfortunately generates tickets from our systems since we monitor every cron output.
Basically in

iRedAPD/samples/logrotate.d/iredapd

Lines 10 to 12 in 114f993

	postrotate
	/usr/sbin/service rsyslog restart
	endscript

Generates output:

/etc/cron.daily/logrotate:

Redirecting to /bin/systemctl restart rsyslog.service

Which causes our systems to log an incident since it`s not expected that a cronjob would generate unhanded messages in the error output.

Could you please change it to something like this, taken from the rsyslog logrotate job:

    postrotate
        /usr/bin/systemctl -s HUP kill rsyslog.service >/dev/null 2>&1 || true
    endscript

Since currently we have to apply this fix after every iRedAPD upgrade.

Thank you for the amazing software!

sqlAlchemy > 2.0 not working

I did a pip install sqlalchemy which installed sqlAlchemy 2.0.15 but using this version, I got a lot of errors like Error while executing SQL command: ObjectNotExecutableError() or AttributeError("'Engine' object has no attribute 'execute'"). Searching the internet I came across https://stackoverflow.com/questions/69490450/objectnotexecutableerror-when-executing-any-sql-query-using-asyncengine which is not direct related, but I downgraded to the latest 1.4 version and now it works.

iRedAPD log not working

I have installed the latest version of iRedAPD and successfully integrated it with Postfix in my Ubuntu after following the INSTALL.md file. But for some reason, logging is not working and I don't see any log being written in /var/log/iredapd/ directory. Permissions and other necessary checks are okay.

Can anyone suggest what is going wrong?

SRS does not rewrite addresses containing "/" and other permitted characters

Addresses like the following don't get rewritten. Here's a sample telnet conversation with the iredapd on the SRS port:

get Aee9Jz5ViQMS/ZuI48++OfQ==_1107588344726_8j8pIBXOEeW369SuUoQ9KA==@in.constantcontact.com

500 Not a valid email address, bypassed.

get Aee9Jz5ViQMSZuI48++OfQ==_1107588344726_8j8pIBXOEeW369SuUoQ9KA==@in.constantcontact.com

200 SRS0=/inf=JK=in.constantcontact.com=Aee9Jz5ViQMSZuI48++OfQ==_1107588344726_8j8pIBXOEeW369SuUoQ9KA==@mydomain.com

Note that if I remove the "/" character, it does work, and the output contains a "/" character.

Looking at the code, I see the function is_email(s) contains the following line:
if not set(s) & set(r'~!$%^*()/ ') and regxes.cmp_email.match(s):

So looks like any email containing a character in that set above is going to trigger a "500 Not a valid email address, bypassed."

The wikipedia article https://en.wikipedia.org/wiki/Email_address states that printable characters including !#$%&'*+-/=?^_`{|}~ are allowed. Even more stuff is allowed when quoted.

The rfc is little bit tougher to parse, but it looks like only back slash and quote character are prohibited.

https://tools.ietf.org/html/rfc5322#section-3.2.4
qtext = %d33 / ; Printable US-ASCII
%d35-91 / ; characters not including
%d93-126 / ; "" or the quote character
obs-qtext

The upgrade scripts doesn't support Debian Bullseye(sid).

BUG: greylisting plugin doesn't check alias domain

FYI: https://forum.iredmail.org/post72964.html#p72964

Does iRedAPD handle each SMTP session sequentially?

I am using Postfix as my MTA and iRedAPD as my greylisting plugin in an 8-core machine. I receive more than 10 simultaneous connections at times to my MTA and noticed that SPF checks are done sequentially for each incoming connection. (If two connections have MAIL FROM domain as a.com and b.com, b.com's SPF record is validated only after a.com's SPF validation is completed.)

Ideally, I expected SPF DNS queries for multiple connections will be mixed if they are processed concurrently (since it's an I/O bound job) but this did not seem to be the case.

Looking at the code, I found that iRedAPD creates a new async_chat server for every incoming connection which gives me the impression that it should support concurrent processing. I know it uses asyncore module which is by default single-threaded. So, using multiple cores even if they are available is out of the question.

But even if it's single-threaded, since SPF processing involves I/O operations, there should be a context switch achieving concurrency. But I observed sequential lookups in every case. To further diagnose this issue, I created a dynamic DNS server with Python and introduced a delay for SPF queries before sending a response (ideally, this delay should force a switch if concurrency is available). Still, the lookups were made sequentially.

Can anyone tell me if there is anything I'm missing or is it the expected behaviour of iRedAPD?

启动iredapd报错

from web import sqlquote
utils.py文件里有“from web import sqlquote”，提示“No module named 'web'”

iRedAPD doesn't support latest SQLalchemy

https://forum.iredmail.org/post88336.html#p88336
duplicate of #15

ANN: We're working on a new milter program to replace iRedAPD

Dear all,

Just want to share with you that we're developing a new milter program to replace iRedAPD, so don't feel sad if not many commits here.

Why?

iRedAPD doesn't get full email message (headers + body), hence it cannot do advanced access control based on info in headers and body. But milter can do this.

More details

• The milter program is written in Golang programming language.
  • The final program is only one binary file, very easy to deploy and upgrade.
  • It's expected to run faster than iRedAPD.
• All iRedAPD plugins will be implemented in first stable release (v1.0).
• Source code will be published when v1.0 is out, all further development will be open source too.
• We accept requests for paid plugin development too.

If you need some new features which are not yet implemented in iRedAPD, reply below to let us know. :)

Updates

• Aug 9: finished milter framework, still tune it and start porting iRedAPD plugins.

TODO

• Plugins:
  • amavisd_wblist
  • greylisting
  • ldap_force_change_password
  • ldap_maillist_access_policy
  • reject_null_sender. Aug 9.
  • reject_sender_login_mismatch
  • senderscore
  • sql_alias_access_policy
  • sql_force_change_password
  • sql_ml_access_policy
  • throttle
  • wblist_rdns
  • whitelist_outbound_recipient
  • reject_to_hostname [Removed in milter]

upgrade_iredapd.sh fails on FreeBSD 13

KERNEL_NAME="$(uname -s | tr '[a-z]' '[A-Z]')" obviously doesn't work on FreeBSD (13.0-RELEASE-p4, current port tree). I replaced that with export KERNEL_NAME="$(uname -s | tr '[:lower:]' '[:upper:]')" to fix this.

On FreeBSD, the script checks for CMD_PIP3='/usr/local/bin/pip3' but the port devel/py-pip doesn't install that file but only /usr/local/bin/pip which is a symlink to /usr/local/bin/pip-3.8.