flashmob / go-guerrilla

Mini SMTP server written in golang

Important

Hi, my name is Philipp and I am one of the contributors to this project. Sadly it seems as if the original owner has abandonded go-guerilla. As I think there are still some nice things to do with it, and there might be some issues here and there, I have decided to revive this project.

I've already mirrored the repository to https://github.com/phires/go-guerrilla as I have not complete full access to this repo and don't know if it will disappear at some point. I will also try to migrate all further relevant informations (e.g. Wiki and Issues) over to the new repo.

If the original owner decides to come back I'll glady hand over full control of the projekt back to him. This should by no means be misinterpreted as a "hostile takeover" or anything. I just want to get this mighty fine piece of software back to speed and give it some further development.

-- 2023-08-31 Philipp

Latest: v1.6.1, tagged on Dec 28, 2019 (Pull requests from #129 to #203)

Go-Guerrilla SMTP Daemon

A lightweight SMTP server written in Go, made for receiving large volumes of mail. To be used as a package in your Go project, or as a stand-alone daemon by running the "guerrillad" binary.

Supports MySQL and Redis out-of-the-box, with many other vendor provided processors, such as MailDir and even FastCGI! See below for a list of available processors.

What is Go-Guerrilla?

It's an SMTP server written in Go, for the purpose of receiving large volumes of email. It started as a project for GuerrillaMail.com which processes millions of emails every day, and needed a daemon with less bloat & written in a more memory-safe language that can take advantage of modern multi-core architectures.

The purpose of this daemon is to grab the email, save it, and disconnect as quickly as possible, essentially performing the services of a Mail Transfer Agent (MTA) without the sending functionality.

The software also includes a modular backend implementation, which can extend the email processing functionality to whatever needs you may require. We refer to these modules as "Processors". Processors can be chained via the config to perform different tasks on received email, or to validate recipients.

See the list of available Processors below.

For more details about the backend system, see the: Backends, configuring and extending page.

License

The software is using MIT License (MIT) - contributors welcome.

Features

Main Features

• Multi-server. Can spawn multiple servers, all sharing the same backend for saving email.
• Config hot-reloading. Add/Remove/Enable/Disable servers without restarting. Reload TLS configuration, change most other settings on the fly.
• Graceful shutdown: Minimise loss of email if you need to shutdown/restart.
• Be a gentleman to the garbage collector: resources are pooled & recycled where possible.
• Modular Backend system
• Modern TLS support (STARTTLS or SMTPS).
• Can be used as a package in your Go project. Get started in just a few lines of code!
• Fuzz tested. Auto-tested. Battle Tested.

Backend Features

• Arranged as workers running in parallel, using a producer/consumer type structure, taking advantage of Go's channels and go-routines.
• Modular backend system structured using a decorator-like pattern which allows the chaining of components (a.k.a. Processors) via the config.
• Different ways for processing / delivering email: Supports MySQL and Redis out-of-the box, many other vendor provided processors available.

Roadmap / Contributing & Bounties

Pull requests / issue reporting & discussion / code reviews always welcome. To encourage more pull requests, we are now offering bounties.

Take a look at our Bounties and Roadmap page!

Getting started

(Assuming that you have GNU make and latest Go on your system)

Dependencies

Go-Guerrilla uses Dep to manage dependencies. If you have dep installed, just run dep ensure as usual.

You can also run $ go get ./.. if you don't want to use dep, and then run $ make test to ensure all is good.

To build the binary run:

$ make guerrillad

This will create a executable file named guerrillad that's ready to run. See the build notes for more details.

Next, copy the goguerrilla.conf.sample file to goguerrilla.conf.json. You may need to customize the pid_file setting to somewhere local, and also set tls_always_on to false if you don't have a valid certificate setup yet.

Next, run your server like this:

$ ./guerrillad serve

The configuration options are detailed on the configuration page. The main takeaway here is:

The default configuration uses 3 processors, they are set using the save_process config option. Notice that it contains the following value: "HeadersParser|Header|Debugger" - this means, once an email is received, it will first go through the HeadersParser processor where headers will be parsed. Next, it will go through the Header processor, where delivery headers will be added. Finally, it will finish at the Debugger which will log some debug messages.

Where to go next?

• Try setting up an example configuration which saves email bodies to Redis and metadata to MySQL.
• Try importing some of the 'vendored' processors into your project. See MailDiranasaurus as an example project which imports the MailDir and FastCGI processors.
• Try hacking the source and create your own processor.
• Once your daemon is running, you might want to stup log rotation.

Use as a package

Go-Guerrilla can be imported and used as a package in your Go project.

Quickstart

1. Import the guerrilla package

import (
    "github.com/flashmob/go-guerrilla"
)

You should use the dep ensure command to get all dependencies, as Go-Guerrilla uses dep for dependency management.

Otherise, $ go get ./... should work if you're in a hurry.

2. Start a server

This will start a server with the default settings, listening on 127.0.0.1:2525

d := guerrilla.Daemon{}
err := d.Start()

if err == nil {
    fmt.Println("Server Started!")
}

d.Start() does not block after the server has been started, so make sure that you keep your program busy.

The defaults are:

• Server listening to 127.0.0.1:2525
• use your hostname to determine your which hosts to accept email for
• 100 maximum clients
• 10MB max message size
• log to Stderror,
• log level set to "debug"
• timeout to 30 sec
• Backend configured with the following processors: HeadersParser|Header|Debugger where it will log the received emails.

Next, you may want to change the interface (127.0.0.1:2525) to the one of your own choice.

API Documentation topics

Please continue to the API documentation for the following topics:

• Suppressing log output
• Custom listening interface
• What else can be configured
• Backends
  • About the backend system
  • Backend Configuration
  • Registering a Processor
• Loading config from JSON
• Config hot-reloading
• Logging
• Log re-opening
• Graceful shutdown
• Pub/Sub
• More Examples

Use as a Daemon

Manual for using from the command line

• guerrillad command
  • Starting
  • Re-loading configuration
  • Re-open logs
  • Examples

Other topics

• Using Nginx as a proxy
• Testing STARTTLS
• Benchmarking

Email Processing Backend

The main job of a Go-Guerrilla backend is to validate recipients and deliver emails. The term "delivery" is often synonymous with saving email to secondary storage.

The default backend implementation manages multiple workers. These workers are composed of smaller components called "Processors" which are chained using the config to perform a series of steps. Each processor specifies a distinct feature of behaviour. For example, a processor may save the emails to a particular storage system such as MySQL, or it may add additional headers before passing the email to the next processor.

To extend or add a new feature, one would write a new Processor, then add it to the config. There are a few default processors to get you started.

Included Processors

Processor	Description
Compressor	Sets a zlib compressor that other processors can use later
Debugger	Logs the email envelope to help with testing
Hasher	Processes each envelope to produce unique hashes to be used for ids later
Header	Add a delivery header to the envelope
HeadersParser	Parses MIME headers and also populates the Subject field of the envelope
MySQL	Saves the emails to MySQL.
Redis	Saves the email data to Redis.
GuerrillaDbRedis	A 'monolithic' processor used at Guerrilla Mail; included for example

Available Processors

The following processors can be imported to your project, then use the Daemon.AddProcessor function to register, then add to your config.

Processor	Description
MailDir	Save emails to a maildir. MailDiranasaurus is an example project
FastCGI	Deliver email directly to PHP-FPM or a similar FastCGI backend.
WildcardProcessor	Use wildcards for recipients host validation.

Have a processor that you would like to share? Submit a PR to add it to the list!

Releases

Current release: 1.5.1 - 4th Nov 2016

Next Planned release: 2.0.0 - TBA

See our change log for change and release history

Using Nginx as a proxy

For such purposes as load balancing, terminating TLS early, or supporting SSL versions not supported by Go (highly not recommended if you want to use older TLS/SSL versions), it is possible to use NGINX as a proxy.

Credits

Project Lead:

Flashmob, GuerrillaMail.com, Contact: [email protected]

Major Contributors:

• Reza Mohammadi https://github.com/remohammadi
• Jordan Schalm https://github.com/jordanschalm
• Philipp Resch https://github.com/dapaxx

Thanks to:

• https://github.com/dvcrn
• https://github.com/athoune
• https://github.com/Xeoncross

... and anyone else who opened an issue / sent a PR / gave suggestions!

go-guerrilla's Issues

Unlimited number of recipients can be added (DOS vulnerability)

Unlimited number of recipients can be added to the client.RcptTo slice by repeating the RCPT TO command. This could lead to a DOS attack causing memory exhaustion.

This affects only the release candidate in master.

To do:

1. Limit the number to 100 max.

2. Additionally, RCPT TO and MAIL FROM should be checked for lengths too

Refactoring

In order to write unit tests, I want to extract storing mechanism (save_mail.go) from the code and move it into a package. Would you accept such refactoring? Are you ok with this layouting? Thanks :)

Support Prometheus by exporting metrics through `/metrics` endpoint

The Prometheus stack (Prometheus + Grafana) can be used to provide slick, powerful and flexible dashboards. All the project needs to do is to export metrics through an endpoint (e.g., /metrics) and let Prometheus scrape it. Prometheus has a powerful data model and query language (PromQL).

Missing go-qprintable Getting Started section of README

I needed go-qprintable for this to compile, which was missing in the getting started section of README.

eparra@eparra:/go$ go get github.com/ziutek/mymysql/thrsafe
eparra@eparra:/go$ go get github.com/ziutek/mymysql/autorc
eparra@eparra:/go$ go get github.com/ziutek/mymysql/godrv
eparra@eparra:/go$ go get github.com/sloonz/go-iconv
eparra@eparra:/go$ go get github.com/garyburd/redigo/redis
eparra@eparra:/go$ go build go-guerrilla.go
go-guerrilla.go:72:2: cannot find package "github.com/sloonz/go-qprintable" in any of:
/usr/local/go/src/github.com/sloonz/go-qprintable (from $GOROOT)
/Users/eparra/go/src/github.com/sloonz/go-qprintable (from $GOPATH)
eparra@eparra:/go$ go get github.com/sloonz/go-qprintable
eparra@eparra:/go$ go build go-guerrilla.go

Guide on how to setup for production usage

Hello and thanks for the great software. I am trying to set this up on a server but am unsure on how to actually make emails sent be received. I'm assuming I need to set up DNS but have no experience with setting up email server DNS. It would be helpful if the documentation or wiki had a section for setting up the domain / DNS.

Config reloading feature

Currently working on it. Hope to have the following features when a SIG_HUP is caught:

1. "allowed hosts" map reloading (need this since we change/add hosts frequently)
2. Backend switching, ie. detect if backend config changed, close old backend and spawn a new one
3. detect if SSL cert changed and reload it (useful since letsencrypt certs expire every few months, so our server would need to reload the ssl certs once per day at least)
4. Listening interface changes. Detect if interface changed. If changed, shutdown old server(s) and start a new one(s) (Not really needed feature, may leave for later)
5. Close and re-open log files. (Need this for logrotate)

That way the server can continue working without disrupting the hundreds of connections it may have open.

Going to implement it using an event based approach using https://github.com/asaskevich/EventBus

Please, don't suggest rotten ssl verions

        ssl_protocols  SSLv2 SSLv3 TLSv1;

SSLv2 doesn't exist anymore, and sslv3 is rotten.

Can we raise a PR on changing the interface completely to more modern flat UI?

refactor: Use context as cancelation signal

When a client is finished, it should be placed into a pool instead of being destroyed. Looking for a idiomatic Go solution with channels

Hello, interested in contributing to this project as I believe it will be very useful for a personal project I have in mind. Is implementing a client pool still one of the contributions you are looking for?

case-insensitive import collision in logrus

When importing as a package in an application already using logrus upon compilation it returns the following error:

src/github.com/flashmob/go-guerrilla/log/hook.go:5:2: case-insensitive import collision: "github.com/Sirupsen/logrus" and "github.com/sirupsen/logrus"

The current logrus readme reports the change in case for the github user asking the update of the package name whenever used to match the new lowercase one.

Could you please update?

Broke the ability to change backend via config

What's the license for this repo?

Can I fork, modify and run on my systems?

Take note of go-falcon

There a another great SMTP server in go called https://github.com/le0pard/go-falcon. It supports POP3 in addition to being a SMTP server which does spam and virus checks. Lots of good things could be borrowed including attachment saving/handling.

Test coverage

Hiya, I used go-guerilla to set up a dummy SMTP server for load testing our senders and I'd like to pay back the favor by increasing test coverage.

Curious about priority packages and testing conventions. Thanks!

Investigate & fix crashes from fuzz testing

The go-fuzz testing left some crashes to invetsigate here
https://github.com/flashmob/go-guerrilla/tree/tests/workdir/crashers

Fuzz testing procedure is described here:
https://github.com/flashmob/go-guerrilla/wiki/Fuzz-testing

Dodo:

1. Write tests to reproduce the crashes
2. Fix the code

Subject starts with ?utf-8?q?b2a9c4ea0f353f9c7f9583b54bbe348a2cc42=5F=5F?=

Some web mail client [SOGo] send the subject starting with ?utf-8?q?the subject....
Shall it be decoded within go-guerrilla?

How do I build/run this?

Apologies for the naive question. I am currently investigating different SMTP servers for receiving email and this one caught my attention (wanting to build that project in Go).

I seem to have problems figuring out how to get started with this project.

To build, just run

$ go build

(...)

This information is a little bit unclear. Running build from the root doesn't seem to do much. How were I to go specifically from checking this project out to actually running it?

Any hint in the right direction would be greatly appreciated!

guerrillamail duplicates a dot character when the dot appears at index 76

Example:
Send email from anywhere to guerrillamail with the following text:
123456789012345678901234567890123456789012345678901234567890123456789012345.com
Result in Show original:
123456789012345678901234567890123456789012345678901234567890123456789012345=
..com
guerrillamail duplicates the dot.
This causes links to be broken.
Good luck :)

send incoming emails to api

Hi, good day, I'd like ask you if it's possible send the incoming emails to our api, for instance, catch the subject, sender and body and send to some api rest through http...thank you!!

Priviledge separation

Hello.

When you executes guerrillad server as root the run with 0 uid (root uid).
I propose you to add priviledge separation using something function similar to https://golang.org/pkg/os/#StartProcess

You can read about github.com/sevlyar/go-daemon using daemons and handling processes in background with gouroutine functionality.

Thank you!

Bug: Leading dots are not transformed when reading in Data state

SMTP spec uses dot stuffing when reading in after the DATA command. However, the current server implements it incorrectly where it doesn't remove a dot if a line with text starts with a dot.

To fix:

Will switch to using go's standard package textproto, using the DotReader.

Additionally, this is an opportunity to change Envelope.Data to a bytes.Buffer, so that the buffer can be re-used after it is placed back in the pool.

The textproto package could also be used for header parsing too.

TLS - use modern configuration introduced in Go 1.8

https://blog.gopheracademy.com/advent-2016/exposing-go-on-the-internet/

Requirements:

• Ability to set the modern TLS config via the json file config. (For each server).

• Sample json config to contain settings as recommended on above blog

Missing table creation in README

Hi there,

I had to run this to get go-guerrilla to run:

CREATE TABLE gm2_setting (
setting_name VARCHAR(128),
setting_value INT NOT NULL DEFAULT 0
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Maybe it should be included in the README.

Don't advertise STARTTLS if certificate loading fails

If loading the certificate fails, we must not advertise STARTTLS nor we should open an TLS-only host.
A check is to be implemented, if the configured certificate is existing and maybe even a check if it is valid.

Put proxy support back in

Even though proxying via Nginx is not necessary these days, proxy support may still be useful for load balancing purposes.

The proxy server uses the XCLIENT command to tell the server behind the proxy what the source IP address is.

Requirements:

Support the XCLIENT command (with an option to enable it in config).

case strings.Index(cmd, "XCLIENT") == 0:
				// Nginx sends this
				// XCLIENT ADDR=212.96.64.216 NAME=[UNAVAILABLE]
				client.Address = input[13:]
				client.Address = client.Address[0:strings.Index(client.Address, " ")]
				fmt.Println("client address:[" + client.Address + "]")
				responseAdd(client, "250 OK")

Just fyi...

You left out:

    $ go get github.com/sloonz/go-qprintable

In your install instructions. I know this isn't a big deal, just figured worth keeping the documentation up to date. :)

Wrong remote IP address

Hello,

In mysql i'm logging all the same IP addresses. Upon further investigation, I see the wrong remote_address being shown in console (verbose) output as well.

It appears the first received email (expected test emails from my web server) remote address is used for all subsequent incoming mail.

Here's my evidence/findings so far.. (addresses obfuscated)

Console output:

INFO[14028] Handle client [<myWebServer>:50219], id: 2    
DEBU[14028] Writing response to client: 
220 mx.mydomain.com SMTP Guerrilla(1.5.1-30-g44d8a9d) #2 (1) 2017-01-21T09:56:12-05:00 gr:9
 
DEBU[14028] Client sent: HELO localhost                  
DEBU[14028] Writing response to client: 
250 mx.mydomain.com Hello
 
DEBU[14028] Client sent: MAIL FROM: <[email protected]>     
DEBU[14028] Writing response to client: 
250 OK
        
DEBU[14028] Client sent: RCPT TO: <[email protected]> 
DEBU[14028] Writing response to client: 
250 OK
        
DEBU[14029] Client sent: DATA                            
DEBU[14029] Writing response to client: 
354 Enter message, ending with '.' on a line by itself
 
DEBU[14029] Email saved c769dd047cbc18eb3eb754d22c5bcb3a (len=20080) 
DEBU[14029] Writing response to client: 
250 OK : queued as c769dd047cbc18eb3eb754d22c5bcb3a
 
DEBU[14029] Client sent: QUIT                            
DEBU[14029] Writing response to client: 
221 Bye

And that mail is recorded in MySQL as coming from MY Web server IP:

+---------------------+-----------------+
| date                | ip_addr         |
+---------------------+-----------------+
| 2017-01-20 18:20:53 | <myWebServer>:52 |
| 2017-01-20 18:40:50 | <myWebServer>:58 |
| 2017-01-21 01:34:31 | <myWebServer>:58 |
| 2017-01-21 01:43:22 | <myWebServer>:58 |
| 2017-01-21 01:46:22 | <myWebServer>:58 |
| 2017-01-21 04:44:12 | <myWebServer>:58 |
| 2017-01-21 06:01:08 | <myWebServer>:58 |
| 2017-01-21 06:03:09 | <myWebServer>:50 |
| 2017-01-21 09:56:13 | <myWebServer>:50 | 
+---------------------+-----------------+
9 rows in set (0.00 sec)

But myWebServer never sent an email at 09:56:13 and a tcpdump on myGuerrillaMail server proves this:

09:56:12.187568 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [S], seq 3530872155, win 14600, options [mss 1460,sackOK,TS val 13352891 ecr 0,nop,wscale 7], length 0
09:56:12.187610 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [S.], seq 1217573791, ack 3530872156, win 14480, options [mss 1460,sackOK,TS val 2926659315 ecr 13352891,nop,wscale 7],
 length 0
09:56:12.332680 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], ack 1, win 115, options [nop,nop,TS val 13352927 ecr 2926659315], length 0
09:56:12.333016 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [P.], seq 1:96, ack 1, win 114, options [nop,nop,TS val 2926659461 ecr 13352927], length 95
09:56:12.477180 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], ack 96, win 115, options [nop,nop,TS val 13352964 ecr 2926659461], length 0
09:56:12.496068 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [P.], seq 1:17, ack 96, win 115, options [nop,nop,TS val 13352968 ecr 2926659461], length 16
09:56:12.496083 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 17, win 114, options [nop,nop,TS val 2926659624 ecr 13352968], length 0
09:56:12.496227 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [P.], seq 96:125, ack 17, win 114, options [nop,nop,TS val 2926659624 ecr 13352968], length 29
09:56:12.663587 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [P.], seq 17:46, ack 125, win 115, options [nop,nop,TS val 13353010 ecr 2926659624], length 29
09:56:12.663781 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [P.], seq 125:133, ack 46, win 114, options [nop,nop,TS val 2926659792 ecr 13353010], length 8
09:56:12.834716 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [P.], seq 46:84, ack 133, win 115, options [nop,nop,TS val 13353053 ecr 2926659792], length 38
09:56:12.834911 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [P.], seq 133:141, ack 84, win 114, options [nop,nop,TS val 2926659962 ecr 13353053], length 8
09:56:12.992272 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [P.], seq 84:90, ack 141, win 115, options [nop,nop,TS val 13353092 ecr 2926659962], length 6
09:56:12.992404 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [P.], seq 141:197, ack 90, win 114, options [nop,nop,TS val 2926660120 ecr 13353092], length 56
09:56:13.150204 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 90:1538, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150213 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 1538:2986, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150238 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 2986, win 159, options [nop,nop,TS val 2926660278 ecr 13353132], length 0
09:56:13.150254 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 2986:4434, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150261 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 4434:5882, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150282 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 5882, win 204, options [nop,nop,TS val 2926660278 ecr 13353132], length 0
09:56:13.150268 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 5882:7330, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150297 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 7330:8778, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150302 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 8778, win 249, options [nop,nop,TS val 2926660278 ecr 13353132], length 0
09:56:13.150358 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 8778:10226, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150365 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 10226:11674, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150380 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 11674, win 295, options [nop,nop,TS val 2926660278 ecr 13353132], length 0
09:56:13.150372 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 11674:13122, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150422 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [P.], seq 13122:14570, ack 197, win 115, options [nop,nop,TS val 13353132 ecr 2926660120], length 1448
09:56:13.150427 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 14570, win 340, options [nop,nop,TS val 2926660278 ecr 13353132], length 0
09:56:13.294485 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 14570:16018, ack 197, win 115, options [nop,nop,TS val 13353168 ecr 2926660278], length 1448
09:56:13.294494 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 16018:17466, ack 197, win 115, options [nop,nop,TS val 13353168 ecr 2926660278], length 1448
09:56:13.294526 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 17466, win 385, options [nop,nop,TS val 2926660422 ecr 13353168], length 0
09:56:13.294552 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 17466:18914, ack 197, win 115, options [nop,nop,TS val 13353168 ecr 2926660278], length 1448
09:56:13.294563 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [.], seq 18914:20362, ack 197, win 115, options [nop,nop,TS val 13353168 ecr 2926660278], length 1448
09:56:13.294585 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 20362, win 430, options [nop,nop,TS val 2926660422 ecr 13353168], length 0
09:56:13.294571 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [P.], seq 20362:20448, ack 197, win 115, options [nop,nop,TS val 13353168 ecr 2926660278], length 86
09:56:13.333669 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 20448, win 430, options [nop,nop,TS val 2926660462 ecr 13353168], length 0
09:56:13.362872 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [P.], seq 197:250, ack 20448, win 430, options [nop,nop,TS val 2926660491 ecr 13353168], length 53
09:56:13.527752 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [P.], seq 20448:20454, ack 250, win 115, options [nop,nop,TS val 13353226 ecr 2926660491], length 6
09:56:13.527774 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 20454, win 430, options [nop,nop,TS val 2926660655 ecr 13353226], length 0
09:56:13.527943 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [P.], seq 250:259, ack 20454, win 430, options [nop,nop,TS val 2926660655 ecr 13353226], length 9
09:56:13.527985 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [F.], seq 259, ack 20454, win 430, options [nop,nop,TS val 2926660656 ecr 13353226], length 0
09:56:13.674598 IP 95.47.x.x.57927 > <myGuerrillaMailServer>.25: Flags [F.], seq 20454, ack 260, win 115, options [nop,nop,TS val 13353263 ecr 2926660655], length 0
09:56:13.674622 IP <myGuerrillaMailServer>.25 > 95.47.x.x.57927: Flags [.], ack 20455, win 430, options [nop,nop,TS val 2926660802 ecr 13353263], length 0

The IP address 95.47.x.x is the remote address I'm expecting to see/log.

MySQL table creation

I tried to create my own table and I'm missing some data. Where is the table schema that was used with this?

Feature request: usage of queues

It would be cool if instead of using a mix of redis/mysql it would use a queue (rabbit/beanstalkd/maybe even sqs) to push new messages into (might not work with large attachments though).

That way the implementation of what is being done with a email is fully seperated from the mx.

let backend processors specify the response error code

the BackendGateway will always return FailBackendTransaction (554) if there is an error in one of the backend processors.
It would be useful if the error specified from the processor would be parsed and returned if valid...similar to this:

// wait for the save to complete
        // or timeout
        select {
        case status := <-workerMsg.notifyMe:
                defer workerMsgPool.Put(workerMsg) // can be recycled since we used the notifyMe channel
                if status.err != nil {
                        if _, err := strconv.Atoi(status.err.Error()[:3]); err != nil {
                                return NewResult(response.Canned.FailBackendTransaction + status.err.Error())
                        }
                        return NewResult(status.err.Error())
                }
                return NewResult(response.Canned.SuccessMessageQueued + status.queuedID)

        case <-time.After(gw.saveTimeout()):
                Log().Error("Backend has timed out while saving eamil")
                return NewResult(response.Canned.FailBackendTimeout)
        }

in this way the backend could respond 421 (for example) and allow the client to retry to send the email.

Another solution could be to add the backend result in the notifyMsg (or a pointer to it)

doc: Document AppConfig / ServerConfig

I'm looking to implement LetsEncrypt but I've got stuck on basic understanding of the code due the lack of documentation. What is AppConfig.AllowedHosts and how is it related to ServerConfig.Hostname ?

latest go-guerrilla doesn't compile

Hello, I tried to go get the latest changes but go install is failing.
flashmob/go-guerrilla/event.go:78: cannot use EventBus.New() (type EventBus.Bus) as type *EventBus.EventBus in assignment: need type assertion

I noticed that the package github.com/asaskevich/EventBus has been changed 6 days ago and created a problem with the compilation of go-guerrilla.
asaskevich/EventBus@5666508

backends: Backend interface should not have private methods

Why does Backend has private methods?

go-guerrilla/backends/backend.go

Line 20 in 9d9d864

type Backend interface {

The readme file mentions that different backends can be implemented but it doesn't seem to be the case because you can't use unexported types. It is misleading at best for several reasons:

• Currently it's impossible to implement the Backend interface due the private methods
• type Envelope is in the envelope package so in the example it should be c *envelope.Envelope not c *guerrilla.Envelope. type BackendResult is located in the backends package so it should be backends.BackendResult not guerrilla.BackendResult

type CustomBackend struct {...}

func (cb *CustomBackend) Process(c *guerrilla.Envelope) guerrilla.BackendResult {
  err := saveSomewhere(c.Data)
  if err != nil {
    return guerrilla.NewBackendResult(fmt.Sprintf("554 Error: %s", err.Error()))
  }
  return guerrilla.NewBackendResult("250 OK")
}

An exported interface with private methods makes no sense, does it?

Expand Logging functionality

• Main log for global events, eg. starting / stopping servers, config reloads, other general info / errors etc.
• Specific logs for each server instance
• ability to work with log rotate https://linuxconfig.org/logrotate-8-manual-page

[Question] Stop output

Hi, How can i tell the package to stop outputting ?

Envelope.RemoteAddress should not include port

Already have a fix - this issue is just to track the fix

What role does redis play?

It looks like redis is used to save data in the channel as it comes in but I don't see anywhere where the data is being read and it seems the data is actually saved to MySQL ultimately too.

What is redis doing?

Streaming data

Is there a way to stream "DATA command" payload. Looking at the code it seems data is kept in memory buffer and then passed it on to the backend processor.

While keeping data in memory gives you better performance. This could be a problem if you have too many connections or large payloads.

Any suggestions?

Thanks.

sighup not available on Windows

How about adding a new 'signal' command to ./guerrillad

To reload config
guerrillad signal reload

For graceful shutdown
guerrillad signal quit

To implement it, assuming that we can write syscall.SIGHUP and syscall.SIGTERM to the signalChannel channel declared in serve.go

Modularize

Hi, I'm currently working on a project that will require an SMTP server with API hooks for processing received mail, and it sounds like you'd like this project to support that, given the Modularize bounty.
I would be interested in doing a code review and implementing a public API for the package, if that's still something you are interested in adding.

Redis backend: failed while db.Prepare(INSERT...) ... Unknown column 'return_path'...

Hello,

I'm attempting to use the guerrilla-db-redis backend. I've populated my mysql DB with the SQL provided in README.md but I still get the following error.

[root@Meteor go-guerrilla]# mysql
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 53
Server version: 5.5.52-MariaDB MariaDB Server

Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> use gmail
Database changed
MariaDB [gmail]> CREATE TABLE IF NOT EXISTS `new_mail` (   `mail_id` int(11) NOT NULL auto_increment,   `date` datetime NOT NULL,   `from` varchar(128) character set latin1 NOT NULL,   `to` varchar(128) character set latin1 NOT NULL,   `subject` varchar(255) NOT NULL,   `body` text NOT NULL,   `charset` varchar(32) character set latin1 NOT NULL,   `mail` longblob NOT NULL,   `spam_score` float NOT NULL,   `hash` char(32) character set latin1 NOT NULL,   `content_type` varchar(64) character set latin1 NOT NULL,   `recipient` varchar(128) character set latin1 NOT NULL,   `has_attach` int(11) NOT NULL,   `ip_addr` varchar(15) NOT NULL,   `delivered` bit(1) NOT NULL default b'0',   `attach_info` text NOT NULL,   `dkim_valid` tinyint(4) default NULL,   PRIMARY KEY  (`mail_id`),   KEY `to` (`to`),   KEY `hash` (`hash`),   KEY `date` (`date`) ) ENGINE=InnoDB  DEFAULT CHARSET=utf8;
Query OK, 0 rows affected (0.01 sec)

MariaDB [gmail]> Bye
[root@Meteor go-guerrilla]# ./guerrillad  -v serve
INFO[0000] guerrillad 1.5.1-30-g44d8a9d                 
DEBU[0000] Build Time: 2017-01-20_01:28:15_-0500        
DEBU[0000] Commit:     44d8a9dc97cd149d7cbce80a293dda9ff248ea97 
DEBU[0000] making servers                               
INFO[0000] Listening on TCP 0.0.0.0:25                  
DEBU[0000] [0.0.0.0:25] Waiting for a new client. Next Client ID: 1 
INFO[0000] pid_file (/var/run/go-guerrilla.pid) written with pid:6849 
FATA[0000] failed while db.Prepare(INSERT...)            error="Received #1054 error from MySQL server: \"Unknown column 'return_path' in 'field list'\""

Shouldn't the schema be 1:1 not 15 of 17? (sorry for the poor illustration!):

[root@Meteor go-guerrilla]# mysql gmail -Ne 'desc new_mail' | nl
     1  mail_id int(11) NO      PRI     NULL    auto_increment
     2  date    datetime        NO      MUL     NULL
     3  from    varchar(128)    NO              NULL
     4  to      varchar(128)    NO      MUL     NULL
     5  subject varchar(255)    NO              NULL
     6  body    text    NO              NULL
     7  charset varchar(32)     NO              NULL
     8  mail    longblob        NO              NULL
     9  spam_score      float   NO              NULL
    10  hash    char(32)        NO      MUL     NULL
    11  content_type    varchar(64)     NO              NULL
    12  recipient       varchar(128)    NO              NULL
    13  has_attach      int(11) NO              NULL
    14  ip_addr varchar(15)     NO              NULL
    15  delivered       bit(1)  NO              b'0'
    16  attach_info     text    NO              NULL
    17  dkim_valid      tinyint(4)      YES             NULL
[root@Meteor go-guerrilla]# git grep return_path | tr , '\n' | nl
     1  backends/guerrilla_db_redis.go: sql += "(`date`
     2   `to`
     3   `from`
     4   `subject`
     5   `body`
     6   `charset`
     7   `mail`
     8   `spam_score`
     9   `hash`
    10   `content_type`
    11   `recipient`
    12   `has_attach`
    13   `ip_addr`
    14   `return_path`
    15   `is_tls`)"

Analytics dashboard

@jordanschalm has started working on Analytics package
https://github.com/flashmob/go-guerrilla/tree/dashboard
We have agreed that we will drop the TLS & password protection requirement, since we do not have Lets Encrypt automation yet, and self signed certs can be MITM'd. So bind to localhost for now.
The way it may be implemented is via the existing logging facility. A goroutine will log take samples periodically and write them to the log. Another goroutine can tail the log and ready the data for presentation.
(Note that in the future, the log could be closed and reopened upon a SIGHUP signal)
Presentation data will be rendered using vanilla Js and a js charting library.

Add an Easter Egg

What software is complete without an easter egg?

The HELP command would be a great candidate for an easter egg, since this command is mostly unused and disabled on many other servers.

Todo:

HELP command to reply with random one line quotes from the Big Lebowski
https://www.rottentomatoes.com/m/big_lebowski/quotes/

The Dude abides.

SSL support without nginx

Some clients are refusing to connect over SSL due to either v2 not being supported, or v3 not agreeing a cipher. Is there some config change required to allow other ciphers, or is nginx still required to handle STARTTLS connections?

Support enhanced status codes

RFC https://tools.ietf.org/html/rfc3463

https://www.iana.org/assignments/smtp-enhanced-status-codes/smtp-enhanced-status-codes.xml

1. Implement enhanced status codes to all status messages

2. Advertise 250-ENHANCEDSTATUSCODES after EHLO

3. Automated tests for all codes

sending email

What solution would you use/propose for sending email ? (I understand that go-guerrilla is only for receiving emails).

I would like to adapt go-guerrilla to build my own webmail. Wouldn't be ok for the local smtp client to talk to this server?
thanks

MAIL FROM empty raises error

Currently Mails are being rejected, if the MAIL FROM is empty (e.g. MAIL FROM:<>).
This is technically allowed according to RfC 5321 and designates a bounce message.

Issue & configure Let's Encrypt certificates

Let’s Encrypt and the golang.org/x/crypto/acme/autocert package’s GetCertificate function.

Requirements:

• Implemented as a set of new commands, eg.

./guerrillad certificate <server-interface> new to issue a new cert & add to config. (using the primary_mail_host as as the FQDN

./guerrillad certificate <server-interface> renew to renew

./guerrillad certificate renew to renew all certificates

Why all the read buffering?

There seems to be a lot of buffering structs added. Why are they being used? Was the go defaults for reading from an io.Reader unsafe?

Are you trying to limit the amount read? You could use an io.LimitReader():

data, err := ioutil.ReadAll(io.LimitReader(client.conn, 1000000))