Mail Server Setup

This document describes, more or less, the setup used for this system's mail server.

Background: the primary reason i've continued to use "shared hosters" for my websites and email over the years is my severe allergy against maintaining internet-facing mail servers. Thanks to spammers, internet mail is largely a minefield and home-managed mail servers have little chance of being seen as "legitimate" in the eyes of the Big Boy mail providers like Google and Microsoft. Setting up this VPS has not elinated that allergy but has forced me to confront it and finally set up a mail server. Co-developer Mark Jamsek provided the initial version of this document and several hours of support in getting it secured and running smoothly afterwards.

Technology Stack:

Plus it requires various types of DNS-level manipulation, including (ideally) the ability to apply a Reverse DNS entry.


This document is written from the perspective of an Ubuntu-derived Linux system using systemd, and requires adaptation for non-systemd and BSD system (e.g. sudo becomes doas and certain paths are different).

Additional Resources

This article has proven invaluable and informative:

DNS-Side Configuration

mydomain.org       A     123.321.123.321
mydomain.org       AAAA  2001:1002:7001:1007:69c0:0c96:aefc:cfae
mail.mydomain.org  A     123.321.123.321
mail.mydomain.org  AAAA  2001:1002:7001:1007:69c0:0c96:aefc:cfae
mydomain.org.      MX 0  mail.mydomain.org.

Sidebar: a common setup is to have mail.x.y be a CNAME for x.y, but RFC2181 strongly admonishes against doing so.

mydomain.org.    IN TXT    "v=spf1 mx -all"

TODO: why?

(a) generate keys

$ sudo mkdir /etc/mail/dkim
$ sudo openssl genrsa -out /etc/mail/dkim/mydomain.org.key 1024
$ sudo openssl rsa -in /etc/mail/dkim/mydomain.org.key -pubout -out /etc/mail/dkim/mydomain.org.pub
$ cat /etc/mail/dkim/mydomain.org.pub  # copy key from between delimiters for dkim record

(Noting that the location of the certificate is not important but must be noted for later reference.)

(b) create TXT record with pubkey (choose selector (e.g., YYYYMMDD))

YYYYMMDD._domainkey.mydomain.org.    IN TXT    "v=DKIM1;k=rsa;p=KEY_HEX_CODE;"

Note that the YYYYMMDD selector is free-form text. Shared hosters apparently commonly use "default" instead of a time-based one.

_dmarc.mydomain.org.    IN TXT    "v=DMARC1;p=none;pct=100;rua=mailto:postmaster@mydomain.org;"

TODO: why?


Obtaining TLS certificates via certbot is trivial but requires environment-specific instructions:

There are other approaches to obtaining certificats, but certbot is both easy to use and freely available, so we won't concern ourselves with them.

These docs assume certbot is used and that the certificates are placed in /etc/letsencrypt/live/mydomain.org.

These docs also assume that one has invoked certbot in a way which allows for automated updates of the certificates. It will install such automation by default if it is able.

Software Packages

In short:

$ sudo apt install dovecot dovecot-core \
    dovecot-imapd dovecot-pop3d dovecot-sqlite \
    dovecot-sieve \
    reddis rspamd \


1. Install it:

$ sudo apt install redis rspamd

2. Setup DKIM:

$ sudo mkdir /etc/rspamd/local.d
$ sudo vi /etc/rspamd/local.d/dkim_signing.conf

That file requires contents like the following:

allow_username_mismatch = true;
domain {
  mydomain.org {
    path = "/etc/mail/dkim/mydomain.org.key";
    selector = "YYYYMMDD";  # match DKIM selector above

3. Enable and start daemons:

$ sudo systemctl enable redis
$ sudo systemctl enable rspamd
$ sudo systemctl start redis
$ sudo systemctl start rspamd


1. Configure OpenSMTPD

$ sudo vi /etc/mail/smtpd.conf

That file should have contents similar to:

pki mail.mydomain.org cert "/etc/letsencrypt/live/mydomain.org/fullchain.pem"
pki mail.mydomain.org key "/etc/letsencrypt/live/mydomain.org/privkey.pem"

table dyndns file:/etc/mail/dyndns_regex
filter check_dyndns phase connect match rdns regex dyndns junk

filter check_rdns phase connect match !rdns junk

filter check_fcrdns phase connect match !fcrdns junk

filter senderscore proc-exec \
        "filter-senderscore -junkBelow 70 -slowFactor 5000"

filter rspamd proc-exec "filter-rspamd"

table aliases file:/etc/mail/aliases

listen on eth0 tls pki mail.mydomain.org filter \
        { check_dyndns, check_rdns, check_fcrdns, senderscore, rspamd }
listen on eth0 port submission tls-require pki mail.mydomain.org auth filter rspamd

action "local_mail" maildir junk alias <aliases>
action "outbound" relay helo mail.mydomain.org

match from any for domain "mydomain.org" action "local_mail"
match for local action "local_mail"

match from any auth for any action "outbound"
match for any action "outbound"

2. Dynamic DNS regex

$ sudo vi /etc/mail/dyndns_regex

And add:



1. Install:

$ sudo apt install dovecot dovecot-core \
    dovecot-imapd dovecot-pop3d dovecot-sqlite \
    dovecot-sieve dovecot-pigeonhole

3. Set certs

$ sudo vi /etc/dovecot/conf.d/10-ssl.conf

And add:

ssl_cert = </etc/letsencrypt/live/mydomain.org.fullchain.pem
ssl_key = </etc/letsencrypt/live/mydomain.org/privkey.pem

3. Set maildir

$ sudo vi /etc/dovecot/conf.d/10-mail.conf

And add:

mail_location = maildir:~/Maildir

(Tweak dir name to suit, but be sure that any relevant config files point to that same thing.)

4. Start daemon:

$ sudo systemctl enable dovecot
$ sudo systemctl start dovecot

Dovecot spam filter

1. Enable sieve

$ sudo vi /etc/dovecot/conf.d/20-imap.conf

With contents:

protocol imap {
    mail_plugins = $mail_plugins imap_sieve

2. Setup spam trainer

$ sudo vi /etc/dovecot/conf.d/90-plugin.conf

With contents:

plugin {
  sieve_plugins = sieve_imapsieve sieve_extprograms
  sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment

  imapsieve_mailbox1_name = Junk
  imapsieve_mailbox1_causes = COPY APPEND
  imapsieve_mailbox1_before = file:/usr/local/lib/dovecot/sieve/report-spam.sieve

  imapsieve_mailbox2_name = *
  imapsieve_mailbox2_from = Junk
  imapsieve_mailbox2_causes = COPY
  imapsieve_mailbox2_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve

  imapsieve_mailbox3_name = Inbox
  imapsieve_mailbox3_causes = APPEND
  imapsieve_mailbox3_before = file:/usr/local/lib/dovecot/sieve/report-ham.sieve

  sieve_pipe_bin_dir = /usr/local/lib/dovecot/sieve

3. Make sieve scripts

$ mkdir -p /usr/local/lib/dovecot/sieve
$ cd /usr/local/lib/dovecot/sieve
$ sudo vi report-ham.sieve

(a) With contents:

require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

if environment :matches "imap.mailbox" "*" {
  set "mailbox" "${1}";

if string "${mailbox}" "Trash" {

if environment :matches "imap.user" "*" {
  set "username" "${1}";

pipe :copy "sa-learn-ham.sh" [ "${username}" ];


$ sudo vi report-spam.sieve

With contents:

require ["vnd.dovecot.pipe", "copy", "imapsieve", "environment", "variables"];

if environment :matches "imap.user" "*" {
  set "username" "${1}";

pipe :copy "sa-learn-spam.sh" [ "${username}" ];


$ sudo vi sa-learn-ham.sh

With contents:

exec /usr/local/bin/rspamc -d "${1}" learn_ham

(d) vi sa-learn-spam.sh

exec /usr/local/bin/rspamc -d "${1}" learn_spam

4. Compile sieve scripts:

$ sudo sievec report-ham.sieve
$ sudo sievec report-spam.sieve
$ sudo chmod 755 sa-learn-ham.sh
$ sudo chmod 755 sa-learn-spam.sh

Client Device

Noting that mail is, when using this guide, configured to use the user's system-level password. Thus changing the system password affects all mail access.