Sending email can be hard...
Sending email from Amazon EC2 instances is not as simple as it looks, most organisations will regard your emails as spam and Amazon restricts the number of emails that you can send. They have obviously had issues with people sending spam emails from EC2 instances in the past. Despite these issues, it is still possible to host your email servers in EC2 and send email externally which is trusted, you just need to jump through a few more hoops to make it happen (or use my postfix profile to take care of some of the hard work). The following guide is intended for people or organisations which want to legitimately host their email out of Amazon EC2 (as I have done for the practicalclouds.com site) - this isn't a guide explaining how you can send thousands of spam messages, sorry! (not my thing, I'd rather write guides on how to block your spam).
Here is a list of actions you should take in order to host/send you email from EC2: -
- Request the remove of your outbound email limit
- Use a good MTA like postfix
- Use DKIM to sign your messages
- Set up SPF DNS records
- Install Antispam and Virus Filters
- Configure TLS and SASL authentication
1. Request removal of out bound email limit.
Once you start sending emails, you are likely to receive an email like this one: -
Dear EC2 Customer,
You recently reached a limit on the volume of email you were able to send out of SMTP port 25 on your instance: Instance ID: i-6832271e
* IP Address: 46.137.8.98
* Start date: 2011-09-02 22:41 +0000
In order to maintain the quality of EC2 addresses for sending email, we enforce default limits on the amount of email that can be sent from EC2 accounts. If you wish to send larger amounts of email from EC2, you can apply to have these limits removed from your account by filling out our online request form.
If you are unaware of your instance having sent emails, we advise checking your instance application(s) to confirm that this activity was intended. It is your responsibility to ensure that your instances and all applications are secured against unauthorized use. For suggestions on securing your instances, visitaws.amazon.com/security.
Regards,
Your Amazon Web Services EC2 team |
To allow your instance to send emails to the outside world you must fill in the online form requesting removal of the limit. Amazon will force you to send your mail from two elastic ip addresses, rather than directly from each instance so you will need to arrange your servers to send email through a designated SMTP relay host. On the plus side, Amazon will set up DNS PTR records for you for these IP addresses so that more sites will regard your mail server/s as legitimate. You should also implement all of the following recommendations in order to keep your emails trusted and out of peoples spam folders!
2. Install Postfix
We require some of the advanced features of postfix - you can use sendmail but it is far more difficult to configure. I'm not going to explain how you install postfix - I assume and strongly recommend you use my postfix profile as this makes it easy to implement all of the steps featured in this guide.
3. Configure DomainKeys Identified Mail (DKIM)
See http://www.howtoforge.com/set-up-dkim-on-postfix-with-dkim-milter-centos...
DKIM allows you to sign your outgoing messages to validate that they have been sent from you and not someone else! It works by creating a pair of keys, a private key only known to your mail servers and a public key which you publish in DNS. When a receiving MTA gets an email from you it can check DNS to see whether it should have a DKIM signature and verify it, if one is present. You can create your DKIM keys by installing the dkim-milter package and running dkim-genkey:-
yum -y install dkim-milter
dkim-genkey -r -d practicalclouds.com
Two files are created. The default.txt file contains the DNS entry you need to add to your domains DNS zone file containing your public key: -
default._domainkey IN TXT "v=DKIM1; g=*; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDHvrMTG2KJhdFfDIj4g8Md3rorTztgLluwd8S0jmk25pJai8+DuIHaT1K6QnKtw1Nz8nWZEr+pfhU76YjxBOgE5Mh9gMxp5sGVs3Bpd5tkqOVy7LAB0x1qEahlGl08opd6RMHn4v6+nTwM9g83fxdgEgceIhkxMlSE8fenP8HnLwIDAQAB" ; ----- DKIM default for practicalclouds.com
The default.private file contains your private key: -
-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQC4NeC1Ujsr0TxFGsd8GQlYkuv6O+Iyaim9HNV30ja9IhxaNV7q
KTFwkzJTNWWD+eW/1imb09p1EHrqbIkGdLZBWusPHDg5CO+bPQYkuOD8x0XYdU6Z
1S9Xg22C+PFJI0Hu7oDBZZ1c1LLiUbs8xgAOnXnvQjYHjvcVXSZbXeo5IQIDAQAB
AoGAPBmn9MCdeZBMf6dbCkwI1pMo1n2PMS+d6otrwiodndtxMKVVEETcUoRT2P4E
3ug7Yzl2Xdbi4Wq9D5rVofuWMnJZiuLZjWOwdpXfdB/ljixEL2lvh/rHRKYQRF+E
l9XchK9JdT2IhPfgY9Ge34Ka/3/o75pX9xtvpw2BLVgKuSECQQDq6wtLIbj8iP87
vTuigII6TpvXOE2FYILuDG9yVazrOUrr2l++0zsRoNtVvsTyIRBdgMITpttfZiAf
2eIeRWx9AkEAyL3jDgYL/onIKAEGyZZ/pzSi9u/ljvX86l/TbTbU9sdDZKH4HMV0
3yRfgp9tQACWVw0tPQcGmUamTTg6yNN0dQJAAjcZMiotPmRkU8HjEBM6JyaanZrw
UDOqyGfhnShlqwotRLAYewou4mmr59EWtlnNd7RBIEJQi0PBRcueMD7jWQJAH7oT
RsYh4yr3nYr7/aRxrIWfNBQaXgv22zwl+F7wLci0mBUZV8LFg/cFuWllJPZyzHNi
sQCJRHRsGExIzjDNeQJBAIRxZkpcLjF+d12oe3zxSk9B4LRD8xwyDFgKO4I7ztOF
e3KXdXBf7lc7Jii5T6VeqroxcUUWSghdW1VzzOh2f7Y=
-----END RSA PRIVATE KEY-----
No, this isn't my real key!
You also need to add two additional DNS entries that declare that all messages from your domain should have DKIM signatures on them :-
_ssp._domainkey IN TXT "dkim=all"
_adsp._domainkey IN TXT "dkim=all"
Now you need to configure postfix to sign your messages with the DKIM signatures. You can do this simply by adding your private key file to your S3 bootbucket and then adding the argument "dkimkey" and specifying the name of the private key file, when you start the postfix profile. It is possible to have multiple different signing keys per domain if you wish but I configure just the one which is called "default". Once configured, you will see DKIM signatures automatically being added to your messages: -
Delivered-To: davidemccormick@gmail.com
Received: by 10.213.2.136 with SMTP id 8cs41089ebj;
Fri, 2 Sep 2011 14:49:53 -0700 (PDT)
Received: by 10.227.197.74 with SMTP id ej10mr1393476wbb.103.1315000193704;
Fri, 02 Sep 2011 14:49:53 -0700 (PDT)
Return-Path: <root@practicalclouds.com>
Received: from practicalclouds.com (ec2-46-137-8-98.eu-west-1.compute.amazonaws.com [46.137.8.98])
by mx.google.com with ESMTP id o3si257772wbh.146.2011.09.02.14.49.52;
Fri, 02 Sep 2011 14:49:52 -0700 (PDT)
Received-SPF: neutral (google.com: 46.137.8.98 is neither permitted nor denied by best guess record for domain of root@practicalclouds.com) client-ip=46.137.8.98;
Authentication-Results: mx.google.com; spf=neutral (google.com: 46.137.8.98 is neither permitted nor denied by best guess record for domain of root@practicalclouds.com) smtp.mail=root@practicalclouds.com; dkim=pass header.i=@practicalclouds.com
Received: by practicalclouds.com (Postfix, from userid 0)
id 1066922267; Fri, 2 Sep 2011 22:49:52 +0100 (BST)
X-DKIM: Sendmail DKIM Filter v2.8.3 practicalclouds.com 1066922267
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=practicalclouds.com;
s=default; t=1315000192;
bh=/qWLUGBjCERM8z31CngA/nq67g+ZePlkPfmTb3fg66Q=;
h=Date:To:Subject:MIME-Version:Content-Type:
Content-Transfer-Encoding:Message-Id:From;
b=FX/qhhw8fk55Msa4ZN6cCpfl4OHuTa7YdCw0KqaJewGN3y9hPrd18h8Z30TbCvMUB
dEQNKForQZzGm5/B4HnHfpsgd1FT0nMXssAj78rQCr/1SQlAyORYxH5lqPYXRDXvew
ZaaYn0Y1Ee6/f4Uv1XpKru0tvfNtAyP694wjgvno=
Date: Fri, 02 Sep 2011 22:49:52 +0100
To: davidemccormick@gmail.com
Subject: you pclouds domain is ready
User-Agent: Heirloom mailx 12.5 7/5/10
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20110902214952.1066922267@practicalclouds.com>
From: root@practicalclouds.com (testmail2)4. Set up SPF (Sender Policy Framework) DNS Records
SPF records instruct receiving MTAs which hosts are allowed to send email on behalf a particular DNS domain. Like DKIM, SPF works because we assume that the company owning a domain are in control of its DNS domain and no one else can change it. The SPF record states specifically which hosts may send mail on behalf of the domain. Create the following SPF record (which is a DNS TXT entry) to state that only hosts with an MX record for your domain (i.e. hosts which receive mail for your domain) are allowed to send email on behalf of it: -
@ IN TXT "v=spf1 mx -all"
You can also optionally allow other specific hosts, such as Googlemail may also send email on behalf of your domain: -
@ IN TXT "v=spf1 mx a:mail.google.com -all"
SPF protection is entirely configured in DNS - you need to make no changes to your postfix configuration.
5. Set up Spam Filtering and Virus Checking
If you are receiving email from other domains, we strongly recommend that you install additional tools which will check it for spam and virus check each message. This can be achieved with postfix by installing amavis, spamassassin and clamav software and integrating it with postfix. You can ask thepostfix profile to automatically install, configure and enable these for you by adding the "antispam" argument which is considerably easier than doing it yourself.
For information on how to do this yourself, try: -
http://fedoraunity.org/solved/server-solutions/postfix-mail-server
http://stevejenkins.com/blog/2011/02/tips-for-installing-amavis-new-clam...
6. Configure TLS and SASL authentication (optional)
It is a good idea to enable SMTP over TLS so that your users or external MTAS can send mails over an encrypted connection to your email server. Also, you can enable SASL authentication so that your server will allow emails from outside if they first authenticate with a valid user name and password. This approach can allow your mobile users to still send via your servers from any location on the Internet and can be used with services such as googlemail which allows you to send email on behalf of another user.
If you don't have a certificate you should probably buy one or generate a self-signed one (for testing): -
cd /etc/postfix
openssl req -new -x509 -nodes -out cert.pem -keyout key.pem -days 3650
chmod 600 *.pem
As part of the postfix profile we alway install certificates and use dovecot to provide SASL. This section in the postgres config file /etc/postfix/main.cf enables SASL via dovecot: -
#TLS - SMTP AUTH
disable_vrfy_command = yes
smtpd_use_tls = yes
smtpd_tls_auth_only = yes
tls_random_source = dev:/dev/urandom
smtpd_tls_cert_file = /etc/postfix/cert.pem
smtpd_tls_key_file = /etc/postfix/pkey.pem
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
# Add some security
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination
I configure Dovecot as below to provide the SASL authentication socket to postfix :-
# 2.0.13: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.35.13-92.fc14.i686.PAE i686 Fedora release 14 (Laughlin) ext4
lda_mailbox_autocreate = yes
mail_location = maildir:/var/spool/mail/%u
mbox_write_locks = fcntl
passdb {
driver = pam
}
protocols = imap pop3
service auth {
unix_listener /var/spool/postfix/private/auth {
mode = 0666
}
}
ssl_cert = </etc/pki/dovecot/certs/dovecot.pem
ssl_key = </etc/pki/dovecot/private/dovecot.pem
userdb {
driver = passwd
} | NOTE: The postfix profile automatically installs dovecot and starts it when it installs postfix, even if you do not enable the POP3 or IMAP4 servers. We use its SASL authentication service with postfix to provide authenticated SMTP. |
7.Example Configs
Here are some example configs, taken from this site and the postfix profile to guide you. These configs are for the practicalclouds.com domain and have all of the options, such as anti-virus checking, enabled.
/etc/posfix/main.cf
queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
mail_owner = postfix
inet_interfaces = all
inet_protocols = all
mydomain=practicalclouds.com
myhostname=mail.practicalclouds.com
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
mynetworks_style=host
unknown_local_recipient_reject_code = 550
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
debug_peer_level = 2
debugger_command =
PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
ddd $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail.postfix
newaliases_path = /usr/bin/newaliases.postfix
mailq_path = /usr/bin/mailq.postfix
setgid_group = postdrop
html_directory = no
manpage_directory = /usr/share/man
sample_directory = /usr/share/doc/postfix-2.7.5/samples
readme_directory = /usr/share/doc/postfix-2.7.5/README_FILES
mailbox_command = /usr/libexec/dovecot/deliver
#TLS - SMTP AUTH
disable_vrfy_command = yes
smtpd_use_tls = yes
smtpd_tls_auth_only = yes
tls_random_source = dev:/dev/urandom
smtpd_tls_cert_file = /etc/postfix/cert.pem
smtpd_tls_key_file = /etc/postfix/pkey.pem
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
broken_sasl_auth_clients = yes
# Add some security
smtpd_recipient_restrictions = permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination
smtpd_milters = inet:localhost:20209
non_smtpd_milters = inet:localhost:20209
content_filter = smtp-amavis:[127.0.0.1]:10024
/etc/postfix/master.cf
#
# Postfix master process configuration file. For details on the format
# of the file, see the master(5) manual page (command: "man 5 master").
#
# Do not forget to execute "postfix reload" after editing this file.
#
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
smtp inet n - n - - smtpd
#smtp inet n - n - 1 postscreen
#smtpd pass - - n - - smtpd
#dnsblog unix - - n - 0 dnsblog
#tlsproxy unix - - n - 0 tlsproxy
#submission inet n - n - - smtpd
# -o smtpd_tls_security_level=encrypt
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticated,reject
# -o milter_macro_daemon_name=ORIGINATING
#smtps inet n - n - - smtpd
# -o smtpd_tls_wrappermode=yes
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticated,reject
# -o milter_macro_daemon_name=ORIGINATING
#628 inet n - n - - qmqpd
pickup fifo n - n 60 1 pickup
cleanup unix n - n - 0 cleanup
qmgr fifo n - n 300 1 qmgr
#qmgr fifo n - n 300 1 oqmgr
tlsmgr unix - - n 1000? 1 tlsmgr
rewrite unix - - n - - trivial-rewrite
bounce unix - - n - 0 bounce
defer unix - - n - 0 bounce
trace unix - - n - 0 bounce
verify unix - - n - 1 verify
flush unix n - n 1000? 0 flush
proxymap unix - - n - - proxymap
proxywrite unix - - n - 1 proxymap
smtp unix - - n - - smtp
# When relaying mail as backup MX, disable fallback_relay to avoid MX loops
relay unix - - n - - smtp
-o smtp_fallback_relay=
# -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq unix n - n - - showq
error unix - - n - - error
retry unix - - n - - error
discard unix - - n - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - n - - lmtp
anvil unix - - n - 1 anvil
scache unix - - n - 1 scache
#
# ====================================================================
# Interfaces to non-Postfix software. Be sure to examine the manual
# pages of the non-Postfix software to find out what options it wants.
#
# Many of the following services use the Postfix pipe(8) delivery
# agent. See the pipe(8) man page for information about ${recipient}
# and other message envelope options.
# ====================================================================
#
# maildrop. See the Postfix MAILDROP_README file for details.
# Also specify in main.cf: maildrop_destination_recipient_limit=1
#
#maildrop unix - n n - - pipe
# flags=DRhu user=vmail argv=/usr/local/bin/maildrop -d ${recipient}
#
# ====================================================================
#
# Recent Cyrus versions can use the existing "lmtp" master.cf entry.
#
# Specify in cyrus.conf:
# lmtp cmd="lmtpd -a" listen="localhost:lmtp" proto=tcp4
#
# Specify in main.cf one or more of the following:
# mailbox_transport = lmtp:inet:localhost
# virtual_transport = lmtp:inet:localhost
#
# ====================================================================
#
# Cyrus 2.1.5 (Amos Gouaux)
# Also specify in main.cf: cyrus_destination_recipient_limit=1
#
#cyrus unix - n n - - pipe
# user=cyrus argv=/usr/lib/cyrus-imapd/deliver -e -r ${sender} -m ${extension} ${user}
#
# ====================================================================
#
# Old example of delivery via Cyrus.
#
#old-cyrus unix - n n - - pipe
# flags=R user=cyrus argv=/usr/lib/cyrus-imapd/deliver -e -m ${extension} ${user}
#
# ====================================================================
#
# See the Postfix UUCP_README file for configuration details.
#
#uucp unix - n n - - pipe
# flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
#
# ====================================================================
#
# Other external delivery methods.
#
#ifmail unix - n n - - pipe
# flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
#
#bsmtp unix - n n - - pipe
# flags=Fq. user=bsmtp argv=/usr/local/sbin/bsmtp -f $sender $nexthop $recipient
#
#scalemail-backend unix - n n - 2 pipe
# flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store
# ${nexthop} ${user} ${extension}
#
#mailman unix - n n - - pipe
# flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
# ${nexthop} ${user}
smtp-amavis unix - - n - 2 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20
127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters
Hope this helps you
