arithmetric / aws-lambda-ses-forwarder Goto Github PK

Have you considered bigger scale events and bursts? Let's say, SES received 5000 messages just now - your script will try to invoke 5000 Lambdas and send out 5000 emails which will break two limits at once - Lambdas limit for concurrent runs and SES send-out rate-limit.

It would be nice to have a pacing mechanism. Or at least support the dead-letter queue (not ideal).

For instance, one possibility could be: when SES email is received, it's saved in S3 and an SNS event is broadcasted (no Lambda is invoked). An SQS queue is subscribed to this SNS topic so that there is always a queue of emails that can be processed synchronously/later on.

Then, Lambda is invoked by CloudWatch Schedule instead, let's say one once every minute. Maximum execution time is set to 1 minute so that it can run always, if necessary. This shouldn't make it more expensive than current model - AWS charges for total, sum time, not invocations.

Then, this smart Lambda function checks how much in trouble the queue is (how many queued messages there are) and decides the number of outgoing SES threads based on that. The number will never go higher than current SES send-rate (could be a part of the config).

So your current send-rate is 30 emails per second, Lambda function sees 1000 messages in the queue - it will run 30 threads to send out emails until AWS kills it after 1 minute. Then, it will be invoked again.

If queue is empty, Lambda will die gracefully, it won't idle for 60 seconds. Or it could poll lazily - every 5-10 seconds.

Curious if you had potentially thought about being able to forward directly via LMTP or SMTP to a final mail server rather than requiring the modification of the From address and sending through SES.

Hi,
I'm having some trouble in forwarding email.
Every time when mail forwarding is done, these kind of codes are included in the mail.
Is there any way to get rid of these codes?

X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
bh=VcUwljfBB3Ih9IZnGfiyRq86HN5fYES5CKp76GqkLsE=;
b=Kg02Ur+JrglkIC/oKiheGqdcAQ5YnKq0PYHLA7yCFgTi5AoNkDLrlR9vx96F85hWgh
5gih0q1GySuWWrW8MljCg5gtqdu0SAWIhfij2GR5DiFXE1L0NQL/9MAEm+U3qHiNsXjP
6y2f2BMFNl9o1JXKRYq4wv4XMhJXgC6lNRsI/q1Wynm7zxXXnYrPW9zkgA5LKs0afQHw
aupdrBZmTTa8VcFBBu8t51s9aTqiAwpZSDC/iT9XJ9UjPGutnI7Mr+TUXXWXEnj4QlZG
hj5RrMTOb1v/UpTPNbt4jfPAOXz30A66TeoaOX/yExLYrouwwy4Ss3IXXbFQ25xB/bjr
gSUw==

--94eb2c0b8d5ef56d5d05528a0878
Content-Type: text/plain; charset="UTF-8"

This policy contains the following error: Has prohibited field Principal For more information about the IAM policy grammar, see AWS IAM Policies.

I was interested in implementing this. I got quite far, but when I attempt to create the example policy listed I see the error listed above.

Any advise?

I am attempting to use this function, and it appears to be executing when I send an email to my verified address, as it is logging an error. However, I'm not sure how to fix this.

SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method.

Where do I enter the signature in my settings or config? Presumably this is something I need to do in Lambda, or SES, or in IAM, or where? I do not see any config in this particular script.

Let's say yourdomain.com is the domain the forwarder is sending emails from, and anotherdomain.com is another domain. Consider these cases:

1. [email protected] sends a mail to [email protected]. The mail is forwarded to recipients correctly without any checks failed;
2. [email protected] sends a mail to [email protected]. The mail ends up with a DKIM(on SES domain) check failure.

After looking at the DKIM-Signature part I found it's weird that the failed one has TWO Feedback-ID's in the h field. I guess it might be the cause and tried to fix the forwarding code to remove the field from the header, and it works.

But I don't know what this field means and what side-effects the removal would introduce.

Here's excerpt from the message fails DKIM validation:

Authentication-Results: mx.google.com;
       dkim=fail [email protected];
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
    s=gdwg2y3kokkkj5a55z2ilkup5wp5hhxx; d=amazonses.com; t=1470162493;
    h=Message-ID:Date:Subject:From:Reply-To:To:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID:Feedback-ID;
    bh=cIc4+mNxsFp+dUwkGlbCwYelaS0phVkPmIICqAGQvPs=;
    b=oQRKCP7Qz2BZ7rNOt64kJZLMQiAw85afNkIS0ZBezqdObXAfEINMPtrG0K+oZps2
    CmPVoLW8UkSkFU4bTtaiaXYw0lsmAjXvpsipQyYQFPYN1MfEWIFQZBxTEJJEVnozkyb
    M31UJ95rnkf437YCCPyFvSMsYHdmhlceK++Fk6QY=

And here is a success one after I removed the field:

Authentication-Results: mx.google.com;
       dkim=pass [email protected];
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple;
    s=gdwg2y3kokkkj5a55z2ilkup5wp5hhxx; d=amazonses.com; t=1470166761;
    h=Date:From:To:Subject:Content-Type:MIME-Version:Message-ID:Reply-To:Feedback-ID;
    bh=EEznD2tyqczD8Zpld/7JgAHSIj9dxh/xhszpF4qOYyQ=;
    b=cMic0Xdgjvkrr5mgutcRyVmxDw2apNOWABeTRx41q9PqrTEymFATlZObJ2kq9Bbz
    X27rFRkfkT+XN6z4qwzvrXXG6VSDXnfZ4B9aM4/9ntQ9wyfjcyZXHaZlbr9i98L2qLY
    cQqvz2ET1Dg7FOsOh6tCUXEMTaCbpE0xo598cN/o=

How much do you typically get charged by using this method?

I guess a more static way of doing things would be to use G Suit, then just forward emails through there.

Any thoughts?

Awesome work, there is only one thing that I would like to see different. If I am sending a mail from, say, a PHP app to a user, let's say a notification, I want this script to ignore this email and not change it and just let it pass through unharmed. If it comes from another domain not in my list of verified domains then it should do its thang.

Forgive me if there is already a way of doing this. If not then this is something I would love to see.

Amazon SES refuses to send email if there are duplicate DKIM-Signature headers.

See the following error message from CloudWatch logs:

InvalidParameterValue: Duplicate header 'DKIM-Signature'.] message: 'Duplicate header \'DKIM-Signature\'.', code: 'InvalidParameterValue', time: Sat Mar 05 2016 20:19:22 GMT+0000 (UTC), requestId: '8c87c2e0-e30f-1234-a6f4-616802c8702d', statusCode: 400, retryable: false, retryDelay: 30 }

I double-checked all the permissions according to readme, but still get the error:

START RequestId: af224ff3-fb47-11e6-9444-1f9e512246cc Version: $LATEST
2017-02-25T10:46:38.924Z	af224ff3-fb47-11e6-9444-1f9e512246cc	{ level: 'info',
  message: 'Fetching email at s3://musweb.mail/o3vrnil0e2ic28trm7dfhrc2v0clambda4nbp0g1' }
2017-02-25T10:46:40.344Z	af224ff3-fb47-11e6-9444-1f9e512246cc	{ level: 'error',
  message: 'copyObject() returned error:',
  error: 
   { [AccessDenied: Access Denied]
     message: 'Access Denied',
     code: 'AccessDenied',
     region: null,
     time: Sat Feb 25 2017 10:46:40 GMT+0000 (UTC),
     requestId: '4CC93788EF0116A6',
     extendedRequestId: 'nGtVxfT3aT/uqPpufrdvjJrLy/ClZ4WeJoqh8AK9eXy5keqqx3Q89SSnkrvIYyxnSS31STpGF00=',
     cfId: undefined,
     statusCode: 403,
     retryable: false,
     retryDelay: 50.05418912041932 },
  stack: 'AccessDenied: Access Denied
    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/s3.js:538:35)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:671:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:673:12)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)' }
2017-02-25T10:46:40.404Z	af224ff3-fb47-11e6-9444-1f9e512246cc	{ level: 'error',
  message: 'Step returned error: Error: Could not make readable copy of email.',
  error: [Error: Error: Could not make readable copy of email.],
  stack: 'Error: Error: Could not make readable copy of email.
    at Response.<anonymous> (/var/task/index.js:145:11)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:358:18)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:671:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:673:12)' }
2017-02-25T10:46:40.405Z	af224ff3-fb47-11e6-9444-1f9e512246cc	{"errorMessage":"Error: Step returned error.","errorType":"Error","stackTrace":["/var/task/index.js:302:28","process._tickDomainCallback (node.js:407:9)"]}
END RequestId: af224ff3-fb47-11e6-9444-1f9e512246cc
REPORT RequestId: af224ff3-fb47-11e6-9444-1f9e512246cc	Duration: 1942.54 ms	Billed Duration: 2000 ms 

Hi!

This is not an issue, but an educational question. I wonder why did you not set up this lambda with an SNS trigger (rule with 1 action: write to S3 + SNS topic) instead of rule with 2 actions: write to S3, trigger lambda?

Thanks!

I have followed exactly what you've spelt out on https://github.com/arithmetric/aws-lambda-ses-forwarder but am stuck. No amount of Google search or No playing with options either has helped. A client is about to terminate the contract - we've stuck at this error for too long.

I have verified my domain. I receive email from anywhere through my verified address to my designated S3 bucket. I also changed the '"functionArn":xxx..."' to '"functionArn": "arn:aws:lambda:us-west-2:MY-ACCOUNT-ID:function:fnForwardEmailToGmail"'.

However the received emails are never forwarded and when I test the function using "SES Email Receiving" I get the error below.

START RequestId: 7f2cd5ed-83ea-11e7-913f-55748388c69f Version: $LATEST
2017-08-18T07:54:44.467Z 7f2cd5ed-83ea-11e7-913f-55748388c69f { level: 'info',
message: 'Fetching email at s3://MY-S3-BUCKET/MY-PREFIX/o3vrnil0e2ic28fgfdstrm7dfhrc2v0clambda4nbp0g1' }
2017-08-18T07:54:46.068Z 7f2cd5ed-83ea-11e7-913f-55748388c69f { level: 'error',
message: 'copyObject() returned error:',
error:
{ [AccessDenied: Access Denied]
message: 'Access Denied',
code: 'AccessDenied',
region: null,
time: Fri Aug 18 2017 07:54:46 GMT+0000 (UTC),
requestId: 'A6285517D1AF2B9D',
extendedRequestId: 'dfH3csS5kHLsYN4ZgIWVliYmuVb1OgCVl6KdUSdZdqwX2T+JdkfZwIyPa5KEgYFiJfZmrwXjXDI=',
cfId: undefined,
statusCode: 403,
retryable: false,
retryDelay: 32.49475641641766 },
stack: 'AccessDenied: Access Denied
at Request.extractError (/var/task/node_modules/aws-sdk/lib/services/s3.js:473:35)
at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:615:14)
at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/var/task/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/var/task/node_modules/aws-sdk/lib/request.js:617:12)
at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:115:18)' }
2017-08-18T07:54:46.127Z 7f2cd5ed-83ea-11e7-913f-55748388c69f {"errorMessage":"Error: Could not make readable copy of email."}
END RequestId: 7f2cd5ed-83ea-11e7-913f-55748388c69f
REPORT RequestId: 7f2cd5ed-83ea-11e7-913f-55748388c69f Duration: 1993.85 ms Billed Duration: 2000 ms Memory Size: 128 MB Max Memory Used: 32 MB

LAMBDA ROLE FUNCTION POLICY
{
"Version": "2016-03-04",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:::"
},
{
"Effect": "Allow",
"Action": "ses:SendRawEmail",
"Resource": ""
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": "arn:aws:s3:::MY-S3-BUCKET/*"
}
]
}

S3 BUCKET POLICY
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "GiveSESPermissionToWriteEmail",
"Effect": "Allow",
"Principal": {
"Service": "ses.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::MY-S3-BUCKET/*",
"Condition": {
"StringEquals": {
"aws:Referer": "MY-ACCOUNT-ID"
}
}
}
]
}

Hi there. If you're interested, I wrote a Serverless (a framework for managing the lifecycle of serverless apps) Service that wrappers your project. Currently it automates the provisioning of the Lambda function, IAM role, bucket policy, and S3 bucket creation (basically, steps 2 & 7 in your instructions.

https://github.com/tonycapone/serverless-ses-forwarder

Let me know what you think.

Thanks.

hi,

i get that issue. thanks.

{
"errorMessage": "Error: Step returned error.",
"errorType": "Error",
"stackTrace": [
"/var/task/index.js:317:28"
]
}

Function might fail if S3 bucket is stored in a EU region. CloudWatch will record an error saying "Please use AWS4-HMAC-SHA256' you need to use Signature Version 4 when initializing the conntection to the S3 bucket". Can be easily fixed by changing line 275 from AWS.S3() to AWS.S3({signatureVersion: 'v4'}).

When I send an e-mail to an assigned address it gets forwarded now (after the Return-Path merge), but all original email header is put into the body of the forwarded mail, so it also loses all properties like subject and message ID.

Any idea what might be causing this? I can forward an example if needed.

Getting this error:

017-12-01T13:23:19.809Z cbb36a51-d69a-11e7-966a-e777dc99be75 { level: 'error',
message: 'Step returned error: Cannot read property 'log' of undefined',
error:
TypeError: Cannot read property 'log' of undefined
at exports.fetchMessage (/var/task/index.js:127:7),
stack: 'TypeError: Cannot read property 'log' of undefined\n at exports.fetchMessage (/var/task/index.js:127:7)' }

Got it all set up nicely for multiple domains, just have one issue.

Some clients (Gmail) are showing messages up as from myself.
I think I have it configured correctly but suspect it is DKIM signing.
Because we're processing the email and using SES to send it on, it's being signed by my AWS identity.

Any way to stop this?

I presume the original mail is being signed by the original sender, but it will be stripped out when SES downloads it to the S3 bucket?

e.g.
From line says: "Other person at [email protected] < [email protected] > "
Reply-to : "[email protected] [email protected]"
Signed-by: my.domain.com

When I receive an email, it is marked as spam (outlook.com).

However, if I specify in the inbound rule that mails are encrypted on S3, it is not marked. But you can't read the mail anymore, because it is encrypted.

Anybody know what I can do about it?

Would you support a PR to add a domain mapping feature to forward all mail from one domain to another with the same username? Configuration might be like:

{
    config: {
      fromEmail: "[email protected]",
      emailBucket: "s3-bucket-name",
      emailKeyPrefix: "emailsPrefix/",
      forwardMapping: {
        "[email protected]": [
          "[email protected]",
          "[email protected]"
        ],
        "[email protected]": [
          "[email protected]"
        ]
      },
      domainMapping: {
        "@foo.example.com": "@example.com"
      }
    }
  }

So [email protected] would forward to [email protected]. The forwardMapping would still take precedent, but if there aren't any matches for forwardMapping, it would check the domainMapping.

Thoughts?

I've currently done this by overriding the transformRecipients function in my own module, but it's not as succinct as it would be integrated directly. I'm happy to provide tests, implementation, and documentation.

Thanks!

Nick

If the Lambda function is invoked with an e-mail that has more than 50 original recipients, it will create an error in SES.

Example: User is part of an e-mail with 100 different addresses. The forwarded e-mail should only be sent to the user's forward address (1 recipient) but instead an error is thrown in SES for attempting to send an e-mail with more than 50 recipients.

I'll update this if I can configure a fix or find the logs

It looks like you addressed a similar issue in #8, but my use-case is a little different.

I use a google apps account with an alias domain so I can send mail from two different addresses, although both are "mailed by" the first:

• [email protected]
• [email protected]

Both domains have DKIM configured, so when I send email from the alias domain, two DKIM-Signature headers are included; one for each domain. This works fine when sending mail to most places, but it fails when being processed by this lambda forwarder. I get this error:

{
    level : 'error',
    message : 'sendRawEmail() returned error.',
    error :
    {
        [InvalidParameterValue : Duplicate header 'DKIM-Signature'.]message : 'Duplicate header \'DKIM-Signature\'.',
        code : 'InvalidParameterValue',
        time : Fri Mar 25 2016 02 : 50 : 53 GMT + 0000(UTC),
        requestId : '64105019-f234-11e5-a148-3d2a02a5aa36',
        statusCode : 400,
        retryable : false,
        retryDelay : 80.16784060746431
    },
    stack : 'InvalidParameterValue: Duplicate header \'DKIM-Signature\'.\n at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/query.js:40:29)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:596:14)\n at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:21:10)\n at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:37:9)\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:598:12)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)'
}

I suspect this issue is actually with SES, but I wanted to document it here, so people at least know that it will affect their use of this project.

I've got this working successfully but wanted to track my bounces and errors via CloudWatch.

In order to do this it seems like I need to add a custom header (X-SES-CONFIGURATION-SET) to my forwarded emails. http://docs.aws.amazon.com/ses/latest/DeveloperGuide/event-publishing-send-email.html

I've tried doing this but I'm some how mutilating the header and it's throwing this error:
message: 'sendRawEmail() returned error.', error: { [InvalidParameterValue: Duplicate header 'Content-Type'.] message: 'Duplicate header \'Content-Type\'.', code: 'InvalidParameterValue',

I inserted the following code into the process.message function, at the very end:
header = header + 'X-SES-CONFIGURATION-SET: cloudwatch_SES_destination\r'; data.log({level: "info", message: "Added Configuration Group Tag"});

it would be great if the emails stored in s3 could be encrypted.

I am not able to get any messages to send using the aws-lambda-ses-forwarder script.

The forwarder basically works but I don't know what happens if the lambda fails. It'd be great to get some alert.

Hello
I am using your lambda forwarder to route [email protected], [email protected] etc to the appropriate people. But the original senders email address is not being saved as a reply to address.
eg
[email protected] sends email to [email protected]
I receive email and the reply to address is configured as [email protected], instead of [email protected].?

A small bug encountered when emailing back and forth with replies via the SES forwarder script in that successive subjectPrefixes can accumulate on each reply received.

Following code corrects, feel free to add:

  // Add a prefix to the Subject
  if (data.config.subjectPrefix) {
    header = header.replace(/^Subject: (.*)/mg,
      function(match, subject) {
        if (subject.indexOf(data.config.subjectPrefix) < 0) {
            return 'Subject: ' + data.config.subjectPrefix + subject;
        } else {
            return 'Subject: ' + subject;
        }
        
      });
  }

I get InvalidParameterValue: Extra route-addr when forwarding an email. Not sure what causes this, but I noticed that the email that fails has charset=windows-1252, and the one that succeeds has charset=UTF-8.

Fails:

...
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="------------F2C3840841699E1CBA46489D"

This is a multi-part message in MIME format.
--------------F2C3840841699E1CBA46489D
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 8bit

some text

--------------F2C3840841699E1CBA46489D
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit

<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=windows-1252">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <p><font face="Helvetica, Arial, sans-serif">no sign</font><br>
    </p>
    <br>
    <div class="moz-signature">-- <br>
      <br>
      <b>John Doe</b><br>
      <a href="https://example.com">Example</a><br>
    </div>
  </body>
</html>

--------------F2C3840841699E1CBA46489D--

Doesn't fail:

...
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary=94eb2c18fe78175025054b289093

--94eb2c18fe78175025054b289093
Content-Type: text/plain; charset=UTF-8

Hello

--94eb2c18fe78175025054b289093
Content-Type: text/html; charset=UTF-8

<div dir="auto">Hello</div>

--94eb2c18fe78175025054b289093--

As the code currently stands emails sent to [email protected] won't be forwarded to [email protected] and [email protected]

I managed to alter the code to get it working but I've never submitted code to github, nor am I particularly proficient at JavaScript so I'll raise this issue and leave it to someone more competent to fix.

FYI - i used the .tolowercase() function in lines 66,68 and 69

Why the script is changing From header to own address?
Are there some reason why it is doing it?
How to turn it off?

I've been using this project for a while and it's awesome, thanks a lot! I stumbled upon an error for the first time today. Error log from CloudWatch is below:

2017-05-03T11:38:51.802Z	f2ab5592-2ff4-11e7-97ca-a1db4277fedb	{ level: 'error',
message: 'sendRawEmail() returned error.',
error: 
{ [InvalidParameterValue: Duplicate header 'Message-ID'.]
message: 'Duplicate header \'Message-ID\'.',
code: 'InvalidParameterValue',
time: Wed May 03 2017 11:38:51 GMT+0000 (UTC),
requestId: '14956f7e-2ff5-11e7-8290-6fdc1cb44eda',
statusCode: 400,
retryable: false,
retryDelay: 16.828257404267788 },
stack: 'InvalidParameterValue: Duplicate header \'Message-ID\'.\n at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/protocol/query.js:47:29)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:673:14)\n at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)\n at Request.<anonymous> (/var/runtime/node_modules/aws-sdk/lib/request.js:675:12)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)' }

Repro step: create a Skype account using [email protected]. The email containing the verification code fails to be forwarded.

https://habd.as/serverless-email-forwards-ses-lambda-crash-course/

I've come across a weird situation where occasionally when an email is sent to the domain where I have the redirect set up, the redirected to domain gets five identical copies of the email.

It looks like lambda is fired off five times one after another but the execution of the lambdas still occur in parallel. I.e. the lambda doesn't happen back to back but all at once.

Has anyone else come across this problem before? I'm still in the process of debugging the cause so any help would be appreciated.

Am I able to do the below with this script?

Email comes into a verified email inbox called '[email protected]'.

Forward this email to...

To: New person [email protected]
From: [email protected]

Thanks!

In Lambda function logs I see:

START RequestId: 12ea793b-9ef2-11e7-b1ef-b360a5b0eab4 Version: $LATEST
Unable to import module 'index': Error at Module.require (module.js:353:17)
END RequestId: 12ea793b-9ef2-11e7-b1ef-b360a5b0eab4
REPORT RequestId: 12ea793b-9ef2-11e7-b1ef-b360a5b0eab4	Duration: 1.88 ms	Billed Duration: 100 ms Memory Size: 128 MB	Max Memory Used: 21 MB

What does it mean?

Thank you to the creators of this script! This is a terrific script and I've been using it for over a month now. It's been working great.

One curious issue I've come across recently is due to people with 'Gmail autoreplies' set up. Seems like between Gmail and Amazon, the original user's from address is getting stripped before it is even seen by this script. This results in undesired behavior:

1. Email is sent to Gmail user (FROM [email protected]), with auto reply configured
2. Auto-reply sends email back to ([email protected]), however, From address is listed as [email protected].

From header reports amazonses.com (??)

From: [email protected]

To header reports the same email address as the one sent.

To: (same as sender address)

Possibly-related MIME headers:

Precedence: bulk
X-Autoreply: yes
Auto-Submitted: auto-replied

At any rate, I can think of a few ways to deal with this:

• Look for and ignore any mail with X-Autoreply: yes header set
• Ignore any email with FROM: [email protected]

General thoughts?

I think this line should use \r\n instead of just '\n`:

as per: https://tools.ietf.org/html/rfc5321#section-2.3.8

Currently it depends on storing incoming e-mails to S3. Why it don't fetch e-mails from event, so we don't need S3?

I appear to get this error when there is no address found in the forward mapping object. Is that by design?

Getting this error in CloudWatch logs. Best I can tell it's because data.event.Records[0].eventVersion is coming through as "2.0" and index.js requires that it's "1.0". Do I need to change my AWS setup somewhere to force 1.0, or do I change index.js to allow this through?

Thanks!

Hi Joe

I am interested in using your code to create a store and forward backup MX on AWS.

As far as I can tell, this would require changing the code to forward to the original addressee. This would appear trivial.

The major difference I think is that the forward would need to be triggered by a poll on the main MX showing it is up and then looping through all emails in the S3 bucket rather than just forwarding the one email in the current code.

I think the store and forward requirement is typical. There may be other code that does this.

If not I'm prepared to contribute to beer money...

There seems to be no filter using the SES verdicts. I would like to be able not to forward SPAM or VIRUS verdicts, an/or mark forwarded mails with a [SPAM], [VIRUS], [SUSPICIOUS] in the subject.

I can write some code for this, but I'm no NodeJS expert.

SES doc: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/receiving-email-notifications-contents.html#receiving-email-notifications-contents-receipt-object

Sending mail from the primary domain of my Google Apps account to this forwarder works, but causes DKIM to fail when received by a Gmail account the email was forwarded to. The reason it fails is because Google Apps originally signed the email's using the From header as one of it's inputs, but that value is modified by this forwarder, so verifying that signature fails on the final receiving end.

I haven't tested, but if the sender had dmarc setup to reject non-aligned messages, then I suspect this email would be dropped/bounced by the receiving mail server. Even without using dmark, having a failed dkim header could potentially introduce delivery issues.

Since the "From" address must be a verified address, as per SES, this may just be a limitation of using SES/Lambda as a mail forwarder.

I configured this for my aws account. The problem that I'm facing is that when I sent an email to my domain.com email, I got 2 of the same emails in my inbox.

When I had a subject prefix added, I received 1 email without subject prefix and 1 with the subject prefix.

Any idea why would this happen?

Hi Guys,

I´ve spent a few hours double checking 99% of the settings according to setup yet I still cannot get over this error message. Any tips?

{ level: 'error',
message: 'Step returned error: Error: Received invalid SES message.',
error: [Error: Error: Received invalid SES message.],
stack: 'Error: Error: Received invalid SES message.\n at exports.parseEvent (/var/task/index.js:59:27)' }

level: 'error',
message: 'Step returned error: Error: Could not make readable copy of email.',
error: [Error: Error: Could not make readable copy of email.],
stack: 'Error: Error: Could not make readable copy of email.\n at Response. (/var/task/index.js:144:11)\n at Request.

Hi there,

Your script only forwards if the email is sent FROM an already verified address.

I'm not running in a sandbox, but SES always seems to think the from address is the original from address when forwarding, and it fails with "Email address is not verified".

If I verify the sender's address in SES, then the script works, but obviously I cannot verify everyone who is going to send me email.

I tried removing the From: original via Recipient in lieu of just From: recipient, but it still fails. Maybe because of the Reply-To, or maybe because of the Return-Path? Not sure.

There was a period of time when the email forwarding lambda was not working so our email account (in gmail) did not receive the emails via AWS, but the files for these emails are stored in our s3 bucket.

Right now, only new emails are being forwarded. How can I export / forward all the contents of the s3 bucket which contains those historical emails that weren't received by our email client (gmail)?

Thanks