mozilla / restmail.net

a REST only mail server for ephemeral email addresses

Email addresses for testing

Have you ever wanted to write an automated test of a service that sends email? If you have, you might have wanted an email address that you can check using a simple REST service that returns JSON...

..that's what restmail.net is.

overview

restmail.net is an email server with a REST API. Any time an email is sent to a @restmail.net address, the email address springs into existence and the message is stored (with a max of 10 emails per "user"). You can query the restmail API to check the email of any user, and you can also delete outstanding messages via the API.

This lets you test services that deliver email.

By default, emails are deleted after one day. You can change this by modifying the expireAfter parameter in config.js. Set it to 0 for no auto-deletion, in which case you will want to keep an eye on your redis database size.

security?

All mail sent to restmail email addresses is completely public. anyone may delete the messages associated with any user. This service is wide open. If that won't do, you should fork and deploy your own instance of restmail, and add sekrets and stuff.

the API

DELETE /mail/<user>

Delete all of the mail for the named user (where user is the user portion of the email address, not including @restmail.net).

Returns 200 on success.

GET /mail/<user>

Returns all mail for the specified user, as an array of JSON blobs, with the newest messages first. Here's example output (with lots of fields stripped out):

[
  {
    "headers": {
      "date": "Fri, 11 May 2012 14:44:37 -0600",
      "mime-version": "1.0 (Apple Message framework v1257)"
    },
    "from": [
      {
        "address": "[email protected]",
        "name": "Lloyd Hilaiel"
      }
    ],
    "to": [
      {
        "address": "[email protected]",
        "name": ""
      }
    ],
    "subject": "this is my first message",
    "text": "it is very pretty.\n"
  },
  {
    "headers": {
      "date": "Fri, 11 May 2012 14:44:52 -0600",
      "mime-version": "1.0 (Apple Message framework v1257)"
    },
    "from": [
      {
        "address": "[email protected]",
        "name": "Lloyd Hilaiel"
      }
    ],
    "to": [
      {
        "address": "[email protected]",
        "name": ""
      }
    ],
    "subject": "this is my second message",
    "text": "it's pretty awesome too.\n"
  }
]

restmail on your domain (deprecated; the public restmail.net is now resricted).

You can use restmail from custom domains: Just set restmail as your mail exchanger:

$ dig -t <my domain>
<my domain>            900     IN      MX      20 restmail.net.

And fetch mail with the full email address:

GET /mail/<user>@<my domain>

done!

restmail.net's Issues

Case sensitivity

The restmail server doesn't handle case very well.

Replicate:

1. send an email to [email protected]
2. get: http://restmail.net/casetest
3. Returned json object reads "Cannot GET /casetest"
4. get: http://restmail.net/caseTest
5. Returned json object reads "Cannot GET /caseTest"

restmail.net returns a 503

restmail appears to be down.

Failed to load resource: the server responded with a status of 503 (Service Unavailable: Back-end server is at capacity)

Update website to link the mozilla/restmail.net, not lloyd/restmail.net

Commit be99ba9 updates the "fork me on github" link to point to the correct repo. Please push it live.

(Sorry, I thought github's edit-this-file-online thingy would let me create a PR, but instead it committed my change straight onto master. Last time I try a shortcut like that.)

restmail.net SSL cert has expired

The SSL certificate for restmail.net expired on 2013-06-11.

Obviously, a site where anyone can read anything doesn't need SSL, so open question as to whether it's useful to have a valid SSL certificate. So this bug is either: 1) get a new certificate, or 2) close this bug as documentation that "yeah, the cert is expired".

Personally, I prefer (1). (OR just stop listening on :443 if we're not providing a valid certificate).

Travis CI free usage ends Dec 3; mozilla repos should switch to other CI platforms

We're opening this issue because your project has used Travis CI within the last 6 months. If you have already migrated off it, you can close and ignore this issue.

Travis CI is ending free builds on public repositories. travis-ci.com stopped providingthem in early November, and travis-ci.org will stop after December 31, 2020. To avoid disruptions to your workflows, you must migrate to another CI service.

For production use cases, we recommend switching to CircleCI. This service is already widely used within Mozilla. There is a guide to migrating from Travis CI to CircleCI available here.

For non production use cases, we recommend either CircleCI or Github Actions. There is a guide to migrating from Travis CI to Github Actions available here. Github Actions usage within Mozilla is new, and you will have to work with our github administrators to enable specific actions following this process.

If you have any questions, reach out in #github-admin:mozilla.org on matrix.

Seeing Restmail timeouts after 60 seconds, in mozillians-tests builds

@jrgm mind taking a look at this?

/cc @karlht @davehunt @jrbenny35

We're still seeing some timeouts in our test infra[0], like so:

=================================== FAILURES ===================================
________________ TestProfile.test_profile_deletion_confirmation ________________
[gw0] linux2 -- Python 2.7.12 /usr/bin/python2.7
tests/test_profile.py:22: in test_profile_deletion_confirmation
    home_page.login(vouched_user['email'])
pages/base.py:58: in login
    login_link = conftest.login_link(email)
tests/conftest.py:66: in login_link
    mail = restmail.get_mail(username)
tests/restmail.py:33: in get_mail
    'ACTUAL_MESSAGE_COUNT': len(restmail)})
E   Exception: Timeout after 60 seconds getting restmail for a951a10a-8ecd-48bd-998b-70bc5a68e14e. Expected 1 messages but there were 0.
------------------------------- pytest-selenium --------------------------------
URL: https://auth-dev.mozilla.auth0.com/login?state=v6wjjHl8zN_zAcPCPG6N21RuK_Qa-2tZ&client=T2tB7Ss8It7PKrw3ijazoXu9PgZniLPD&protocol=oauth2&nonce=JrrDkOV6j6XMAu8ZBqjixnCny6u48biP&redirect_uri=https%3A%2F%2Fweb-mozillians-staging.production.paas.mozilla.community%2Foidc%2Fcallback%2F&response_type=code&scope=openid%20email%20profile
WARNING: Failed to gather log types: Message: GET /session/XXXXXXXX-XXXX-XXXX-XXXX-XXXX8a174a14/log/types did not match a known command
Build info: version: '3.4.0', revision: 'unknown', time: 'unknown'
System info: host: 'WIN-SB3ER6JQ6ME', ip: '10.100.24.247', os.name: 'Windows 10', os.arch: 'x86', os.version: '10.0', java.version: '1.8.0_91'
Driver info: org.openqa.selenium.firefox.FirefoxDriver
Capabilities [{moz:profile=C:\Users\Administrator\AppData\Local\Temp\rust_mozprofile.81Nt4SQZfqRs, rotatable=false, timeouts={implicit=0.0, pageLoad=300000.0, script=30000.0}, pageLoadStrategy=normal, moz:headless=false, platform=ANY, moz:accessibilityChecks=false, moz:useNonSpecCompliantPointerOrigin=false, acceptInsecureCerts=false, browserVersion=61.0, platformVersion=10.0, moz:processID=2796.0, browserName=firefox, javascriptEnabled=true, platformName=windows_nt, moz:webdriverClick=true}]
Session ID: XXXXXXXX-XXXX-XXXX-XXXX-XXXX8a174a14
Stacktrace:
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0 (None:-2)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance (None:-1)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance (None:-1)
    at java.lang.reflect.Constructor.newInstance (None:-1)
    at org.openqa.selenium.remote.http.W3CHttpResponseCodec.createException (W3CHttpResponseCodec.java:150)
    at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode (W3CHttpResponseCodec.java:115)
    at org.openqa.selenium.remote.http.W3CHttpResponseCodec.decode (W3CHttpResponseCodec.java:45)
    at org.openqa.selenium.remote.HttpCommandExecutor.execute (HttpCommandExecutor.java:164)
    at org.openqa.selenium.remote.service.DriverCommandExecutor.execute (DriverCommandExecutor.java:82)
    at org.openqa.selenium.remote.RemoteWebDriver.execute (RemoteWebDriver.java:637)
    at org.openqa.selenium.remote.RemoteWebDriver.execute (RemoteWebDriver.java:694)
    at org.openqa.selenium.remote.RemoteExecuteMethod.execute (RemoteExecuteMethod.java:33)
    at org.openqa.selenium.remote.RemoteLogs.getAvailableLogTypes (RemoteLogs.java:103)
    at org.openqa.selenium.remote.server.handler.GetAvailableLogTypesHandler.call (GetAvailableLogTypesHandler.java:37)
    at org.openqa.selenium.remote.server.handler.GetAvailableLogTypesHandler.call (GetAvailableLogTypesHandler.java:28)
    at java.util.concurrent.FutureTask.run (None:-1)
    at org.openqa.selenium.remote.server.DefaultSession$1.run (DefaultSession.java:176)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (None:-1)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (None:-1)
    at java.lang.Thread.run (None:-1)
Sauce Labs Job: https://saucelabs.com/jobs/76d2faff6e17401b8559acd6644d9f83
 generated xml file: /home/jenkins/slave/workspace/mozillians.stage@2/results/junit.xml 
 generated html file: /home/jenkins/slave/workspace/mozillians.stage@2/results/index.html 
=========================== short test summary info ============================
FAIL tests/test_profile.py::TestProfile::()::test_profile_deletion_confirmation
=============================== warnings summary ===============================

[0] https://qa-master.fxtest.jenkins.stage.mozaws.net/job/mozillians.stage/485/testReport/junit/tests.test_profile/TestProfile/test_profile_deletion_confirmation/

restmail.net is down

https://restmail.net is down

https://www.webpagetest.org/result/180403_T7_d0dfda27fdfebe9e7b535350713e073c/

All I know is that it was up 16 hours ago, when our last test ran: https://qa-master.fxtest.jenkins.stage.mozaws.net/job/fxapom.stage/279/

/cc @jrbenny35 @davehunt

limit uses of restmail.net

1. if you send a bulk list longer than N, we flat out reject
2. domain MUST be on a whitelist of domains ([restmail.net] to start).

/cc @stephendonner

database can grow until you run out of memory

There's nothing to stop the database from growing indefinitely.

This bit restmail.net

res.json not returning?

Have been debugging restmail this morning, thought I'd log a bug for the latest thing instead of leaving its history undocumented.

It looks like redis is fine, forever list shows reasonable uptime for the emailserver.js and webserver.js processes, and the emailserver log is quiet.

But the webserver log has lots and lots of entries of the form

127.0.0.1 - - [Tue, 09 Jul 2013 19:35:15 GMT] "GET /mail/bidpom_68319f9e-e8ce-11e2-91bc-406c8f3aa05a HTTP/1.1" 200 2 "-" "python-requests/0.13.3 CPython/2.7.1 Darwin/11.4.2"
TypeError: Cannot call method 'lrange' of null
    at /home/app/code/webserver.js:39:6
    at callbacks (/home/app/code/node_modules/express/lib/router/index.js:272:11)
    at param (/home/app/code/node_modules/express/lib/router/index.js:246:11)
    at param (/home/app/code/node_modules/express/lib/router/index.js:243:11)
    at pass (/home/app/code/node_modules/express/lib/router/index.js:253:5)
    at Router._dispatch (/home/app/code/node_modules/express/lib/router/index.js:280:4)
    at Object.handle (/home/app/code/node_modules/express/lib/router/index.js:45:10)
    at next (/home/app/code/node_modules/express/node_modules/connect/lib/http.js:203:15)
    at Object.logger [as handle] (/home/app/code/node_modules/express/node_modules/connect/lib/middleware/logger.js:155:5)
    at next (/home/app/code/node_modules/express/node_modules/connect/lib/http.js:203:15)

What's odd is that the affected code should never be reached: it should bail at if (!db) res.json([]), but it doesn't--it keeps on trucking.

Taking a break, then we'll get back to this.

Add open source software license

This Mozilla repository has been identified as lacking a license. Consistent with Mozilla's Licensing Policy an open source license should be applied to the code in this repository.

Please add an appropriate LICENSE.md file to the root directory of the project. In general, Mozilla's licensing policies are as follows:

• Client-side products created by Mozilla employees or contributors should use the Mozilla Public License, Version 2.0 (MPL).

• Server-side products or utilities that support Mozilla products may use either the MPL or the Apache License 2.0 (Apache 2.0).

In special cases, another license might be appropriate. If the repository is a fork of another repository it must apply the license of the original. Similarly, another license might be appropriate to match that of a broader project (for example Rust crates that Firefox depends on are published under an Apache 2.0 / MIT dual license, as that is the dual license used by the Rust programming language and projects).

Please ensure that any license added to the LICENSE.md file matches other licensing information in the repository (for example, it should match any license indicated in a setup.py or package.json file).

Mozilla staff can access more information in our Software Licensing Runbook – search for “Licensing Runbook” in Confluence to find it.

If you have any questions you can contact Daniel Nazer who can be reached at dnazer on Mozilla email or Slack.

OPENLIC-2023-01

Email subject appears twice in email json

javascript

[{"text":"\n",
"headers":{"received":"---------------",
"message-id":"[email protected]",
"date":"Wed, 23 May 2012 15:03:21 +0100",
"from":"---------------",
"user-agent":"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1",
"mime-version":"1.0",
"to":"[email protected]",
"subject":"testering!!",
"content-type":"text/plain; charset=ISO-8859-1; format=flowed",
"content-transfer-encoding":"7bit"},
"subject":"testering!!",
"priority":"normal",
"from":[{"address":"---------------","name":"---------------"}],
"to":[{"address":"[email protected]","name":""}]
}]

CORS Support

We would need this to run our test suite in the browser.

mozilla/fxa-js-client#39

Hosting and Deployment of restmail.net

@6a68 added my keys and I've done a push to update restmail.net to the latest code. I think we should consider formalizing ownership and maintenance of this service a little more. I'm not entirely sure what the current setup is, but ISTM we should have something like:

• an awsbox deployed into mozsvc dev (prod?) AWS environment
• restmail.net domain managed by mozsvc route53

This would make it easier for us to keep the service running and up-to-date, without dependency on any key individuals. Thoughts?

https://restmail.net/mail/<user> does not return any JSON

Hi restmail team!
WebQA team is using this to test Persona (http://beta.123done.org/).
We create a new restmail account on Persona site and then check the confirmation email message from that account. The problem is that when accessing https://restmail.net/mail/ no JSON is returned.
Is there a known issue about this?
Thanks!

Sometimes emails arrive to wrong email boxes

cc @jrgm @rfk

See http://restmail.net/mail/pushtest

signin0.7721520143095404%40restmail.net arrived to the pushtest mailbox

@jrgm has more information on this. Seems like a bug in restmail updates

CODE_OF_CONDUCT.md file missing

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please reach out to [email protected].

(Message COC001)

redis acts like a db, not a cache

The default redis install process seen in post-create.sh sets redis to never evict stuff from memory. This led to an outage today because redis had gradually sucked up over a gig of memory, starving the VM.

The fix is pretty simple, we just set maxmemory to a limit in the redis conf file, and specify an eviction algorithm using maxmemory-policy.

@jedp is preparing a PR that'll set TTLs on keys. Once that's done, redis will reap expired keys by scanning 1000/second and killing expired ones. To guarantee we don't run out of memory, though, we can additionally set maxmemory-policy volatile-lru to evict least-recently-used expired keys when maxmemory is reached.

Annoyingly, installing redis via its install_server shell script doesn't give us any time between creating the config file and starting the server--and I've had trouble, for some reason, with /etc/init.d/redis_6379 stop not actually doing anything on the VM.

I assume we're building redis because the amzn repo doesn't have a redis package. I kinda think the best solution is to add an extra repo like EPEL, get redis from it, and then append the maxmemory and maxmemory-policy lines to it, assuming the init.d script will work properly with the upstream package:

echo "maxmemory 524288000 >> /etc/redis/6379.conf"
echo "maxmemory-policy volatile-lru >> etc/redis/6379.conf"
/etc/init.d/redis_6379 stop
/etc/init.d/redis_6379 start