knadh / listmonk Goto Github PK

Just stumbled on this today, very nice work.

As a feature request, I was wondering the ability to implement functionality to send out a campaign containing contents of an RSS Feed.

Here are two use cases:

1. At a specific period, Listmonk checks for the existence of a new article in the RSS / XML feed and sends a messaeg to subscribers with a custom template and the capability of either displaying the Title of the Article (with link to Original Source), or the actual content.

2. Similar to 1 - However on a scheduled campaign, check the RSS/XML contents and send a digest of information (Ie, Send articles to subscribers of anything that has appeared within the past 3 days - either an excerpt, title, or full content). If no new articles, skip sending the campaign.

Mailchimp handles 2 quite well, and I've yet to see an open source mailing list manager that gets it right. Mailtrain handles Scenario #1, PHP List attempts to handle #1 and #2. So I thought I would see if this would make sense inside of your application.

Can you split main package to cmd/listmonk
And other stuff provide like listmonk package for easy to integrate to other apps?
Also why you dont use something like xorm to abstract db access?
This can bring ability to use sqlite or mysql too

listmonk --install connects to database with the same name as database user.
version 0.3.3

System environment

Ubuntu 18.04 LTS

[app]
# Interface and port where the app will run its webserver.
address = "0.0.0.0:9000"

# Public root URL of the listmonk installation that'll be used
# in the messages for linking to images, unsubscribe page etc.
root = "https://listmonk.mysite.com"

# (Optional) full URL to the static logo to be displayed on
# user facing view such as the unsubscription page.
# eg: https://mysite.com/images/logo.svg
logo_url = ""

# (Optional) full URL to the static favicon to be displayed on
# user facing view such as the unsubscription page.
# eg: https://mysite.com/images/favicon.png
favicon_url = ""

# The default 'from' e-mail for outgoing e-mail campaigns.
from_email = "XXX <[email protected]>"

# List of e-mail addresses to which admin notifications such as
# import updates, campaign completion, failure etc. should be sent.
# To disable notifications, set an empty list, eg: notify_emails = []
notify_emails = ["[email protected]"]

# Path to the uploads directory where media will be uploaded.
upload_path = "uploads"

# Upload URI that's visible to the outside world. The media
# uploaded to upload_path will be made available publicly
# under this URI, for instance, list.yoursite.com/uploads.
upload_uri = "/uploads"

# Maximum concurrent workers that will attempt to send messages
# simultaneously. This should depend on the number of CPUs the
# machine has and also the number of simultaenous e-mails the
# mail server will
concurrency = 100

# The number of errors (eg: SMTP timeouts while e-mailing) a running
# campaign should tolerate before it is paused for manual
# investigation or intervention. Set to 0 to never pause.
max_send_errors = 1000


# Database.
[db]
host = "demo-db"
port = 5432
user = "listmonk"
password = "listmonk"
database = "listmonk"
ssl_mode = "disable"

# TQekh4quVgGc3HQ


# SMTP servers.
[smtp]
    [smtp.postal]
        enabled = true
        host = "smtp.gmail.com"
        port = "465"

        # cram or plain.
        auth_protocol = "plain"
        username = "[email protected]"
        password = "XXXXX"

        # Maximum time (milliseconds) to wait per e-mail push.
        send_timeout = 5000

        # Maximum concurrent connections to the SMTP server.
        max_conns = 10

Description

• The application timed out while connecting to the SMTP server and no mails were sent. Subsequently, the campaign status changed to finished.

• Tried it with OAuth2 authentication mechanism for the SMTP server but didn't Plain mechanism is being used for all other auth methods (other than cram)

Likely issue

plain authenticaiton mechanism is being used in case of auth auth mechanisms other than cram.

First off, thank you so much for open sourcing this tool :)

Is the list subscription status currently used for anything? It seems to be for double opt-in, but it looks like it isn't used yet. I've set up my mailing list with double opt-in by only adding a list to a subscriber after they confirm, but that's a bit annoying because there's no subscription management page before I added users to the list (so I can't use the built-in UI to let users wipe their data before they confirm).

I'm running the installation automatically via a Kubernetes job, and currently I need to pass y into the binary:

yes | ./listmonk --install

This is a little hacky and I would prefer and -y flag for the install to run it automatically

Integrate with

https://github.com/c1982/bomberman
https://github.com/trumail/trumail

Also something where we can send extremely fast using SNAT/DNAT iptables stuff.

Cloudflare API to generate DKIM etc and push to domains.

create advanced list segmentations.
Opener IP to email etc relations.
TCP-SMTP Proxy server can work aswell.

We need a subscription form

At First: Thank you very much for this incredibly nice software.

The Issue: When sending Emails with tracking features (Tracking Pixel and/or Tracking link) the content is only accesible after restarting listmonk in the commandline - otherwise i get a json response message:"not found"

Is there any way to solve this?

Thank you

The UI is built using Ant Design components and it supports responsiveness to a reasonable extent with special tags and attributes. Most elements don't seem to respond implicitly.

Need to figure out how Ant Design does responsive components and implement it.

Hey,
I'm testing listmonk as an option to send out newsletters.
I created a template and wanted to set the {{ UnsubscribeURL }} function in the footer.

When I click on preview I get an error:

{"message":"Error compiling template: error compiling base template: template: base:58: function "UnsubscribeURL" not defined"}

Did it do something wrong?

Sorry, first time I am running docker on windows to install and setup listmonk I am getting below error. Where to change the IP address which I can point to local?

I hope it is trying to connect local DB

docker-compose run --rm app ./listmonk --install to setup the DB.

PS C:\listmonk> docker-compose run --rm app ./listmonk --install
Starting listmonk_db_1 ... done
2019/10/14 04:26:31 main.go:107: reading config: config.toml
2019/10/14 04:26:34 main.go:189: error connecting to DB: dial tcp 23.217.138.110:5432: connect: connection refused
PS C:\listmonk>

Steps to reproduce

• Downloaded the latest release from github:
  curl -sL https://github.com/knadh/listmonk/releases/download/v0.2.2-alpha/listmonk_0.2.2-alpha_linux_amd64.tar.gz | tar xz

• Created a new config file
  ./listmonk --new-config

• Spawned an ephemeral postgres container
  docker run -p 5432:5432 -e "POSTGRES_DB=listmonk" -e "POSTGRES_USER=listmonk" -e "POSTGRES_PASSWORD=listmonk" postgres:latest

• Configured DB
  ./listmonk --install

• Created an uploads folder in the current directory
  mkdir uploads

• Started listmonk
  ./listmonk

• Visited http://localhost:9000/campaigns/media and uploaded a file
  

What happened
Got a 404 on the uploaded file. http://localhost:9000/uploads/DeepinScreenshot_select-area_20191009181805.png

Expected scenario
The file should have been previewed correctly, not 404.

Pressing on + New Campaign lands to a blank page,

Displaying error of Error Fetching Data Cannot Read Property 'id' of unidentified.

Will you use Postfix for handling bounces (tailing and parsing its log, just like cuttlefish does, or implementing a full golang smtp server with go-smtp ?

Hi,

RFC 8314 recommends to use port 465 for SMTP Submission. And if I set port 465 in config.toml, I will get:

⇨ http server started on [::]:9000
2019/11/08 10:55:21 manager.go:166: start processing campaign (Test campaign)
2019/11/08 10:55:21 manager.go:349: campaign (Test campaign) finished
2019/11/08 10:55:26 manager.go:243: error sending message in campaign Test campaign: timed out
2019/11/08 10:55:26 notifications.go:28: error sending admin notification (Finished: Test campaign): timed out

Many thanks,
HLFH

The motivation is explained in https://softwarerecs.stackexchange.com/q/71336/1877
If listmonk is indeed compatible with HTML emails, that should be stated in its README.

Thanks and Merry Christmas!

--

Basile Starynkevitch - http://starynkevitch.net/Basile/
[email protected] Bourg-La-Reine, near Paris, France

I'm using docker-compose to start and stop my containers. As soon as listmonk is starting it has a status of "Restarting (1) Less than a second ago". Logs output something along the lines of

... main.go:112 reading config config.toml ...main.go:236 error connecting to DB dial tcp lookup demo-db on 127.0.0.11:53 no such host

I am unable to shell into the container and make adjustments to the config.toml because it is restarting. How should I proceed?

• Enable subscribers to download all data on them
• Enable subscribers to wipe all data on them

It'd be nice if there was some basic UI customization that can be configured so as to make it clear to the user that this is being hosted by the company. For example, this could be as minimal as allowing a logo upload. A more involved customization could be choosing company branding colors to serve as the color palette/theme.

I have got the setup server ready, but the installation failed with the following error.

[root@letter listmonk]# ./listmonk --install
2019/10/10 00:49:51 main.go:108: reading config: config.toml
2019/10/10 00:49:51 main.go:136: unable to initialize embedded filesystem: no ID found in the file
2019/10/10 00:49:51 main.go:137: using local filesystem for static assets
2019/10/10 00:49:51 main.go:213: failed to initialize local file for assets: stat frontend/build: no such file or directory

I am unsure why I am having file system error, could you please advice?

2020/02/14 18:48:37 manager.go:169: start processing campaign (Test 1)
2020/02/14 18:48:37 manager.go:352: campaign (Test 1) finished
2020/02/14 18:48:37 manager.go:246: error sending message in campaign Test 1: EOF
2020/02/14 18:48:38 notifications.go:38: error sending admin notification (Finished: Test 1): EOF

Importing CSV file fails immediately. Log mentions an unknown email header and then mentions not finding the header.

Logs:

2019/12/04 16:39:29 processing 'people.csv'
2019/12/04 16:39:29 ignoring unknown header 'email'
2019/12/04 16:39:29 'email' column not found in '/tmp/listmonk504876770'

CSV structure:

email,name
[email protected],"Name"

I'm getting this error when following the setup guide:

listmonk/listmonk   latest              66e0d30eac19        6 weeks ago         22.8MB

 root  Soundwave  SSH  ~  #  docker run listmonk/listmonk

2019/10/23 14:10:30 main.go:107: reading config: config.toml

2019/10/23 14:10:30 main.go:189: error connecting to DB: dial tcp: lookup demo-db on 67.207.67.2:53: no such host

This is running on DigitalOcean with their default docker appliance. Is this an issue with listmonk or DigitalOcean?

Currently, only the subscribers API is documented, while there are many more handlers to document.

The docs are published at https://listmonk.app/docs/apis/apis/ and the source is at https://github.com/knadh/listmonk-site

Hello,

In All campaigns tab, I have listed 18 test campaigns. I would like to delete them.
How to do that?
Can you please support it in the WebUI?

Thanks,
HLFH

I just got this deployed to AWS elastic beanstalk. I have nginx sitting in front of it so I can do the basic auth stuff like suggested in some of the notes, and it's all running on Docker. When I visit my page, I get a 500, so I had some questions here.

1. How do I do logging?
2. Nginx error.log is reporting crypt_r() failed (22: Invalid argument). Any idea if that's listmonk related or not?
3. Is there a way to configure where errors are logged?

Hello,

Is there a way to send, via api, an mail to a list, with a predefined template and given params ?

I have a template with multiple placeholder {{message}} {{ticketLink}} {{title}} {{ incident_id }} and i wish to automate email sending in a external script which will consume the api with theses params :

• listNameOrID
• templateNameOrID
• params
  • title = “incident on dhcp”
  • message = “there is an incident”
  • linkTicket = “https://whatever.com/INC-5432”
  • incident_id = 354543

Can i achieve this with the last alpha ?

When I start a scheduled Campaign it gets sent instantly instead of waiting for the set time to send the emails.

I am using the latest version (0.4.0-alpha)

Am I doing something wrong?

It would be great to have support for Amazon SES or third-party services to send emails. It is already supported or in features list down the line?

I Would like to work on it if possible :)

@knadh

I get the following error after starting up a docker image.

2019/11/15 12:01:15 main.go:112: reading config: config.toml
2019/11/15 12:01:15 main.go:284: no SQL queries loaded: Error preparing query 'get-dashboard-stats': pq: relation "lists" does not exist

Hi Kailash,

Wonderful software. This is more of a question/request than an issue. Will it be possible for you consider licensing this software under Apache or other more lenient license? Here is my reasoning:

For low-level/core software GPL and AGPL make perfect sense as it forces everyone to at least open-source their changes and everyone can benefit from that. A good example for this can be Email server(like Postal).

But for a software like Listmonk, which manages email list/segmentation and sending logic, there will be cases where organizations have to make some changes. Also due to nature of AGPL, there is not enough clarity if software used over network(even excluding API) is considered distribution and hence needs open-sourcing all parts.

this link is on the https://listmonk.app/docs/external-integration/ page in the Using API's subheading

In documentation, {{ .TrackView }} is found instrad of {{ TrackView }}.
This leads to an error when previewing the campaign or the template.

Hi,

first of all thanks for your work and time, to create a simple application for managing mails and newsletter.

I have got a question on campaigns and sending mails on a scheduled time. If I create a new campaign and configure date and time, it is not saved when clicking on the "continue" button. After klicking "continue" again, the campaign is save as "draft".

If I edit this campaign, the schedule is not set. If I do configure it again, I can tell "schedule", and it is set to status "schedule". But it will never be executed. There is some note under "schedule", that is always the same: "Scheduled — Tue 14 Jan 2020, 08:01 AM" - is that okay? I scheduled it on another time today, but this note would not tell, that its schedule correct.

Do I am something wrong, or does the application need more planning time in the future? I am only waiting minutes for executing tasks in the future.

If I can assist you in any way, please tell me.

Thanks!

Platform: Windows 10, 1903
Postgre Server: Free plan of ElephantSQL

Everything is working fine, the UI shows up in the browser but after creating a campaign when I hit the "Continue" button it throws an error.

Hi,

In a test campaign, I can set in From address field: "Bécassine [email protected]"
If I do that, the received email shows:

From: =?UTF-8?q?B=C3=A9cassine_ [email protected], [email protected]

So I get in From the username weirdly written on the accent part, and the host, which did not need to be written here.

Therefore, I can also set in From address field: "Becassine [email protected]".
If I do that without accent, the received email shows:

From: Becassine [email protected]

And it is correct.

Can you fix it?

Thanks,
HLFH

Currently Listmonk stores the uploaded media on the local disk. While this works well, sometimes there might be a requirement of having a remote backend (which effectively makes listmonk as stateless). It's more easier to horizontally scale out since there's no volume attached with the app. This is also something the 12factor app suggests.

I've tried my hands on it and created an interface for the FileManager which supports three operations

• Get
• Upload
• Delete

type FileManager interface {
	Upload(string, io.Reader) error
	Delete(string) error
	Get(string) error
}

In uploadFile I pass the appropriate FileManager (disk/s3) and call the upload() method implemented by the interface.

Issues faced so far:

• handleGetMedia serves the media from the local path, I am unsure how this can be migrated to S3. S3 generates it's own URLs, can I return the same? Downsides will be high bandwidth costs to S3 (but then again, the same is true for any paid CDN)

Before continuing this, I wanted to have a discussion on whether there's interest in adding a remote storage backend?

Hey there! Great work on this project!
I installed the app based on the description you have been provided.
I run email service using poste. I almost not touched the config they provided. (smtp was run on port 25)
so on the config.toml, on the smtp part there are two fields which needed to complete. This is my config:
`
[smtp]
[smtp.my0]
enabled = true
host = "aalimoshaver.com"
port = "25"

    # cram or plain.
    auth_protocol = "plain"
    username = "aalibash"
    password = "somepassword 😉"

    # Maximum time (milliseconds) to wait per e-mail push.
    send_timeout = 5000

    # Maximum concurrent connections to the SMTP server.
    max_conns = 10

[smtp.postal]
    enabled = true
    host = "aalimoshaver.com"
    port = "25"

    # cram or plain.
    auth_protocol = "plain"
    username = "aalibash"
    password = "somepassword 😉"

    # Maximum time (milliseconds) to wait per e-mail push.
    send_timeout = 5000

    # Maximum concurrent connections to the SMTP server.
    max_conns = 10

when I try to test withSend test messagesin campaign part, I see this error:Error sending test: 550 I cannot deliver mail for someRealEmail.`
I also used thunder bird to check remote connection of my mail server, poste. But that was fine. from there for the Outgoing part, It shows me these:
SMTP (protocol) - smtp.aalimoshaver.com (Server hostname) - 25 (port) - STARTTLS (SSL) - Normal Password (Authentication)
Can you help me fix this out?
Many thanks!
I also thought to contribute to this project as a developer. I can help you out, specially on the templating part.
Thanks again!

Hello,
Is it possible to embed images into html? Mail clients doesn't like images from external www servers and usually blocks it. Embedding makes email larger but it allow display mail without accepting "show remote content".

A transactional email, or triggered email, is an email sent to a recipient after an action. Without campaigns.

I did you the docker image

After running listmonk on my cloud-hosted VM for a couple days, I can't connect anymore with postgres giving sorry, too many clients already. This is the second time this happened.

What's going on here? Shouldn't connections only be open for the duration of a request?

It seems weird to me that no one else ran into this issue yet (or at least opened a ticket for it), so I'm wondering if it could have anything to do with my set up. Feel free to ask for any information about the environment that might be relevant.

I think any credentials should be configurable via environment variables.
I want to deploy listmonk to Kubernetes, I’m creating a Helm chart and I want to leverage a Postgres depedency chart which exposes its password via a secret. I’d like to mount that secret into the listmonk pod via environment-variables.
Also saving the credentials into a secret allows encrypting these values.
If you give me some guidance on code structure I could create a PR request implementing the feature.

Tried setting up the project via docker.
Cloned, edited the config.toml file. And Updated data there.
Then ran the below and please find the response as well below.

➜ docker-compose up -d
Starting listmonk_db_1 ...
listmonk_demo-db_1 is up-to-date
Starting listmonk_db_1 ... error

ERROR: for listmonk_db_1  Cannot start service db: driver failed programming external connectivity on endpoint listmonk_db_1 (d1928b4afe6a4cfea6e6ac4e121ce983b95b29eb8b7473bb814fb09a469a09af): Bind for 0.0.0.0:9432 failed: port is already allocated

ERROR: for db  Cannot start service db: driver failed programming external connectivity on endpoint listmonk_db_1 (d1928b4afe6a4cfea6e6ac4e121ce983b95b29eb8b7473bb814fb09a469a09af): Bind for 0.0.0.0:9432 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.

docker app is using 9432 by default.

How do I resolve this?

This is more of a feature thought but perhaps you could punt on actually doing authentication and use GoTrue (https://github.com/netlify/gotrue). It might be better to keep third party dependencies out but I've used GoTrue and been happy with it.

Hello, our SMTP servers are set-up in such way that they do not require any authentication, however putting "plain", username "", password "" or "none" and deleting the username and password completely results in error 535.

Could you please add a way to add SMTP servers that do not require any authentication?

Thanks for creating this nice piece of software. Looks very promising.

Is there already a way to have a public subscription form?

I see you can get an unsubscribe link in the email like this: http://listmonk.mysite.com/subscription/ea1fbbf7-sds22-4ec9-6346h-0187461038ad/df2fe833-25f2-4444hh-baf5-sdhshdhshs2442

Is there also such a thing to subscribe?
I create a public list and the total public lists in the dashboard is 2. I would expect there would be a public endpoint like http://listmonk.mysite.com/public/ but I get a 404.

When creating a new subscriber via the api and I don't pass a name:

curl https://service.com/api/subscribers -H 'content-type: application/json' --data '{"email":"[email protected]"}'

The server responds correctly with

{"message":"invalid or empty name \"\""}

But if the status is missing I get a postgres error instead:

curl https://service.com/api/subscribers -H 'content-type: application/json' --data '{"email":"[email protected]", "name": "Nicolai Schmid"}'

{"message":"Error creating subscriber: pq: invalid input value for enum subscriber_status: \"\""}

Maybe the validation for the status-field is missing