pietheinstrengholt / rssmonster

Google Reader inspired self-hosted RSS reader written in VueJS with an Express NodeJS backend. RSSMonster is compatible with the Fever API.

RSSMonster

Copyright (c) 2023 Piethein Strengholt, [email protected]

Background

RSSMonster is an easy to use web-based RSS aggregator, created as an alternative for Google Reader. RSSMonster features a lightweight fluid responsive design. It is written in JavaScript: Vue.js 3 for the frontend and Express for the backend. It uses Bootstrap for making the design responsive.

RSSMonster tries to mimic the behaviour of Google Reader. It marks items when you start scrolling. It knows what content is hot. It features progressive web app support, drag and drop support for managing feeds, dark mode, and more! RSSMonster is compatible with the Fever API. Feel free to add any contributions or new features.

Prerequisites

• NodeJS 16.x or higher
• Git
• A Mysql installation (other databases will as well with some configuration)

How to get everything installed

• Clone this repository git clone https://github.com/pietheinstrengholt/rssmonster.git .
• Run npm install in both the client and server folder
• Find the .env.example file in the root of both the client and server folder. Copy and rename the files to .env
• Edit .env inside the server folder and enter your Mysql or Database server login data (at least fill DB_DATABASE, DB_USERNAME and DB_PASSWORD).
• Edit .env inside the client folder. Change the VITE_APP_HOSTNAME, so it points to the back-end.
• Run ./node_modules/.bin/sequelize db:migrate && ./node_modules/.bin/sequelize db:seed:all in side the server folder. this will add all needed database tables and content to your mysql database. Alternatively you can also uncomment the //force: true in the app.js inside the server folder to create the schema structure.
• Optional: Add a cronjob to crawl http://localhost/api/crawl every 5 minutes.

Development

If you would like to run RSSMonster in development mode I recommend to run:

• Inside the client folder: npm run dev.
• Inside the server folder: npm run debug.

Production

If you would like to run RSSMonster in production mode I recommend to run:

• Update the VITE_APP_HOSTNAME and VITE_NODE_ENV inside the file client/.env. Most likely you want to remove port 3000 and point to the url where the backend will be running. For production, make sure you set the VITE_NODE_ENV to production.
• Update the NODE_ENV inside the file server/.env. For production, make sure you set the NODE_ENV to production.
• Inside the client folder build all the static files with: npm run build.
• Move the dist output folder created inside the client folder to the server folder. The NodeJS server is also capable of serving out static content.
• Inside the server folder: npm run start.

Docker for development

• Run the following command to build all the images: docker-compose build
• Run the following command to start the containers: docker-compose up
• The client will be running on port 8080 and communication with the backend takes place via 3000. Make sure these ports aren't being used. The mysql database is accessible via port 3306.

Docker for production

The production version has the server and client combined into a single container. The VueJS is also compiled into an optimized version. To build this single image, run the following command: docker build -t rssmonster . Lastly you need to run the docker container. You need to provide the correct environment variables for the database server to connect to. Here's is an example: docker run -d -t -i -e NODE_ENV=production -e DB_HOSTNAME=localhost -e DB_DATABASE=rssmonster -e DB_USERNAME=rssmonser -e DB_PASSWORD=password -p 3000:3000 rssmonster

AWS Beanstalk

• Setup your AWS Security credentials: https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html
• Download and install the Beanstalk CLI: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb-cli3.html
• Run the following command: eb init
• Select your region and use the aws-access-id & aws-secret-key from step 1.
• Provision a new environment: eb create rssmonster-app --envvars "NODE_ENV=production" --database --database.engine mysql --database.username rssmonster --database.password "fvX2Ht8jXxXEYlZ6"
• SSH into the new environment: eb ssh rssmonster-app
• Migrate the database content: /app/server/node_modules/.bin/sequelize db:migrate && /app/server/node_modules/.bin/sequelize db:seed:all

Reeder (iOS) integration support (via Fever API)

RSSMonster is compatible with the Fever API. Apps like Reeder (iOS) will support this. To use the Reeder API (http://feedafever.com/api), point to the following url:

http://yourRSSMonsterurl/api/fever

Any username and password will work.

Credits

The following scripts and plug-ins are used within RSSMonster

• NodeJS https://nodejs.org/en/
• Twitter bootstrap: https://twitter.github.io/bootstrap/
• Feedparser: https://github.com/danmactough/node-feedparser/
• VueJS: https://vuejs.org/
• Vue infinite scrolling: https://github.com/PeachScript/vue-infinite-loading
• Waypoints: https://github.com/imakewebthings/waypoints

rssmonster's Issues

Authentification

Because this is self-hosted, different users is not as important, but having a way to authenticate oneself would be nice.
I don't want anyone to mark all my feeds as read. And it would also help with security be reducing the attack surface.

Does RSSmonster support the "hot" links feature?

Since Fever has stopped development, I'm looking for another way to find "hot" links in my RSS feeds. Does RSS support this? If not, is it a planned feature?

Cannot compile assets

I'm trying to build a Docker image for the application. After installing a lot of dependencies, I could get npm so far as to build all the native tools it needs. But when it comes to compiling the Vue sources, the process (npm run production) ends with

> [email protected] production /var/www/rssmonster
> cross-env NODE_ENV=production node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js

 95% emitting ERROR  Failed to compile with 3 errors13:29:08                         

These relative modules were not found:

* ./components/Home.vue in ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue
* ./components/Quickbar.vue in ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue
* ./components/Sidebar.vue in ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue
      Asset    Size  Chunks             Chunk Names
/js/main.js  124 kB       0  [emitted]  /js/main

ERROR in ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue
Module not found: Error: Can't resolve './components/Home.vue' in '/var/www/rssmonster/resources/assets/js'
 @ ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue 291:0-41
 @ ./resources/assets/js/App.vue
 @ ./resources/assets/js/main.js
 @ multi ./resources/assets/js/main.js

ERROR in ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue
Module not found: Error: Can't resolve './components/Quickbar.vue' in '/var/www/rssmonster/resources/assets/js'
 @ ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue 292:0-49
 @ ./resources/assets/js/App.vue
 @ ./resources/assets/js/main.js
 @ multi ./resources/assets/js/main.js

ERROR in ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue
Module not found: Error: Can't resolve './components/Sidebar.vue' in '/var/www/rssmonster/resources/assets/js'
 @ ./node_modules/babel-loader/lib?{"cacheDirectory":true,"presets":[["env",{"modules":false,"targets":{"browsers":["> 2%"],"uglify":true}}]],"plugins":["transform-object-rest-spread",["transform-runtime",{"polyfill":false,"helpers":false}]]}!./node_modules/vue-loader/lib/selector.js?type=script&index=0!./resources/assets/js/App.vue 290:0-47
 @ ./resources/assets/js/App.vue
 @ ./resources/assets/js/main.js
 @ multi ./resources/assets/js/main.js
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] production: `cross-env NODE_ENV=production node_modules/webpack/bin/webpack.js --progress --hide-modules --config=node_modules/laravel-mix/setup/webpack.config.js`
npm ERR! Exit status 2
npm ERR! 
npm ERR! Failed at the [email protected] production script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/rssmonster/.npm/_logs/2018-04-09T13_29_08_361Z-debug.log

My current Dockerfile looks something like this:

FROM alpine:3.7

# install packages
RUN apk --no-cache add php7 php7-fpm php7-mysqli php7-json php7-openssl php7-curl \
    php7-zlib php7-xml php7-phar php7-intl php7-xmlreader php7-ctype \
    php7-mbstring php7-pdo_mysql nginx supervisor curl php7-mcrypt nodejs \
    imagemagick-dev autoconf automake gcc g++ make libtool nasm build-base libpng-dev

# install Composer
ADD https://getcomposer.org/download/1.6.2/composer.phar /usr/bin/composer
RUN chmod +rx /usr/bin/composer

# setup user accounts
RUN adduser -D rssmonster
RUN adduser nginx rssmonster

# fetch rssmonster source
WORKDIR /var/www
ADD https://github.com/pietheinstrengholt/rssmonster/archive/master.tar.gz .
RUN tar xzf master.tar.gz && mv rssmonster-master rssmonster

# (removed a bunch of nginx, php-fpm, supervisor etc. setup stuff)

# install PHP dependencies
USER rssmonster
#RUN COMPOSER_CACHE_DIR=/tmp/.composer composer install --no-dev --no-progress --no-suggest --prefer-dist && rm -rf $COMPOSER_CACHE_DIR

# build assets
RUN npm install && \
    npm run production && \
    rm -rf node_modules

Implement RSS Autodiscovery

Many websites have the feature to automatically point to the RSS feed location. Instead of typing in the RSS feed it would be a nice feature to only add the url and let the tool find up the RSS feed location based on the metadata in the header. Here’s how this can be implemented:

http://www.rssboard.org/rss-autodiscovery

No posts available at first load

always shows as above and javascript console shows:
Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user’s experience. For more help http://xhr.spec.whatwg.org/

is this project dead?

hello is this project dead? i see no update since 2 years ago

Simplify API_URL_ROOT configuration

It would be nice to have an ENV variable for this, or even detect it in the browser with window.location.host or similar.

Production docker build fails

Hey there, the production docker build fails. Changing the node base image to node 16 seems to resolve some of it, but then the API gives 404 errors to the client. Would you be able to take a look?

Fix issue when adding links with https

Installation from docker now working?

Hi I tried to install it via docker like described but when I run it I get PHP errors

Warning: require_once(/var/www/bootstrap/../vendor/autoload.php): failed to open stream: No such file or directory in /var/www/bootstrap/app.php on line 3

Fatal error: require_once(): Failed opening required '/var/www/bootstrap/../vendor/autoload.php' (include_path='.:/usr/local/lib/php') in /var/www/bootstrap/app.php on line 3

What have I done wrong?

Validate feed saves it

I just got around to testing #49. Validating a feed now adds it to the DB, instead of waiting to click save.

Links are inactive in top menu

Hi, I have tried to install rssmonster on a Raspberry Pi server (Debian stretch with PHP7, Nginx) and followed all your instructions one after the other - when I finally go into the public/index.php page, I get the interface but none of the menu links ("add new feed" etc...) are active - they can be clicked but they do nothing.

Obviously I must have something misconfigured somewhere, but do you have any idea what it could be ?
Thanks for your help

Fix mobile marking articles ahead

When in mobile view, sometimes the articles are already marked as read, while they are not on the screen yet.

Feature Request: sqlite support

I'm well aware the project is WIP but for a future reccomendation, could sqlite support be considered? I hate setting up mysql for anything, not to mention it's not a dep of anything running on my server. It would be nice to just have one file serve as the database. It's not suggested for traffic/user heavy situations but I'm only pulling feeds from small blogs.

fix travis CI after adding db:migrate step to package.js

The line "prestart": "./node_modules/.bin/sequelize db:migrate && ./node_modules/.bin/sequelize db:seed:all" in the package.json breaks Travis CI.

Feedbar Error in Foreach Loop

Warning: Invalid argument supplied for foreach() in /home/blah/public_html/phppaper/feedbar.php on line 21

This is a fresh install, I have not modified anything.

Media-RSS support

Some video platform like Youtube or Peertube use the Media-RSS extension to provide metadata about the videos. They generally do not use the whole tags available, but provide tags for the video description, the thumbnail, and the number of views.

Youtube feeds appear totally empty, and Peertube feeds are missing the thumbnails. I would love if rssmonster could display the thumbnail and the description for videos for both Youtube and Peertube.

An extract of a Youtube feed:

  <media:group>
   <media:title>Can You Observe a Typical Universe?</media:title>
   <media:content url="https://www.youtube.com/v/q-6oU3jXAho?version=3" type="application/x-shockwave-flash" width="640" height="390"/>
   <media:thumbnail url="https://i2.ytimg.com/vi/q-6oU3jXAho/hqdefault.jpg" width="480" height="360"/>
   <media:description>Sign Up on Patreon to get access to the Space Time Discord! 
https://www.patreon.com/pbsspacetime

Check out the Space Time Merch Store
https://pbsspacetime.com/

The moment you started observing reality, you hopelessly polluted any conclusions you might make about it. The anthropic principle guarantees that you are NOT seeing the universe in most typical state. But used correctly, this highly controversial idea can be extremely powerful. So, how do you correctly use the anthropic principle?

Hosted by Matt O'Dowd
Written by Matt O'Dowd
Graphics by Leonardo Scholzer &amp; Adriano Leal
Directed by: Andrew Kornhaber
Executive Producers: Eric Brown &amp; Andrew Kornhaber 
...</media:description>
   <media:community>
    <media:starRating count="7709" average="4.92" min="1" max="5"/>
    <media:statistics views="176224"/>
   </media:community>http://www.rssboard.org/media-rss
  </media:group>

How it is displayed:

An extract of a Peertube feed:

<media:group>
                <media:content type="application/x-bittorrent" url="https://framatube.org/static/torrents/ed5c048d-01f3-4ceb-97db-6e278de512b0-1080.torrent" isDefault="true">
                </media:content>
                <media:content type="application/x-bittorrent" url="https://framatube.org/static/torrents/ed5c048d-01f3-4ceb-97db-6e278de512b0-720.torrent">
                </media:content>
                <media:content type="application/x-bittorrent" url="https://framatube.org/static/torrents/ed5c048d-01f3-4ceb-97db-6e278de512b0-480.torrent">
                </media:content>
                <media:content type="application/x-bittorrent" url="https://framatube.org/static/torrents/ed5c048d-01f3-4ceb-97db-6e278de512b0-360.torrent">
                </media:content>
                <media:content type="application/x-bittorrent" url="https://framatube.org/static/torrents/ed5c048d-01f3-4ceb-97db-6e278de512b0-240.torrent">
                </media:content>
                <media:rating>nonadult</media:rating>
            </media:group>
            <media:thumbnail url="https://framatube.org/static/thumbnails/ed5c048d-01f3-4ceb-97db-6e278de512b0.jpg" height="122" width="223">
            </media:thumbnail>

How it is displayed:

What do you think?

Add a license

I can't find a license file, is it possible to add one? I'm assuming it's MIT, at least that's what the composer file says.

I wanted to add RSS Monster to the awesome-selfhosted list but it requires a license.

Don't rebuild docker on every config change

Right now because the configurations are in the client and server directories, any change requires rebuilding the docker image. It would be nice to not have to do so.

Possible solutions:

• move the config to another direcory
• better use of docker (I don't know if this is possible without running into the same bug as #22)

sql server connection fails

i keep getting original: { Error: Client does not support authentication protocol requested by server; consider upgrading MySQL client at Packet.asError (C:\Users\Karth\rssmonster\server\node_modules\mysql2\lib\packets\packet.js:684:17) at ClientHandshake.execute (C:\Users\Karth\rssmonster\server\node_modules\mysql2\lib\commands\command.js:28:26) at Connection.handlePacket (C:\Users\Karth\rssmonster\server\node_modules\mysql2\lib\connection.js:449:32) at PacketParser.Connection.packetParser.p [as onPacket] (C:\Users\Karth\rssmonster\server\node_modules\mysql2\lib\connection.js:72:12) at PacketParser.executeStart (C:\Users\Karth\rssmonster\server\node_modules\mysql2\lib\packet_parser.js:75:16) at Socket.Connection.stream.on.data (C:\Users\Karth\rssmonster\server\node_modules\mysql2\lib\connection.js:79:25) at Socket.emit (events.js:198:13) at addChunk (_stream_readable.js:288:12) at readableAddChunk (_stream_readable.js:269:11) at Socket.Readable.push (_stream_readable.js:224:10) at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17) code: 'ER_NOT_SUPPORTED_AUTH_MODE', errno: 1251, sqlState: '08004', sqlMessage: 'Client does not support authentication protocol requested by server; consider upgrading MySQL client' } }
when i am trying to run the rssmonster is there anyfix to it??

Adding a category fails

The reason is in addCategory, req.body == {name: {name: '...'}}.
Probably something in the Vue template.

shall we have User mangment?

Missing feedbar?

Feedbar is not included and throwing an error

Adding a feed fails

My guess is this only happens when there are no categories because of this:

rssmonster/server/controllers/feed.js

Lines 125 to 126 in d07b9c1

	Feed.create({
	categoryId: 1,

Clear the add new feed modal when feed is selected

The new feed modal is pre populated when a feed is selected. The behavior is that the fields should be blank and only the catagory is selected.

server: Log requests

It would be nice to have a pretty log of all requests

MySQL connection fails

I'm setting up RSS Monster on a new machine, and connection to mysql is failing. It looks like the mysql client needs to be updated.
Could you please look into this?
I'm using the dockerhub image which was really simple to setup 😃

   { Error: Client does not support authentication protocol requested by server; consider upgrading MySQL client,
       at Packet.asError (/app/server/node_modules/mysql2/lib/packets/packet.js:684:17),
       at ClientHandshake.execute (/app/server/node_modules/mysql2/lib/commands/command.js:28:26),
       at Connection.handlePacket (/app/server/node_modules/mysql2/lib/connection.js:449:32),
       at PacketParser.Connection.packetParser.p [as onPacket] (/app/server/node_modules/mysql2/lib/connection.js:72:12),
       at Socket.Connection.stream.on.data (/app/server/node_modules/mysql2/lib/connection.js:79:25),
       at PacketParser.executeStart (/app/server/node_modules/mysql2/lib/packet_parser.js:75:16),
       at Socket.emit (events.js:188:13),
       at readableAddChunk (_stream_readable.js:269:11),
       at addChunk (_stream_readable.js:288:12),
       at Socket.Readable.push (_stream_readable.js:224:10),
       at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:145:17),
     code: 'ER_NOT_SUPPORTED_AUTH_MODE',,
     errno: 1251,,
     sqlState: '08004',,
     sqlMessage:,
      'Client does not support authentication protocol requested by server; consider upgrading MySQL client' } }

Fix duplicate articles showing up

By exception some duplicate articles show up in the total overview.

Refreshing feeds triggers error handler

When clicking "Refresh feeds", the error handler runs even though the crawl handler was called.
I couldn't figure out why this is the case.

Client npm audit critical issue

npm audit in client warns about a critical security issue.

Simplify configuration

I find it confusing that the sequelize conf is laying around in server/config but isn't used at all.
I think the best thing would be to change server/utils/database.js to use the sequelize config.

An advantage of using the sequelize conf would be getting 3 environments: development, test, production. You could add a DB_ENVIRONMENT variable to choose one of them.

database.js could look something like this:

// I did NOT test this

require('dotenv').load();

const sequelize_config = require("../config/config.js");

const Sequelize = require('sequelize');

const cfg = sequelize_config[process.env.DB_ENVIRONMENT];
const sequelize = new Sequelize(cfg.database, cfg.username, cfg.password, cfg);

module.exports = sequelize;

I haven't checked in the sequelize docs if there's a method to automatically load the correct configuration from its folder, so there might be a better way to do this.

Separate docker for production

Because production needs aren't the same as dev ones, I want to write a separate docker-compose file optimized for that case.
This should also ease setup for new users.

Fever API

The reason I wanted to use RSSMonster in the first place was the Fever API support.
I like having native clients, and already have some I don't want to ditch.

stuck at client installation

the program is stuck and installation and shows / extract:eventsource: sill extract [email protected] extracted to C:\Users\Karth\rssmonster\client\node_modules\.staging\eventsource-93c5d94f (286606ms)

Public demo

Please set up a public demo somewhere so potential user can evaluate the software.