vitormarcal / chatvault

WhatsApp convo backups.. Features directory import, auto email backup, and a user-friendly frontend. View messages effortlessly.

chatvault

Chat Vault is a Kotlin Spring Boot application designed to store backups of WhatsApp conversations from various sources, such as API imports, email, and directory monitoring, and provide easy access to these conversations through a frontend that resembles a chat application, like WhatsApp.

This project is still in development, and some features may not be fully implemented.

Key Features

• Directory importing: Place files exported from whatsapp in specific directories to be imported into the application.
• Automated Email Backup: Set up an email in ChatVault and export your messages as attachments to that email. Chat Vault identifies this email and archives the messages and attachments automatically.
• Intuitive Frontend: To facilitate access to archived messages and their attachments, Chat Vault also includes a user-friendly frontend. Easily navigate conversations, search messages, and view attachments.

How to export via the Whatsapp app

Please read the official Whatsapp FAQ.

With the imported file, to ingest it into ChatVault you can:

• map a shared folder between the ChatVault import directory and your cell phone;
• send by email for automatic import into ChatVault;
• zip the imported file and upload it to the ChatVault interface.

Repository structure

This repository is divided into two main modules. They are the modules: frontend (javascript, vue, nuxt) and backend (kotlin, java, spring boot, gradle)

Run Frontend module

The front end module is a Vue/Nuxt application and it serves what will be rendered by the browser: html, css, javascript and static assets. To run the conventional way, with npm commands, follow the Readme in the frontend directory.

You can run it with npm:

npm run dev

The frontend application will listen on port 3000 by default, unless you ran the backend application before (the backend listens on 8080), in which case the frontend will pick up a random port.

Run Backend module

You can run the backend application without an IDE:

./gradlew run

The backend application will listen to 8080 port by default.

Docker

Note that downloading container images might require you to authenticate to the GitHub Container Registry steps here. You can use compose.yml to create a database and build the frontend and backend project locally.

docker-compose -f compose.yml

There are docker image packages on github. You can download the latest image with:

docker pull ghcr.io/vitormarcal/chatvault:latest

Environment variables

For docker, the variables must be in upper case and where is "." it must be "_": some.environment.variable is like SOME_ENVIRONMENT_VARIABLE in docker

Environment variables	obs	example
Database	required
spring.datasource.url	required	jdbc:postgresql://database_host:5432/database_name
spring.datasource.username	required	user
spring.datasource.password	required	secret
--------------------------	--------------------------	---------
Email import	feat not required
chatvault.email.enabled	not required	true
chatvault.email.host	required to feat	imap.server.com
chatvault.email.password	required to feat	secret
chatvault.email.port	required to feat	993
chatvault.email.username	required to feat	someuser
chatvault.email.debug	not required	true
--------------------------		--------------------------
chatvault.host	not required	https://somehost.com ,http://localhost:3000
spring.servlet.multipart.max-file-size	not required	500MB
spring.servlet.multipart.max-request-size	not required	500MB
chatvault.msgparser.dateformat	not required but recommended	dd/MM/yyyy HH:mm

---

• If not defined chatvault.msgparser.dateformat, the application will not be able to resolve ambiguities in certain situations

chatvault's Issues

few refactorings

the title

Import chat via email scan

The user must export the chat via email and then the app must import it into the base.

Move chat navbar actions to chat config panel

The title.

Question on using the docker container?

Hi
So i was wondering if someone could she some light, So created the docker container and the web interface shows two options
I assume one is manual and the other is the info i put in the import folder in the docker container,
The question is what do i export on whatsapp to put in the import folder?

Thanks

feat filter gallery by file type

the title

(bug:backend) Message date with unexpected format prevents message import

When the date format is not what is expected by the regex, the message import does not happen because a message line is not identified.
For example, the message could come from WhatsApp with this format (and possibly others):
"16/11/2023 18:44" for Portuguese
"16/11/2023, 18:44", for French
"16/11/23, 18:44", for English
An alternative way must be found to be able to treat the date appropriately. Worst case scenario, put in as much format as possible...

allow upload of a non-existent chat

the tile

import chat via a zip file

Import chat via scan dir

The user must export the chat to a folder monitored by the application, which must import it appropriately.

(feature:frontend) improve exception handle

Show meaningful message errors

feat find all attachment messageId from gallery

the title

Frontend: Fix baseurl api

Frontend: resolves hidden chat list on desktop browsers

(feat:backend) Properly handle API errors

A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Have standard API error feedback.
Currently any RuntimeException is returning 500

Import by filesync

Import chat by file sync

Style list chat area

Because the development was guided by functionality first, the styles ended up not being developed in an ideal way, read: it's not pretty at all.
This issue is for me to remember to take the time to improve the design of the area in general.

import chat via frontend

Import a new chat or add messages to an existing chat

• Upload file to server
• style upload component

Update group profile image

Click on the image to view, when viewing an input will appear to change the image.
The backend will receive a request for an image file and chatId, and save it in the chat bucket with the name profileimage.jpg.
Front will always query this bucket for an image called profileimage.jpg.
If it does not exist, it will return a default image.

Frontend: Fix sort messages items

Each bulk message is reversed.

feat(frontend) add lazy loading images

Add lazy loading images on click gallery

Create a media gallery

A way to view or list all media present in the chat without having to scroll through the message history

(feat:frontend) add loading request status

Add loading request status

Edit message from the interface

Frontend: List messages by chat

Create frontend gui to list messages by chat

Remove legacy import feature

Remove legacy feature import.

(feat:frontend) store some data in localstorage

Page size, active author..

Doc: Update Readme

Update Readme

Infra: Create Dockerfile

Create docker file.

feat download all chats

Download all chats in the same format as exported by WhatsApp. This way, the data can be used in other applications or reinjected when necessary.

(feat:frontend) (feat:backend) Create authentication and authorization features

the title says it all

Remove legacy chat import

ChatVault is a rewrite of another app created in Node. That's why I created some APIs to facilitate the migration from the old one to this one.
As I was basically the only user of the old app, I am deleting the dead code since I no longer need this functionality.

(feat:frontend) add icon to open or close the chat configuration

A simple icon to open or close the chat configuration, the functionality already exists by clicking on the chat bar but it is not intuitive

Frontend: Toggle chat view on mobile devices

When on a mobile device, when clicking on a chat, the chat list should be hidden and the selected chat should occupy the screen, when clicking on the selected chat profile arrow, the selected chat should be hidden and then the chat list chat should appear again.

(feat:backend) deduplicate messages when importing

When importing messages, check if they already exist in the database and import only new ones.
It must be something performative, so a one-on-one search of the bank should not be the solution. Maybe checking older messages and doing some list cutting might work:

1. Check the oldest message, does it exist in the bank? Yes
2. check the newest one, is it available in the bank?
   2.a yes, then ignore import.
   2.b no, so take the message from the middle of the list, it exists in the bank, yes, discard the previous messages as they are in the bank and this is now the oldest, go back to step 1 until only new messages remain.
   Another way would perhaps be a checksum/hash form but I don't know if it's viable yet.

INFRA: remove dependency of the frontend build on the backend java module

the title

Config frontend module

A frontend directory module with all the source code and assets.

Create a lateral area to chat config

Remove action options from message area and create a lateral area for them

Update to nuxt 3

Fix set fileType from email handler txt to text

the title

Fix code scanning alert - Cross-site scripting

Tracking issue for:

• https://github.com/vitormarcal/chatvault/security/code-scanning/1

Improve dockerfile cache

At the backend stage, removing the frontend folder from the COPY improves the build experience as the frontend folder will not invalidate the backend cache

Frontend: List all chats

Create a frontend gui to list all chats

(feat:backend) add i18n in message regex

When analyzing messages, the application looks for a pattern to know whether the text file is a message file. Should I leave it flexible to add a pattern from another language

(feat:backend) enable configuration of the bucket name for each chat group.

enable configuration of the bucket name for each chat group.
It is currently a uuid but it might not be interesting to leave it hardcoded

Download attachments via API

Allow access to files via API, so that it can be made available in a future frontend.

The API must inform the chatId and messageId.
Just the messageId would work, but having both IDs makes it a little harder for a bad actor to scan the files if they gain access.

Backend: Export chat and attachments

To keep data independent of the platform, it is important to have a way to export chats and media to a .zip file

I imagine the functionality as follows: search for all messages by chatId, transform them into message lines and join them in the same way that wahtsapp originally exports. The text file will be compressed together with the media into a zip with the name of the chat.

Wrong message date in chat list area and message navigation bar area

The date is formatted showing one month before the current date:
2023-11-08T16:03:00 shows : 08/10

Create Dockerfile

Create a docker file build

Todo chat import

Allow import a chat with todo name. Ex:
Todo imported at 2023-05-01
Why:
When there is an automatic import, whether via file synchronization or email import, the program does not know the name or who the chat is from, so having a way to import and signal that it is a TODO is useful. In the future, I will develop message delivery from a TODO (or another chat, if imported with error) to another chat.

Allow Set page size frontend

allow changing message request page size parameter