Convert messages from a Signal SQLite database export to Markdown.
Unlike my signal_md which requires output from signald
, this one requires nothing beyond this Python script, configuration, and a tool to export the DB.
I probably should've called it signal_sqlite_csv_md
because the script doesn't read directly from the SQLite DB, instead it parses a CSV export from it. I tried directly accessing the DB and gave up ๐คฃ
A big shoutout to Florian Engel whose post [1]
saved me hours ๐ค.
The SQLite DB is encrypted but it's easy to decrypt because you have the key!
The attachments are not in the DB, they're stored in the file system in a series of folders with 2 digit Hex labels. The files have names like "000ec9a54abe93416284f83da2f9f8d124778f22191d9422ed9829de2b22c1b7
" with no suffix but don't worry, that info is in the DB and the script takes care of adding the suffix e.g. ".jpg
".
Another approach would be to query the SQLite DB directly on device but that's a future thing for me. A good reference is https://github.com/idanlevin/signal-query
The code in this repo relies heavily on my message_md classes which contain generic Message
, Person
, Group
and other classes and the methods to convert messages to Markdown files. Be sure to read the README
and the configuration guide for that repo first.
- Extracting Messages from Signal Desktop by Florian Engel guided my way
- DB Browser for SQLite to get your data
- message_md upon which this tool depends
- Get your Signal data
- Configure this tool
- Run this tool
- Be happy
The tool needs two files of date exported from Signal's SQLite database:
messages.csv
- the actual messagesconversations.csv
- the mapping ofconversation-id
to people and groups
This steps below desribe how to get these two sets of data out of Signal.
Steps
- Install DB Browser for SQLite - [2]
- NOTE: I had to try multiple older versions before I got one that would open the file
- the one that worke for me was v3.13 from https://nightlies.sqlitebrowser.org/win64-prerelease/
- Find the key to your SQLite DB, see [1]
- For me, on Windows, with user
micro
it was here:C:\Users\micro\AppData\Roaming\Signal\config.json
- For me, on Windows, with user
- Find the path to your Signal
db.sqlite
database file- For me, it was here:
C:\Users\micro\AppData\Roaming\Signal\sql\
- For me, it was here:
- Launch "DB Browser for SQLite (SQLCipher)" -- not the one without
(SQLCipher)
- Click "Open Database"
- Choose
Raw key
from the menu to the right of the "Password" field - In the "Password" field, type
0x
and then paste the key you found in step 2 - Right click on "messages" and click "Export as CSV file"
- Right click on "conversations" and click "Export as CSV file"
- Find the attachments
- Mine were under:
C:\Users\micro\AppData\Roaming\Signal\attachments.noindex
- Mine were under:
- Copy the attachments to the same folder (no subfolders) as the CSV file
- the
cp_signal_attachments.sh
shell script made it easier for me - I had to use
dos2unix
on that shell script file before it worked - NOTE: I can improve this later to get the files directly, being lazy!
- the
The next step is to configure this tool.
You'll need to define each person that you communicate with in people.json
and the groups in groups.json
. This way the tool can associate each message with the person that sent it and who it was sent to.
Samples of these configuration files are in the message_md
repo here upon which this tool depends.
This part is tedious the first time and needs to be updated when you add new contacts or Groups in Signal, i.e. a pain.
Someday I can automate this but for now, no pain, no gain ๐.
The next sections describes how people are identified and where to find the identifiers for groups.
People are found in the conversations.csv
file via their phone number in the e164
column. The conversations.csv
file is assumed to be under the source folder
- Open the
conversations.csv
file in your favorite editor - Look at the first row
id,json,active_at,type,members,name,profileName,profileFamilyName,profileFullName,e164,serviceId,groupId,profileLastFetchedAt
- If there's a
groupId
field value on a given row, then that's a group- the
name
field will tell you the name of the group
- the
""id"":""a1760c87-d3d0-40f6-9992-ac0426efcc14""
""groupId"":""FdibKUgQIZPilWQu3jbgEB+tajc3RUKuoyYNZp4bRhQ=""
""name"":""Family"
-
Add the corresponding row to
groups.json
:- set group
id
to theid
fromconversations.csv
- set the
conversation-id
to thegroupID
from Step 3 - set
slug
to a one-word or hyphenated keyword (slug
) for this group - set
description
to thename
either the name fromconversations.csv
or something else e.g. "Family"
- set group
-
Repeat Steps 3 and 4 for every row
Once you have the two CSV export files and you your people.json
and groups.json
configured, you're finally ready to run this tool.
The command line options are described in the message_md repo.
Example:
# python3 signal_sqlite_md.py -c ../../dev-output/config -s ../../signal_sqlite/ -f messages.csv -d -o ../../dev-output -m spongebob -b 2023-12-20
where:
c
onfig settings are in../../dev-output/config
s
ource folder is../../signal_sqlite
f
ile of CSV messages ismessages.csv
in thes
ource foldero
utput the Markdown files to../../dev-output
m
y slug isspongebob
b
egin the export from2023-12-20
In the messages.csv
file, the attachments are referenced in this part of the message
""path"":""0b\\0b82ab19cb4cab30f5041f7705aa890833cab2c32d662c2792814e0268c90e6c""