thephm / sms_backup_md Goto Github PK

SMS Backup and Restore Tool to Markdown converter

License: Apache License 2.0

Python 100.00%

sms_backup_md's Introduction

sms_backup_md

This tool takes the XML output file from the Android SMS Backup and Restore Tool and generates a set of Markdown files and corresponding attachment files on your filesystem.

See the docs\guide.md for more information.

message_md dependency

The code in this repo relies heavily on my message_md classes which contain generic Message, Person, Group and other classes and the methods to convert messages to Markdown files. Be sure to read the docs for that repo first. I abstracted them from the older sister (first child) tool signal_md.

Make sure to install libxml e.g. sudo apt-get install python-lxml

If that gave an issue, I also had to pip install --upgrade lxmls

How to use it

If your name is bob and you want to convert all messages since 1990-01-01

python3 sms_backup_md.py -m bob  -c ../../dev-output/config -f ../../sms/sms-20240505003555.xml -o ../../dev-output -d -b 1990-01-01

License

Apache License 2.0

sms_backup_md's People

Contributors

Stargazers

Watchers

sms_backup_md's Issues

Group MMS messages always show me as the sender

Bug where for MMS messages between my wife and another person were always showing me as the sender.

Latest change broke adding of some messages

Turns out that the imdn_message_id (SMS) and m_id (MMS) is not always there. The code wasn't allowing a non-existent value.

SMS Backup & Restore creates two MMS messages

Seeing the same media message twice in the SMS Backup & Restore

The first entry:

date="1708898378000" which is Feb 25, 2024, 9:59:38 PM
ct_t="null"
mms.parts.cl="null"
mms.parts.part.name="2222712841905038343.png"

<mms date="1708898378000" spam_report="0" ct_t="null" msg_box="1" address="+12895551212" sub_cs="null" re_type="0" retr_st="null" re_original_body="null" d_tm="null" exp="604800" locked="0" msg_id="0" app_id="0" from_address="null" m_id="MxgGFNuX-CTa6QzX=Thm3doQ" retr_txt="null" date_sent="1708898376" read="1" rpt_a="null" ct_cls="135" bin_info="0" pri="129" sub_id="1" re_content_type="null" object_id="null" resp_txt="null" re_content_uri="null" ct_l="null" re_original_key="null" d_rpt="129" reserved="0" using_mode="0" _id="1535" rr_st="0" m_type="132" favorite="0" rr="129" sub="null" hidden="1" deletable="0" read_status="null" d_rpt_st="0" callback_set="0" seen="1" re_recipient_address="null" device_name="null" cmc_prop="null" resp_st="null" text_only="0" sim_slot="0" st="null" retr_txt_cs="null" creator="com.samsung.android.messaging" m_size="403905" sim_imsi="null" correlation_tag="null" re_body="null" safe_message="0" tr_id="" m_cls="personal" v="18" secret_mode="1" re_file_name="null" re_count_info="null" readable_date="Feb 25, 2024 4:59:38 p.m." contact_name="Bob">
    <parts>
      <part seq="0" ct="image/png" name="2222712841905038343.png" chset="null" cd="null" fn="null" cid="null" cl="null" ctt_s="null" ctt_t="null" text="null" sef_type="0" decorate_bubble_value="null" data="iVBORw0KGgoAAAANSUhEUgAABD..."
    </parts>
    <addrs>
      <addr address="+12895551212" type="137" charset="106" />
    </addrs>
  </mms>

The second entry I think is the read acknowledgement or the end of the sending of the message? It has no field mms.parts.name like the above record and two fields (ct_t and mms.parts.part.cl) are now filled in:

date="1708898378030" which is Feb 25, 2024, 9:59:38 PM (30 milliseconds later?)
ct_t="application/vnd.wap.multipart.related"
mms.parts.part.cl="2222712841905038343.png"
read="1"

  <mms date="1708898378030" msg_box="1" read="1" status="3" ct_t="application/vnd.wap.multipart.related" m_type="132" text_only="0" m_size="403905" m_id="MxgGFNuX-CTa6QzX=Thm3doQ" address="+12895551212" s_chat="1" readable_date="Feb 25, 2024 4:59:38 p.m." contact_name="Bob">
    <parts>
      <part seq="0" ct="image/png" cl="2222712841905038343.png" data="iVBORw0KGgoAAAANSUhEUgAABD..."
    </parts>
    <addrs>
      <addr address="+12895551212" type="137" />
    </addrs>
  </mms>

Ignore messages from phone numbers 5 or less characters long

To exclude messages from short codes like car dealerships, 611, or 2FA code generators, drop messages with phone numbers less <= 5 characters long

SMS backup creates two messages for Android phones

I noticed some messages with the same person show up twice in the SMS Backup Tools XML file, one <sms and one <mms

Example:

<sms date="1711148075900" type="2" body="Dinner" imdn_message_id="afcc7c46-a900-4e31-be69-917bb9d4074a" updated_timestamp="1711229182172" delivered_timestamp="1711148078463" transaction_id="1412919488860956" date_sent="1711148076650" read="1" sub_id="1" message_type="30" recipients="+12895551212" status="3" displayed_counter="1" seen="1" content_type="text/plain" creator="com.samsung.android.messaging" address="+12895551212" remote_uri="tel:+12895551212" session_id="999cbc7d52859d5438f9a97a839720f9e62970c0" service_type="1" display_notification_status="2" s_chat="1" readable_date="Mar 22, 2024 6:54:35 p.m." contact_name="Bob" />

<mms date="1711148075000" spam_report="0" ct_t="application/vnd.wap.multipart.related" msg_box="2" address="+12895551212" sub_cs="null" re_type="0" retr_st="null" re_original_body="null" d_tm="null" exp="604800" locked="0" msg_id="0" app_id="0" from_address="null" m_id="afcc7c46-a900-4e31-be69-917bb9d4074a" retr_txt="null" date_sent="1711148076" read="1" rpt_a="null" ct_cls="135" bin_info="0" pri="129" sub_id="1" re_content_type="null" object_id="null" resp_txt="null" re_content_uri="null" ct_l="null" re_original_key="null" d_rpt="129" reserved="0" using_mode="0" _id="1773" rr_st="0" m_type="128" favorite="0" rr="129" sub="null" hidden="1" deletable="0" read_status="null" d_rpt_st="0" callback_set="0" seen="1" re_recipient_address="null" device_name="null" cmc_prop="null" resp_st="null" text_only="1" sim_slot="0" st="null" retr_txt_cs="null" creator="com.samsung.android.messaging" m_size="6" sim_imsi="null" correlation_tag="null" re_body="null" safe_message="0" tr_id="" m_cls="personal" v="18" secret_mode="1" re_file_name="null" re_count_info="null" readable_date="Mar 22, 2024 6:54:35 p.m." contact_name="Bob">
  <parts>
    <part seq="0" ct="application/smil" name="null" chset="null" cd="null" fn="null" cid="&lt;smil&gt;&gt;" cl="smil.xml" ctt_s="null" ctt_t="null" text='&lt;smil&gt;&lt;head&gt;&lt;layout&gt;&lt;root-layout/&gt;&lt;region id="Text" top="0" left="0" height="100%" width="100%"/&gt;&lt;/layout&gt;&lt;/head&gt;&lt;body&gt;&lt;par dur="5000ms"&gt;&lt;text src="text000001.txt" region="Text" /&gt;&lt;/par&gt;&lt;/body&gt;&lt;/smil&gt;' sef_type="0" decorate_bubble_value="null" />
    <part seq="0" ct="text/plain" name="null" chset="106" cd="null" fn="null" cid="&lt;text.000001&gt;" cl="text.000001.txt" ctt_s="null" ctt_t="null" text="Dinner" sef_type="0" decorate_bubble_value="null" />
  </parts>
  <addrs>
    <addr address="+12895551212" type="151" charset="106" />
  </addrs>
  </mms>

Messages with MMS images not placed in the same file as the conversation

Seeing MMS images in files separate from the other messages instead of in the same file.

These seperate files also don't have my slug in the people field e.g. bob

---
tags: [chat]
people: [, paul-finn]
date: 2024-07-13
time: 15:09
service: sms
---

Bob at 15:09
[[20240713_140056.jpg]]

Incomplete handling of MIME types

In MMS messages, anything that was an image is being mapped to .jpg unless the MIME type was plain\text

When I ran it today with messages from an iPhone, there were video/3gpp video attachments so those didn't parse. There was also a text/x-vcard attachment.

Needs to work with all MIME types and ignore application/smil