[Bug] App version 1.4.0 crashes on start about contactdiary

[apozas]

Oh wow! That is definitely not supposed to happen. Can I ask you a bit more information? Namely:

• Where did you download the app from (F-Droid, Play Store, this repo)?
• Was it a clean install, or an update? (I can guess this one, but just to make it sure)
• Do you remember what was the app version before the update?
• If you try to open again after the crash, does it open? (this one I can also guess it, but again, just in case)
• Do you know how to do a logcat? Probably this information will be useful in debugging what's going wrong.

Thank you for your patience, and apologies for the inconvenience.

[riuri]

Where did you download the app from (F-Droid, Play Store, this repo)?

F-Droid

Was it a clean install, or an update? (I can guess this one, but just to make it sure)

Update

Do you remember what was the app version before the update?

I don't remember.

If you try to open again after the crash, does it open? (this one I can also guess it, but again, just in case)

It crashed again on all the times I tried opening again.

Do you know how to do a logcat? Probably this information will be useful in debugging what's going wrong.

Done with Error filter which has interesting information

01-13 12:00:02.758  7169  7169 E AndroidRuntime: FATAL EXCEPTION: main
01-13 12:00:02.758  7169  7169 E AndroidRuntime: Process: com.apozas.contactdiary, PID: 7169
01-13 12:00:02.758  7169  7169 E AndroidRuntime: java.lang.RuntimeException: Unable to start activity ComponentInfo{com.apozas.contactdiary/com.apozas.contactdiary.MainActivity}: android.database.sqlite.SQLiteException: duplicate column name: Masks (code 1): , while compiling: ALTER TABLE ContactDB ADD COLUMN Masks TINYINT
01-13 12:00:02.758  7169  7169 E AndroidRuntime: #################################################################
01-13 12:00:02.758  7169  7169 E AndroidRuntime: Error Code : 1 (SQLITE_ERROR)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: Caused By : SQL(query) error or missing database.
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	(duplicate column name: Masks (code 1): , while compiling: ALTER TABLE ContactDB ADD COLUMN Masks TINYINT)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: #################################################################
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2957)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3032)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.app.ActivityThread.-wrap11(Unknown Source:0)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1696)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.os.Handler.dispatchMessage(Handler.java:105)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.os.Looper.loop(Looper.java:164)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.app.ActivityThread.main(ActivityThread.java:6944)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at java.lang.reflect.Method.invoke(Native Method)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: Caused by: android.database.sqlite.SQLiteException: duplicate column name: Masks (code 1): , while compiling: ALTER TABLE ContactDB ADD COLUMN Masks TINYINT
01-13 12:00:02.758  7169  7169 E AndroidRuntime: #################################################################
01-13 12:00:02.758  7169  7169 E AndroidRuntime: Error Code : 1 (SQLITE_ERROR)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: Caused By : SQL(query) error or missing database.
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	(duplicate column name: Masks (code 1): , while compiling: ALTER TABLE ContactDB ADD COLUMN Masks TINYINT)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: #################################################################
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteConnection.nativePrepareStatement(Native Method)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteConnection.acquirePreparedStatement(SQLiteConnection.java:1096)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteConnection.prepare(SQLiteConnection.java:661)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteSession.prepare(SQLiteSession.java:588)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteProgram.<init>(SQLiteProgram.java:59)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteStatement.<init>(SQLiteStatement.java:31)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteDatabase.executeSql(SQLiteDatabase.java:2109)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteDatabase.execSQL(SQLiteDatabase.java:2039)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at com.apozas.contactdiary.FeedReaderDbHelper.onUpgrade(FeedReaderDbHelper.kt:47)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteOpenHelper.getDatabaseLocked(SQLiteOpenHelper.java:299)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.database.sqlite.SQLiteOpenHelper.getWritableDatabase(SQLiteOpenHelper.java:194)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at com.apozas.contactdiary.MainActivity.restrict15LastDays(MainActivity.kt:294)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at com.apozas.contactdiary.MainActivity.onCreate(MainActivity.kt:60)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.app.Activity.performCreate(Activity.java:7183)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1220)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2910)
01-13 12:00:02.758  7169  7169 E AndroidRuntime: 	... 9 more

[apozas]

Indeed, that is very useful information. The error seems to be when migrating to the new database format, it's trying to generate a column that is already there. This is very weird, since (a) I did something very wrong, or (b) something went wrong when first opening the app, and the modification of the database stopped at the middle of the process. (a) seems a bit unlikely, because otherwise I would have expected many more people facing this problem (unless nobody uses the app any more 😅).

Is your phone rooted? If so, I know a workaround that should fix this in your case. If not, I need to keep thinking a bit more. I'm really sorry for the hassle.

[riuri]

No, I don't have root unfortunately.

[apozas]

So, here is what I have been thinking of. On one hand, I am quite confident that uninstalling and reinstalling will solve the problem, but at the cost of losing all the information you had stored. I believe that you would like to avoid that, so I think there is a second option that could work, and should not result in information loss. This second option requires running adb commands to pull the database from the phone to your computer, (maybe) editing it as needed in the computer, and pushing it back to the phone. Would you be up to trying this one?

[riuri]

Yes, let's try the adb route first if possible.

[apozas]

Alright, let's see.

First step: Pull the database from the phone to a computer.

Connect your phone to the computer, and run adb backup -f contactdiary.ab com.apozas.contactdiary. You will be asked to unlock the phone and confirm the operation there. This will produce the file contactdiary.ab in your root directory (/home/$USER on Linux/Mac or I guess C:/ on Windows)

Now to convert the .ab file into a compressed folder, run

dd if=contactdiary.ab bs=1 skip=24 | python -c "import zlib,sys;sys.stdout.write(zlib.decompress(sys.stdin.read()))" > contactdiary.tar

on Linux, or use the Android Backup Extractor tool on Windows.

Unzip the file, and get the file in apps/com.apozas.contactdiary/db/ContactDiary.db. This is the file that stores the database. Make a copy of it, just in case (call this copy, for instance ContactDiary-Copy.db). Also, you may later need ContactDiary.db-wal and ContactDiary.db-shm, so keep them close.

Second step: Look at the database, and modify if required.

In order to explore the file, you'll need a tool like DB Browser for SQLite. Open it, and in the main view there will be four tabs: Database Structure, Browse Data, Edit Pragmas, and Execute SQL.

Click on Edit Pragmas, and go to User Version. I would expect that there you will see a number, smaller than 5 (which one is it, by the way?). Set it to 5.

Now, click on Browse Data. Here, you should see all the entries you have been writing in the app. You should check two things here: (1) if you see that the two last columns are Masks and Ventilation, and (2) if the column CloseContact does not contain numbers other than -1, 0 or 1. If (1) and (2) are true, then you can skip to the third step.

If (1) is not true (from the error message you posted, I assume that you should see, at least, the Masks column), click on the Execute SQL tab, in the field write

ALTER TABLE ContactDB ADD COLUMN Ventilation TINYINT

and press the blue, right pointing triangle (or Ctrl + R) to run the command.

If (2) is not true, it's nothing to worry about, but the entries will not have the appropriate information about distance being kept. In order to put it in the right form, you should change all 1s to 0s, and all 3s to 1s. You can leave the -1s that you see untouched, and if you see other numbers, let me know (but this should not happen).

Press the button Write Changes (the button above Edit Pragmas and Execute SQL), and let's go back to the phone.

Third step: remove the files in the phone, and push the modified database.

First, in the phone settings, select the app and Force Close it.
Back to the computer, run the following commands

adb shell run-as com.apozas.contactdiary rm databases/ContactDiary.db

adb shell run-as com.apozas.contactdiary rm databases/ContactDiary.db-wal

adb shell run-as com.apozas.contactdiary rm databases/ContactDiary.db-shm

These three will remove the database and temporary files from the phone. The .db-wal and .db-shm are temporary files, so they are safe to delete. But you kept copies of it in the first step just in case.

Next, copy the modified ContactDiary.db to the phone via

adb push ContactDiary.db /data/local/tmp

And finally, place it in the appropriate folder with

adb shell run-as com.apozas.contactdiary cp /data/local/tmp/ContactDiary.db databases

Now, the app should open without any problem (fingers crossed).

Finally, if anything goes wrong, just copy the backup .db, .db-wal and .db-shm files back to the phone using the last two commands, and at least we will be in the starting point to try something else.

Hope this helps. Let me know of any doubts you may have.

[riuri]

I had to use the android backup extractor abe on linux because the backup was encrypted.

There was a ContactDiary.db-journal as well which might indicate that something didn't finish. No -shm nor -wal present. I removed the journal before proceeding.

A redacted sqlite session follows:

sqlite> pragma user_version;
3
sqlite> pragma user_version=5;
sqlite> pragma user_version;
5
sqlite> .schema ContactDB
CREATE TABLE ContactDB (_id INTEGER PRIMARY KEY,Type TEXT,Name TEXT,Place TEXT,Timestamp INT,Phone TEXT,Relative TINYINT,Companions TEXT,EncounterType TINYINT,CloseContact TINYINT,Notes TEXT, Duration INT);
sqlite> ALTER TABLE ContactDB ADD COLUMN Masks TINYINT;
sqlite> ALTER TABLE ContactDB ADD COLUMN Ventilation TINYINT;
sqlite> .schema ContactDB
CREATE TABLE ContactDB (_id INTEGER PRIMARY KEY,Type TEXT,Name TEXT,Place TEXT,Timestamp INT,Phone TEXT,Relative TINYINT,Companions TEXT,EncounterType TINYINT,CloseContact TINYINT,Notes TEXT, Duration INT, Masks TINYINT, Ventilation TINYINT);
sqlite> SELECT CloseContact FROM ContactDB;
sqlite> -- All 3 and 1
sqlite> .quit

Then I proceeded to the command-line and ran:

$ adb shell run-as com.apozas.contactdiary rm databases/ContactDiary.db
run-as: package not debuggable: com.apozas.contactdiary

But at this point I do have access to the information so I cleared the data and input it again on the app.

To avoid a similar problem from happening to other users (that may not be thinking on filling a bug report), I believe you can use BEGIN TRANSACTION and END TRANSACTION on sqlite for atomic changes. I don't know if those include the pragma user_version but it's worth checking.

[apozas]

Thanks riuri for your patience and for the details. So, from this

But at this point I do have access to the information so I cleared the data and input it again on the app.

I understand that you have the app working, by clearing the data and introducing everything manually, right? I am happy that you have the app working, but very sorry that you had to do this, in the end.

There are a few things that I can read from the sqlite session that are a bit intriguing. The most important one is that the column Masks is not present in the table, which is very weird given the error trace that you obtained. Also, the version of the app prior to updating was 1.2.0, as evidenced by the user_version and the columns themselves (there is no EndTime column either). I don't know if it's the fact of jumping two versions of the database could have caused this, although the code is built for it not to be the case.

I appreciate the tip on the BEGIN_TRANSACTION and END_TRANSACTION. I will look into it, because I had not seen it almost anywhere when I was looking for SQL queries on Android.

In any case, I am happy that you didn't lose the information, but very sorry for all this hassle. Thank you very much for your patience.