csafe-isu / handwriterapp

README

Handwriter is designed to assist forensic examiners by analyzing handwritten documents against a closed set of potential writers. It determines the probability that each writer wrote the document. Whether you are a forensic document examiner, legal professional, academic, or simply curious about how statistics are applied to handwriting, handwriter provides an automated way to evaluate handwriting samples.

VIEW A DEMO

View a demo of handwriter on handwritten documents from the CSAFE Handwriting Database. Go to demo.

SIMULATE CASEWORK WITH EXAMPLE DOCUMENTS

INSTALLATION

Handwriter requires R, RStudio IDE, and JAGS.

• Install R and RStudio from POSIT
• Install JAGS from SourceForge

LAUNCH THE APP

Open RStudio, navigate to the console window, and type:

install.packages("devtools")
devtools::install_github("CSAFE-ISU/handwriterApp")
library(handwriterApp)
handwriterApp()

In the pop-up window, click Open in Browser. If you use the app in the pop-up window instead of in a browser, some links will not work.

Click Simulate Casework and follow the instructions in the app.

handwriterapp's Issues

Add tests to package

Ask for feedback on app

Comments from Jeni

Below are my comments/edits:

Overall: Is there eventually going to be a footer added to match the look of the other apps?

Page 1, main paragraph: I’m suggesting a rewrite of the second sentence, see the 2 bullet points below. It just sounds a bit odd starting that sentence with “Made to aid forensic examiners…” and it was a little long. I split that second sentence into two sentences.
Option 1: This tool is designed to assist forensic examiners by analyzing handwritten documents against a closed set of potential writers. It determines the probability that each writer wrote the document.
Option 2: This tool assists forensic examiners by analyzing handwritten documents against a closed set of potential writers. It determines the probability that each writer wrote the document.

Page 1, sidebar: I think you could remove the word “following” and it would still make sense. Change “start” to “begin” to match the button text. And change “app” to “handwriter”. Press the button to begin using handwriter to compare a questioned document to known samples.

Starting on Page 5, “Known Writing” Sidebar: Is there an extra space or too much space between the sentence “Select three known…” and the Browse box input area? Ignore this comment if this was the intended look.

Page 17, second paragraph:
I’m not sure what the naming convention is, but is there a reason why “App” is included in handwriter’s name? If it is needed, I would remove “application” because it sounds kind of redundant with App and application back-to-back.
A suggested sentence rewrite: The methods used in the handwriter package are open-source and transparent, as described in Crawford, Berry, and Carriquiry (2020) and Crawford, Ommen, and Carriquiry (2023).

Page 18: Remove “The” before CSAFE’s name

Release handwriterApp 1.0.0

First release:

• usethis::use_news_md()
• usethis::use_cran_comments()
• Update (aspirational) install instructions in README
• Proofread Title: and Description:
• Check that all exported functions have @return and @examples
• Check that Authors@R: includes a copyright holder (role 'cph')
• Check licensing of included files
• Review https://github.com/DavisVaughan/extrachecks

Prepare for release:

• git pull
• urlchecker::url_check()
• devtools::check(remote = TRUE, manual = TRUE)
• devtools::check_win_devel()
• git push

Submit to CRAN:

• usethis::use_version('major')
• devtools::submit_cran()
• Approve email

Wait for CRAN...

• Accepted 🎉
• usethis::use_github_release()
• usethis::use_dev_version(push = TRUE)
• usethis::use_news_md()

Give the user progress updates when processing documents

Right now, when the user selects known writing samples, handwriter processes the samples behind the scenes but all the user sees is the spinner. The user might appreciate seeing a notification every time a sample is processed.

Separate Demo and Casework Simulation Use Cases

The Problem

Amy Crawford found that cluster templates with K=40 clusters yield the highest accuracy when the number of known writers is 100+. The drawback is that fitting a model and analyzing questioned documents with this many known writers takes several hours, so this scenario is unsuitable for a handwriter demo.

I tried using a template with K=40 clusters and only 5 known writers. Fitting the model was much faster, but the accuracy on the questioned documents was terrible: 0%! The model training documents only used 22 of the K=40 clusters, but around 1/6 of the graphs in the questioned documents fell into the other 18 clusters. This meant that a large portion of the questioned documents' data was thrown out and not used to estimate the questioned writers' profiles.

I found that if I used a smaller cluster template with K=5 or K=8 clusters, and 5 known writers, the model achieved 100% accuracy on two questioned documents. However, the model tanked on the 20 other questioned documents that I tested. We could use this small cluster template, these known writers, and the 2 "successful" questioned documents as handwriter demo data because the model and analysis run quite fast. However, we don't want to allow users to use the small cluster template on other data because the results are likely to be quite poor.

The Solution

Give users the option to see a demonstration of handwriter with data that we provide them or allow users to analyze their own data to simulate casework.

Option 1: Demo

Handwriter will use the small cluster template, the 5 known writers, and the 2 questioned documents. Users won't have the option to select their own data.

Option 2: Casework Simulation

Handwriter will use a template with K=40 clusters. Users will upload their own data but be required to use at least 100 known writers. We can also include a link to the CSAFE Handwriting Database if they would like to download data.

Bug with questioned document tab

If the user fits a model and then starts a new session, an error occurs if the user tries to process the questioned document without setting the start on stop characters on the known writer tab.

Change app to allow the user to analyze more than one questioned document at a time

Right now the app only allows a user to analyze a single questioned document at a time. I made this choice initially to make the Questioned Documents tab display simpler. Users might want to analyze multiple questioned documents for the same set of known writers and handwriter allows this, so the app should be updated.

Convert app to an R package

Follow the instructions in https://mastering-shiny.org/scaling-packaging.html chapter 20.1 to convert the app to an R package.

Incorporate feedback

TO DO

• Add a screen before Setup that gives people the option to download example data from https://drive.google.com/drive/folders/1_FVKu6tH06w25zNwDR-xrXe_z9u0Epua?usp=sharing
• Add instructions on required naming convention and prompt user to rename files before the Known and Questioned screens
• Add helper text for all plots
• Add the known writer profile with the highest posterior probability to the QD profile plot

Update app to use handwriter 3.1.0