iansan5653 / open-mcr

:pencil: Exam bubble sheet scorer. Created with OpenCV and Python.

Free and Open-Source Multiple Choice Exam Reader

Warning As per the license of this software, no warranty is implied. The software is stable but there still may be bugs. Given that students' grades are at stake, please be sure to audit the results - particularly when working with low-quality scans.

Background

Commercially available OMR (optical mark recognition) exam sheets, scanners, and processing software can cost educators and educational institutions thousands of dollars per year. In response to this, OpenMCR has been developed as a free and easy-to-use alternative. The tool includes a multiple choice exam sheet and works with any scanner and printer.

This software and the corresponding multiple choice sheet were developed as an independent study project by Ian Sanders, a mechanical engineering student at the University of South Florida, under the direction of Dr. Autar Kaw.

For a detailed discussion of the algorithm and features in use, please read the report as submitted for the original independent study course.

Installation Instructions

Currently, packaged executables are only provided for Windows. For other operating systems, see Running From Source.

Windows Installation

To install the utility, simply download the latest .exe file from from the releases page and install it. After installation, check your Start menu for a shortcut.

Running Without Install

For users who cannot or do not want to install the software on their machine, ZIP file packages are also available for each release. Download the file, extract it anywhere on your machine, and run the main.exe file. This method does not require administrator priveleges.

Running From Source / CLI

If you wish to customize the software, use the command line interface, or run it on a non-Windows device, you can run the Python program directly from the source files. This requires Python and Pip to be installed on your machine.

1. Clone the reposotory using Git, or download and extract the latest Source code (zip) file from releases.
2. Open a terminal / command prompt in the extracted directory.
3. On Mac machines with fresh Python installations, you will to update TKinter from the default.
4. Run pip3 install -r requirements.txt to install dependencies.
5. Run python3 src/main.py for the CLI interface or python3 src/main_gui.py for the graphical interface.

Note: On MacOS, if you see a black screen when running the GUI, you need to uninstall Python, install TKinter, and then reinstall Python. The easiest way to do this is to manage your installation with Homebrew as described here.

Note: On Linux machines, you may see an error message that opencv or tkinter are not found. If you see this, install those dependencies by running sudo apt-get install python3-opencv python3-tk and then try again.

Codespaces

For development, a pre-made environment is available in Codespaces:

Note

The configuration auomatically installs the desktop-lite feature for running the GUI. This environment can be accessed in your browser through the forwarded 6080 port. The desktop password is vscode.

Printable Multiple Choice Sheet

The multiple choice sheet that must be used with this software is available for printing here:

• 75 Question Variant
• 150 Question Variant

Usage Instructions

Run the program by finding it in your Start menu after installing.

For full operating instructions, see the Manual or click the Open Help button in the software.

Feedback / Bug Reports

To report a bug, request a new feature, or provide feedback, please file an issue.

License

Software License

Copyright (C) 2019 Ian Sanders

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

For the full license text, see license.txt.

Multiple Choice Sheet License

The multiple choice sheet distributed with this software is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license (CC BY-NC-SA 4.0). In summary, this means that you are free to distribute and modify the document so long as you share it under the same license, provide attribution, and do not use it for commercial purposes. For the full license, see the Creative Commons website.

Note: You are explicitly allowed to distribute the multiple choice sheet without attribution if using it unmodified for educational purpose and not in any way implying that it is your own work. This is an exception to the Creative Commons terms.

open-mcr's Issues

Add support for analyzing results using the MC Test Analysis software

A project preceding this one was to create a multiple choice test analysis software that outputs statistics and plots for analyzing test results.

Currently the output files are not yet compatible with this software, and it shouldn't be hard to add this support.

App repo: https://github.com/gadenbuie/mctestanalysis

Add unit tests

This is critically important - there is no automated testing. We need unit and/or end-to-end testing examining as many scenarios as possible. (edit: we do have E2E testing now but could still use some unit tests).

Add 200question option?

Hey, how long would it take to alter the script for a test with 200 questions? I am trying to prepare a simulation exam for medical students (offered for free) and the exam is a 200 questions one.

I would appreciate and try and help with this feature (I'm a linux engineer, but I'm trying to learn some OpenCV anyway)

Remove extra column names from results

Empty columns shouldn't have column names left.

"Failed to execute script main" error when starting OpenMCR

U nfortunately the program does not start on my windows 11.
I get the following information"Failed to execute script main"
What could be the reason ? Please, give me some hint in this matter.

Add docstrings to functions

All functions should be well documented

Add type definitions for OpenCV and Numpy

For now I am limiting this to solely what types I am actively using in the libraries. IE, if I call numpy.function(A,B,C) then I only define:

def function(A: int, B: int, C: int) -> int: ...

Even if the full function is actually something like:

def function(A: int, B: int, C: int, D: float = 45.5) -> int: ...

Add more specific error messages for some cases

Add an option to sort results by student's name

This should be a checkbox - either sort or leave in same order.

does not work on 150questions

it does not work on 150questions
the error is:
error:field.Lnastname:1

Add original filenames to results.csv

This will help in finding the original file if needed to verify results.

Add open source license for code

Validate arrangement files and keys

The utility can check if keys have blank answers, or if arrangement files don't correspond completely.

Read all bubbles in a single operation

Currently the utility processes an image and then reads each zone as a separate operation. It would be much more efficient to read the entire sheet at once and calculate the threshold at that time, as the current method results in reading all the name bubbles twice.

Command line arguments don't support quoted file paths

The command line argument parsing functionality is using the native pathlib.Path constructor method to parse file path arguments. However, this fails when a path is quoted using double quotes, like: --formmap = "C:/Test User/data/map.csv". Because this path has a space, double quotes are necessary. But this fails because the quotes are never stripped.

Solution is to just create a custom parsing method that strips quotes if present.

Sign the installation binaries

Currently installing the program results in a number of errors due to unsigned binaries. We can improve the experience by signing them.

Support other sheet layouts with more questions

Per discussion in #29, a feature to consider for future versions would be alternative sheet layouts with more than 75 questions.

Having the test form code column right next to the question columns is confusing because it looks like another question

Add Creative Commons license for the exam sheet PDF / SVG

Unable to find document corners on low-resolution image

Hi iansan,
i am using below sheet as input but giving error couldn't find document corners.

Transpose the answer key sheet if there's only one key

When using the rearrangement method, the program should transpose the answer key sheet as there will only be one key and this is more intuitive.

Remove extra unneeded columns from output files

Add datetime to results files to prevent overwriting existing files

ie YYYYMMDDHHmmss_scored-results.csv

Cannot find corner marks when scanning software dithers the image

When the UI is closed without completing testing, it is never terminated and remains running in the background

Repro:

1. Run python src/main.py
2. Press the close button.
3. The process is still running - you can see how the console has not returned control to the user:
   

add 100 and 200 question sets also..

add 100 and 200 question sets also..

Threshold for determining bubble fill is static but should be dynamic

Currently, if a scanner that darkens the image is used, extra bubbles (especially the W and M bubbles) will be processed as filled. If a scanner that lightens the image is used, not enough bubbles will be considered. The threshold should either be dynamically created or at least improved per-letter.

Trial downscaling larger images to speed up processing

Gracefully handle when an image is sideways

The software should either provide a more useful error message (it might be simpler to just say "Image X is invalid" rather than "Image X is rotated" since detecting rotation may be difficult), or simply process the image as if it was not sideways (maybe by re-running with 90deg rotation after failure).

Rewrite the manual to make it clearer for new users

Change "Sort results by name" to something clearer

Maybe "Sort results by student name"?

List of images that are bad answer sheets.

I have used the software to check about 200 answer sheets in one go. The software was unable to handle bad answer sheets .

So, I had to physically identify and separate the bad answer sheets.

If its okay, I would like to add a feature where the software will generate a separate output file to show the list of images that were bad-answer-sheets.

bad answer sheets are those that do not have the required corners - it could be due to...

• bad scans
• dirty / stained pages
• images that are not answer sheets at all
• or any such reason

Add option to save blank answers as "G"

The program does not start automatically after installation

Reported by Dr. Kaw - need to verify on other devices as well.

Add a command line interface

A CLI would be very useful for debugging and when users want to do bulk processing. The CLI should accept all of the UI inputs as arguments and skip the UI prompt.

CLI design guidelines: https://clig.dev/

The UI currently looks like this:

For each input, there would need to be a command line option (ie, --output-path = ../output).

Value 'tp.Optional' is unsubscriptable

Value 'tp.Optional' is unsubscriptable

Add support for more variations of the OMR sheet design

Can I use a different Design of OMR Sheet with this script. Can you help me in designing my custom design sheet as a consultant.

Incorrect extracted data when reading slightly distorted images

Greetings,
when I use some images, the program extracts incorrect data.
Please see the attached images.
Thanks
Ammar

Add automated build / type checking

Add automatic type checking using Pipelines so we can have some checks before merging a PR.

Output total score percent as a value out of 100

Currently outputs on a 0-1 scale.

Does not run on Linux due to ICO file as app icon

Reported here #39 (comment).

Convert examples to end-to-end snapshot-style tests

We have several examples committed to the repository. It should be simple to convert these to end-to-end tests.

Find a better name for the project

The user interface can be too long for some screens

Consider condensing or using a grid layout

Output results as 1 or 0 instead of TRUE or FALSE

Error "<Field.LAST_NAME: 1>" thrown when scoring 150-question exam sheet

The script does not process the sheet with 150 questions! It always gives an error :
Error : <Field.LAST_NAME: 1>.
please note that, I am using an image as an answer sheet with Student ID = 9999999999

Output keys.csv in column form instead of row

This is to make it compatible with existing test analysis software.

Example:

Test Form Code	A	B
Q1	A	A
Q2	C	C
Q3	E	E
Q4	B	B
Q5	E	E
Q6	C	C
Q7	D	D
Q8	E	E
Q9	A	A
Q10	B	B
Q11	C	C
Q12	B	B
Q13	C	C
Q14	A	A
Q15	E	E
Q16	E	E
Q17	E	E

Utility crashes if no key arrangement file is provided

Add "Q" prefix to question number column heading

MCTA answer key files are not trimmed down to the number of actual questions used

Reported by Dr. Kaw over email:

In the answer key generated for MCTA, it populates all 76 rows for 75 Qs when the test may be only 38 rows with 37Qs... That gives error in MCTA program... MCTA worked well after I removed the rows.