pjrinaldi / wombatforensics

linux c++, fox-toolkit, multi-threaded forensic gui tool

Wombat Forensics

Wombat Forensics is a new Forensic Analysis tool built entirely in C and C++. The GUI is built using the FOX-Toolkit, so it may one day work on Windows, Linux and Macintosh systems. The current design is Linux specific, but if there ever is a need I can eventually implement various IF's to make it work on the Windows and Mac. The application is designed to be user friendly, fast, and always maintain GUI responsiveness. Having used AD Lab, X-Ways, Encase, Axiom, and Autopsy, I found various things that would bother me or were slow and unresponsive. I decided for a resource intensive application, using C/C++ was important.

Repository Change

• I am no longer building code on github. I have moved my code to the website www.wombatforensics.com and am hosting my repositories on a vps using fossil for the repositories rather than git. I decided to stop using github due to all the AI crap and scraping code. My code isn't fancy or great, and it is free, but I just don't like the idea of scraping without my ok and since github is free, that is part of the price for free access. So I am leaving the historical bits of my repositories, but moving them all to fossil repositories. Feel free to check them out, they aren't as fancy or featureful as github, but it fits my needs.

In the Works Features

• I currently building mobile extraction support. It will load a zip or ufd/ufdx and display the root directory and let you navigate the directories. It is very rudimentary and more features will come. There is no waiting hours to navigate the directory of a phone extraction. That is the design behind it same as a forensic image.
• I can read/parse squashfs forensic images, as well as my custom wombat forensic image, which i changed the format and am using walafus code to implement.

Current Features

• The goal is to implement all these features in the redone fox-toolkit version prior to the next release (v0.5), this will take a while, but I'll get it there in the end.

• Tagging - You can Tag (bookmark) files by simply right clicking on a file and selecting the tag to apply. (v0.2)

• Report Preview - This feature generates the report in a preview window as you add evidence or tag files to different sections. If thumbnailing has been done, then thumbnails will display properly in the preview report. (v0.2+)

• Publish Report - Publish a report and export/link files to the HTML Report. Places the report in the reporting directory set by the user from the settings dialog and in a unique sub folder based on date/time. (v0.2+)

• Manual Carving - You can manually carve sections from the hex and tag them to your report. (v0.1+)

• Image/Video Thumbnailing Viewer is a separate window, and once loaded has no lag when scrolling. You can go as fast as you want. (v0.2+)

• If running a large ingest or just need a break, you can launch xchomp (pacman clone). (v0.2+)

• Any post processing (digging deeper) is done in the background on other threads, so you can continue to review the evidence without any lag. (v0.1+)

• Filtering will gray out non-matches, rather than hide the files in case you still want to look at them. (v0.1+)

• You can have multiple "File Properties", "File Hex", "Viewer Windows", etc. open for different files for comparative analysis. (v0.2+)

• File signature analysis is done at ingest. (v0.1+)

• Hashing files when "Digging Deeper" (cause Wombat's dig). (Blake3). (v0.3+)

• Semi-Smart Carving for JPG's, GIF's, PNG's, PDF's, MPG1/2's, as well as the ability for the end user to enter carving information for simple header/footer file size carving. The semi-smart carving uses block information for other carved headers to restrict the file's. It isn't the best carver, but it is better than basic and is a good start. (v0.3+)

• $I30 Parser, Lnk Parser, PreFetch Parser, Recycle Bin (INFO2, $I) Artifact Parser, Zip Archive Parser (v0.3+)

• Zip File Extraction added to "Digging Deeper" capability (v0.3+)

Current Viewers

• The goal is to implement all these viewers in the redone fox-toolkit version prior to the next release (v0.5), which will take a while, but I'll get it there in the end.
• Image (v0.1+)
• Video (v0.1+)
• Text (v0.1+)
• HTML (v0.1+)
• PDF (v0.3+)
• External Viewer feature where you can specify external programs (v0.1+)
• Registry Viewer (v0.3+)

Comparison with Other Forensic Suites

• Coming Soon...

Reasoning

I wanted to improve my general forensic knowledge and understanding of what the existing tools are doing as well as learn more about specific artifacts. One way to do that is to build a forensic tool. This will give me better knowledge of how the black box tools are working. I didn't want to reinvent the wheel, so I am making use of as many open source libraries and tools as possible, to include the libsmraw, libewf, liblnk, libmagick, and so on.

Why is user friendly so important?

When I started this project, there were no GUI forensic tools for linux, and so I would be working with the major GUI forensic tools in Windows. I don't like Windows at all, and all the existing GUI forensic tools have issues that bothered me. They would experience GUI freezes or GUI tearing and redrawing issues which is annoying and interrupts my workflow and train of thought. Some of the tools have too many ways to do the same thing or options that are very confusing and not explained very well or intuitive as to their meaning. I shouldn't have 3 ways to do the same thing, it is overkill and poor design. A program can be robust, but provide a sensible set of features/options to a user.

Wombat Forensics is user oriented in its design. You can select an item or check multiple items. The program is multithreaded. The GUI is always responsive because resource intensive tasks are done on different threads. Large files will not slow down the application. The hex viewer was designed to load portions of a file at a time, therefore it is always fast to view and does not eat up RAM.

I find it a waste of real estate to have a treeview in one frame and a table listing in another frame. Therefore I have combined the file structure treeview and the file information table into one view with a toolbar. Rather than expand the tree so it gets so wide you can't view anything other than the file names, as you double click items, the child contents of that parent will load in the treeview and the parent will go into the path toolbar. Each parent folder/file is a toolbar button so you can click backwards and back through the tree from the evidence images to the currently selected file/folder. The columns sort of autoresize when data is loaded but then they don't autoresize so the user can control the resizing of the views. Also, the user can show/hide columns by right mouse clicking on any column header and selected the column from the subsequent right click menu.

I run timezone data in UTC for all features. I always prefer to work in UTC and then there is an option to automatically convert to a preferred timezone when generating the report. This timezone selection is in the settings dialog. If you change the timezone, the preview report will update the dates and times accordingly, as well as the timezone listed in the report. It will not update an already generated "Published Report", you will have to generate a new report to show the new timezone.

Why open source?

Closed source tools are a black box. Wombat Forensics is open source and integrates open source libraries such as the libaff and libewf, enabling an examiner to review code to find out how the tool yields results and verify that the results are what they should be.

When will it be ready?

Operating systems, file systems and software programs are constantly changing, so digital forensics must also constantly change. So it will never be finished, but hopefully in a polished state for some kind of usage by v1.0. With all that said and done, I am a single developer who has a full time job and spends about 3 hours/day for 3 days/week (avg) working on this project. Plus to ensure it is the most efficient and most user friendly interface, I have no issues reworking entire portions to fix issues.

wombatforensics's Issues

Add Image-Done/ Open Case started

Original issue 8 created by pjrinaldi on 2013-08-10T21:06:58.000Z:

Need to abstract treeview further to retrieve image name and info so i can add more images and setup the full path in the treeview as well as the respective volume/fs info.

getting close to having treeview done enough to move on to another aspect of the program.

Current Program Structure

Original issue 5 created by pjrinaldi on 2013-08-01T16:46:08.000Z:

Going to use xml files for the configuration/settings. but will also store the settings in a db for quicker access

CREATE TABLE settings(settingsid, settingstype [default, current], settingsmode [tsk, gui], settingsname [docarving, dokff], settingsvalue[0,1])

possibly might do settingsvalue as [autoadd, manual]

The same goes for the cases table and log table. these will be also be stored in a .xml file.

The program will have a "Cases" folder created in the user-data directory.
Each case created will have a folder "CaseName" with the respective xml,db settings in there. Each piece of evidence will have a folder "EvidenceName" with the respective xml, db settings accordingly.

implement remove evidence

Original issue 18 created by pjrinaldi on 2013-09-19T03:28:04.000Z:

Implement remove evidence which makes deleted = 1 in evidence table. also implement remove case which makes deleted = 1 in cases table. This will remove it from gui, but preserve it in the database.

implement file signature analysis

Original issue 14 created by pjrinaldi on 2013-09-18T14:11:49.000Z:

Implement code for file type categorization. Need gui elements and the model and the whole shebang.

new/open course when course is open

Original issue 23 created by pjrinaldi on 2013-09-28T20:06:58.000Z:

when a new course is created and a course is already open, i need to clear the treeviews and views.

since i'm gonna do it more than one place, i should abstract the functions to a function.

Click on Image/Volume/Partition|FS/Directory - what should show in hex view

Original issue 45 created by pjrinaldi on 2013-11-04T14:35:08.000Z:

When a directory is clicked on, it should show the actual directory entries hex for the files (if possible) (dcat seems to get the info i need.)

for the omniviewer, i should show an html table of the directory entry structure with the file properties that are listed and extracted from the hex.

If I can't implement this with the framework, I may need to build my own set from scratch using the tsk library.

Rename Class/Files

Original issue 13 created by pjrinaldi on 2013-09-13T18:03:00.000Z:

Rename wombatcasedb to wombatdata or wombatdatabase.h

Proper Hex to string

Original issue 7 created by pjrinaldi on 2013-08-09T12:18:57.000Z:

trying to figure out how to display hex as text/numbers

Here's a simple piece of code to do the trick:

unsigned int hex_digit_value(char c)
{
if ('0' <= c && c <= '9') { return c - '0'; }
if ('a' <= c && c <= 'f') { return c + 10 - 'a'; }
if ('A' <= c && c <= 'F') { return c + 10 - 'A'; }
return -1;
}

std::string dehexify(std::string const & s)
{
std::string result(s.size() / 2);

for (std::size_t i = 0; i != s.size(); ++i)
{
    result[i] = hex_digit_value(s[2 * i]) * 16
              + hex_digit_value(s[2 * i + 1]);
}

return result;

}
Usage:

char const a[] = "12AB";

std::string s = dehexify(a);
Notes:

A proper implementation would add checks that the input string length is even and that each digit is in fact a valid hex numeral.

Dehexifying has nothing to do with ASCII. It just turns any hexified sequence of nibbles into a sequence of bytes. I just use std::string as a convenient "container of bytes", which is exactly what it is.

std::auto_ptr

Original issue 35 created by pjrinaldi on 2013-10-19T03:50:21.000Z:

read up on auto_ptr and possibly convert all my std::* to auto_ptr for memory management and efficiency.

add evidence already added to the case

Original issue 22 created by pjrinaldi on 2013-09-25T20:34:26.000Z:

need to handle if evidence is already added to the case. so when evidence is going to be added, i'll need to compare evidence names from db with new evidence to determine if it exists, if it does, then don't open and let user know with a prompt. else, add evidence the same.

implement the necessary columns for dir treeview

Original issue 40 created by pjrinaldi on 2013-10-23T14:06:54.000Z:

Need to implement columns that are relevant and necessary.

Signature Type, File Extension,proper date time Mac columns. Use qdatetime for simple conversion.

unqlite implementation

Original issue 11 created by pjrinaldi on 2013-09-09T12:11:47.000Z:

Update dbschema to account for multiple evidence id's per case.

Look into migrating away from sqlite and into postgresql.

This is a pain in the arse to get up and running before i can continue to work on my code.

change the temp file names

Original issue 42 created by pjrinaldi on 2013-10-23T20:40:52.000Z:

need to change the tmp file name since it's the fileid which duplicates between images. should probably make it the unique id to avoid duplication since if i made it the fileid-filename that could be duplicate also.

Sql Issues

Original issue 3 created by pjrinaldi on 2013-07-30T16:39:24.000Z:

Need to figure out why i'm getting so many sql segmentation fault issues...

need to implement horizontal scrollbar in hex editor

Original issue 48 created by pjrinaldi on 2013-11-09T05:06:51.000Z:

i think it redraws the screen with every scroll bar click....

so i need to take the horizontal width into account and only draw part of it.

TSK

Original issue 2 created by pjrinaldi on 2013-07-25T12:00:54.000Z:

Decide if I want to git the sleuthkit framework codebase to incorporate into my project. or if i want to re-invent the wheel completely and dissect the code and build accordingly

Make MainWindow Pluggable

Original issue 1 created by pjrinaldi on 2013-07-24T16:57:12.000Z:

Setup MainWindow ui to be pluggable, so its just containers, that the plugins fill.

treeview checkbox column

Original issue 32 created by pjrinaldi on 2013-10-12T17:18:51.000Z:

need to implement treeview checkbox column to enable options to process checked, selected or listed.

bounds checking

Original issue 41 created by pjrinaldi on 2013-10-23T18:53:37.000Z:

ensure I am doing proper bounds checking to ensure I don't have any buffer overflows.

Better Scheduler

Original issue 12 created by pjrinaldi on 2013-09-13T15:45:36.000Z:

Current scheduler loops over each task and processes it. it works, but it is slow. since it's only processing 1 file at a time. I need to build a scheduler that will take all the files, and process as many as it can at once.

create hex editor

Original issue 49 created by pjrinaldi on 2013-11-11T04:04:39.000Z:

need to create the better hex editor from lfhex code. need to modify and design to fit my needs with the side by side offset/hex/ascii setup and highlight tooltip to show what the values are.

Variable ChangeOver

Original issue 34 created by pjrinaldi on 2013-10-19T03:46:13.000Z:

I want all variables to be std::string, vector, list, etc if i can. Then i just convert the values to what I need to put it in the gui, but the actual computational code would work without QT.

wombatvariable.h, sleuthkit.h/cpp, wombatdatabase.h/cpp, etc.

makes it easier to deal with for computational conversions and its pretty easy to simply make it a QString as needed for the gui stuff. which is less than the calculation/computation stuff.

fork tsk and afflib

Original issue 28 created by pjrinaldi on 2013-10-06T17:33:08.000Z:

fork tsk and afflib so I can modify as needed to work with my code and to be able to get proper error information.

TreeView Icons

Original issue 38 created by pjrinaldi on 2013-10-22T17:05:47.000Z:

need to implement treeview icons for different file types at some point.

get image tree display in dirtreeview is wrong

Original issue 37 created by pjrinaldi on 2013-10-22T17:05:01.000Z:

missing root directory as in "/".

fixed the implementation and the associated child items so that they are associated with the correct parent.

add progresswindow job functions to the export data job

Original issue 36 created by pjrinaldi on 2013-10-20T01:34:36.000Z:

need to add progress window features to the export data job once exporting files is working.

Show File/Get Contents

Original issue 39 created by pjrinaldi on 2013-10-23T01:53:57.000Z:

need to implement the mkdir solution that i used in export file for when a directory is selected in the treeview since it is not a file...

treeview column autosizing

Original issue 27 created by pjrinaldi on 2013-10-02T21:00:52.000Z:

need to autosize the treeview column when you click on the expand/collapse icons.

and autoexpand columns for each item so it displays correctly.

export files to image

Original issue 30 created by pjrinaldi on 2013-10-12T02:54:42.000Z:

Not sure if I should create an option to create an image, but I will definitely create an option to export files to an image using afflib. From there I'll decide if I need to make it available to make an image from evidence. Not sure I want to combine analyzing with creating. Maybe I'll do a second simplified tool to create images.

File Type Extension Module

Original issue 43 created by pjrinaldi on 2013-10-23T22:19:06.000Z:

Add a File Type by Extension Module to the framework. For now, I'll just add file type by signature for a column

deleted files display

Original issue 50 created by pjrinaldi on 2013-11-18T01:23:03.000Z:

Since there are inconsistencies in deleted files, I think its best to treat them differently or represent them differently. I'll have to figure out a suitable way to do this and then implement it. Maybe as I write up the error in the way ad lab displays things carved from deleted files, I'll think of how to do it.

If I can carve from the raw image rather than individual files, I can compare what's been carved at a given sector and what was supposed to be their according to the filesysem to give a probablilty of accuracy or correctness.

highlight hex

Original issue 47 created by pjrinaldi on 2013-11-06T18:03:16.000Z:

when highlight hex, need to popup a tooltip that displays the value in the various converted formats.

omni viewer

Original issue 17 created by pjrinaldi on 2013-09-19T03:15:18.000Z:

need to get the omni view tab working.

Report Generator

Original issue 46 created by pjrinaldi on 2013-11-05T16:52:01.000Z:

Items should be added to a report generator in the spirit of case notes creator. incorporate this style functionality into the analysis so you select the parts and then have a list of items to stick into a report wherever you see fit.

hex/txt viewers

Original issue 24 created by pjrinaldi on 2013-09-28T20:17:39.000Z:

hex viewer and text viewer seem to get the stderr information instead of the actual file contents of the item selected. although, the actual file is known from the output command?

have to look into this one and figure out what is going wrong.

Tmp File Creation

Original issue 26 created by pjrinaldi on 2013-09-29T20:55:01.000Z:

may want to create tmp files during application initialization.
then when i close the app, delete the tmp files. this way, its faster to read the contents and write the contents rather than having to write to the file.

Log File

Original issue 9 created by pjrinaldi on 2013-08-12T17:26:36.000Z:

split log file columns by \t.

caseid, imageid, type, date/time, message.

get the results:

include <sstream>

include <fstream>

include <string>

include <vector>

std::ifstream file("file.txt");

std::string line;
std::string partial;

std::vector<std::string> tokens;

while(std::getline(file, line)) { // '\n' is the default delimiter

std::istringstream iss(line);
std::string token;
while(std::getline(iss, token, '\t'))   // but we can specify a different one
    tokens.push_back(token);

}

display file in viewer's

Original issue 21 created by pjrinaldi on 2013-09-24T15:32:57.000Z:

Might want to abstract the displaying a file in the viewer's so that the function to display a file in the viewer is to load the tmp file(s) into their respective viewers.

when you click on a file, it simply writes the new data to the tmp file(s) and then calls the refresh viewer(s) function which reloads the tmp file(s) into the viewer(s).

enable ability to remove cases/evidence from program

Original issue 15 created by pjrinaldi on 2013-09-18T15:24:16.000Z:

Need to add columns isdeleted to cases and evidence table so I can create remove evidence and remove case features. Instead of deleting entries, I'd rather call update cases set isdeleted = 1 from the default of 0. Same for evidence. This allows linked data to stay and allow cases and evidence to be hidden where needed.

need to update all db queries to account for isdeleted = 0.

Progress Window

Original issue 10 created by pjrinaldi on 2013-08-13T20:54:34.000Z:

Need to move progress window from the plugin to the main window

object/pointer passing

Original issue 44 created by pjrinaldi on 2013-10-24T01:41:15.000Z:

need to review my code and ensure that i am passing as few objects as possible. wherever i am passing an object, see if i can't pass a smaller pointer (*) to the object around, such as wombatvariable.

Custom QStandardItem

Original issue 6 created by pjrinaldi on 2013-08-06T18:26:34.000Z:

Need to subclass QStandardItem to provide a custom variable to store the fileid, for easier reference later.

directory treeview clicking errors

Original issue 19 created by pjrinaldi on 2013-09-20T13:24:41.000Z:

Need to if/else for when you don't click on a file and you click on image/volume/partition/fs entry so it does nothing and doesn't error out.

menu/toolbar disable/enable

Original issue 20 created by pjrinaldi on 2013-09-20T14:12:13.000Z:

need to work out when to enable/disable the menu/toolbar buttons based on such things as no case open, no evidence to remove, etc...

remove tmp files on exit

Original issue 25 created by pjrinaldi on 2013-09-29T16:34:33.000Z:

Need to remove all the temp files on exit.

abstract resize columns for other views

Original issue 31 created by pjrinaldi on 2013-10-12T17:09:04.000Z:

may need to work on abstracting the resize columns for other views using view->model() to compare to the currently selected model, but will wait and see what arises with the design and implementation

properties view tab

Original issue 16 created by pjrinaldi on 2013-09-19T03:13:59.000Z:

need to implement the properties view for a file. (of course, need to determine if the properties is necessary and has info the list won't have.

export file

Original issue 29 created by pjrinaldi on 2013-10-12T02:48:29.000Z:

With the save method working, I can export a file using the copy file to a location path.

GUI Elements

Original issue 4 created by pjrinaldi on 2013-08-01T16:37:37.000Z:

Working on building the gui elements, tabs, file viewers, column displays, treeviews, etc. as well as the necessary settings and infrastructure and forms/dialogs

idea for organizing the data

Original issue 33 created by pjrinaldi on 2013-10-17T18:12:45.000Z:

I might want to group stuff by category/application as ief does.
Web browsers, four browsers and has relevant data extracted and pretty print from the browsers.

Same for gigatribe and others. To maintain this idea, they would be plugins which contain the algorithms and return the data. The gui would have to be in the main app.

Just an idea I meant want to do as I build analysis modules and features. Granted, its a ways away, but one idea.