Multimedia Retrieval

Please install Python 3.8.10.

• Update pip: python -m pip install --upgrade pip
• Install packages for easier/faster/better installation of other packages: pip install Cython wheel
• Install required packages: pip install -r requirements.txt
• Check if installation was succesfull: python -c "import open3d as o3d; print(o3d.__version__)"

To visualize the meshes, run mesh_viewer.py. Additional arguments can be provided:

python ./Rorschach/visualization/mesh_viewer.py --mesh_path ./data/AircraftBuoyant/m1337.obj --visualization_method shade
python ./Rorschach/visualization/mesh_viewer.py --mesh_path ./data_cleaned_/AircraftBuoyant/m1337.obj --visualization_method shade
python ./Rorschach/visualization/mesh_viewer.py --mesh_path ./data_normalized/AircraftBuoyant/m1337.obj --visualization_method shade

python ./Rorschach/visualization/mesh_viewer.py --mesh_path ./data/AircraftBuoyant/m1337.obj --visualization_method wired
python ./Rorschach/visualization/mesh_viewer.py --mesh_path ./data_cleaned_/AircraftBuoyant/m1337.obj --visualization_method wired
python ./Rorschach/visualization/mesh_viewer.py --mesh_path ./data_normalized/AircraftBuoyant/m1337.obj --visualization_method wired

After running any of the above commands, the resulting window should look similar to the following:

1. Run python ./Rorschach/preprocessing/patch_meshes.py to resample the meshes, this includes remeshing.
2. Run python ./Rorschach/preprocessing/preprocess.py to normalize the meshes.
3. Run python ./Rorschach/visualization/visualize_data.py to generate the relevant histograms and boxplots to visualize the resampling and normalisation steps. This will also print relevant statistics of the original, resampled and normalized datasets.

To start the querying front end, run the following command:

• python -m streamlit run ./Rorschach/querying/frontend.py

This automatically opens the web page http://localhost:8501/.

Run the following PowerShell script to run the complete pipeline:

cd INFOMR
. .\pipeline.ps1

This pipeline includes:

• Resampling meshes
• Normalizing meshes
• Visualizing the difference before and after resampling and normalization
• Feature extraction
• Visualizing the results of the feature extraction
• Collecting the |c| closest meshes for each mesh in the dataset according to different distance functions
• Evaluating the different distance functions based on the collected neighbors
• Plotting the results of the evaluation
• Plotting the dimensionality reduction results on the extracted features.

This pipeline takes 160 minutes to run on an average laptop with 16GB of RAM and an Intel Core i7 processor.

• The assignment
• Open3D - Getting started
• Princeton Shape Benchmark dataset
• Labeled PSB dataset
  • Download link
• ShapeDatabase provided by TA's for testing