fraunhofer-iis / camorph

The official repository for camorph, a python toolbox for converting various camera parameter conventions.

Camorph

Paper

This is the gitlab project for camorph, a camera parameter converting tool written in python 3.9.

Documentation

Documentation can be found here.

Quickstart

Some quick tips on how to use camorph.

Installation

If you want to work on camorph, clone the repo and install the conda environment by running

conda env create -f camorph_win.yml

Otherwise, you can also install camorph with pip

pip install git+https://github.com/Fraunhofer-IIS/camorph

Usage as Python Library

Use the library to convert camera parameter representations:

import camorph.camorph as camorph
 
 
cams = camorph.read_cameras('COLMAP',r'\\path\to\colmap')
 
camorph.visualize(cams)
 
camorph.write_cameras('fbx', r'\\path\to\file.fbx', cams)

camorph.read_cameras() takes a format name and path as a string and returns a list of Cameras.

camorph.visualize() creates a visualization of Cameras with Matplotlib.

camorph.write_cameras() takes a format name, a path as a string and a list of cameras and writes the output file(s) to the specified path.

Usage as CLI

You can use the command line interface by calling

python -m camorph -h

or when installed with pip

camorph -h

This will output the help for the command line interface

usage: camorph [-h] -i input_path [input_path ...] -if input_format [-o output_path] [-of output_format] [-v]
               [-c config] [-ft file_type] [-pt] [-cr crop] [-s scale] [-id image_dir] [-ci]

Convert Cameras from different formats to each other

optional arguments:
  -h, --help            show this help message and exit
  -i input_path [input_path ...], --input input_path [input_path ...]
                        the input path of the camera file(s) to read
  -if input_format, --input_format input_format
                        the format of the input camera file(s)
  -o output_path, --output output_path
                        the output path where the camera file(s) should be saved
  -of output_format, --output_format output_format
                        the format of the output camera file(s)
  -v, --visualize       when this parameter is present, the cameras will be visualized.
  -c config, --config config
                        the path to a config.json file for missing crucial properties
  -ft file_type, --file_type file_type
                        some formats support different types of output files, for example bin for binary and txt for
                        ascii files
  -pt, --posetrace      treat the input as a posetrace and ignore any source images.
  -cr crop, --crop crop
                        crop source image attributes by the specified top left and bottom right corner. Format:
                        "leftcorner_x,leftcorner_y,rightcorner_x,rightcorner_y". ATTENTION: THIS DOES NOT MODIFY THE IMAGES, ONLY THE PROPERTIES IN THE FILE!
  -s scale, --scale scale
                        scale source image attributes by the specified factor. ATTENTION: THIS DOES NOT MODIFY THE IMAGES, ONLY THE PROPERTIES IN THE FILE!
  -id image_dir, --image-dir image_dir
                        replace the directory for the source images with this.
  -ci, --check-images   check if images exist and are of the right resolution.

For example

camorph -i \path\to\json -if nerf -o \path\to\output -of fbx

Crucial Properties

If there are missing crucial properties, camorph will automatically create a config.json in the target folder. To edit this file, please refer to the "Crucial Properties" section of the documentation

Pose Trace

If you want to convert a camera animation, which does not have source images (sometimes also referred to as pose trace), you can add the -pt argument to ignore any source image requirements.

Cropping and Scaling

This option crops and scales the attributes of images in the file. For example, colmap stores the resolution of the images, which can be modified by this parameter.

❗❗❗ THIS DOES NOT MODIFY THE IMAGES THEMSELVES! ❗❗❗

Currently Supported Formats

• Computer Graphics

  • FBX (Key: "fbx")

• Photogrammetry

  • COLMAP (Key: "colmap")
  • Meshroom (Key: "meshroom")
  • Reality Capture (Key: "reality_capture")
  • Local Light Field Fusion (Key: "llff")

• Game Engines

  • Unity (Key: "unity")

• Virtual Reality

  • MPEG OMAF (Key: "mpeg_omaf")

• Machine Learning

  • NeRF (Key: "nerf")

Citation

@article{Brand2022CAMORPHAT,
  title={CAMORPH: A TOOLBOX FOR CONVERSION BETWEEN CAMERA PARAMETER
CONVENTIONS},
  author={B. Brand and Michel B{\"a}tz and Joachim Keinert},
  journal={The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences},
  year={2022}
}

camorph's Issues

Colmap to nerf image paths error and permissions error

Im getting this error when trying to run colmap to nerf. Image paths in my cameras.txt are set to the absolute paths, and I'm running conda as admin;

(camorph_win) d:\.repos\camorph>python -m camorph -i D:\.repos\camorph\in\txt\source -id D:/.repos/camorph/in/txt/images -if colmap -ft txt -o D:/.repos/camorph/out -of nerf
D:\.repos\camorph\camorph\ext\NeRF\NeRF.py:197: UserWarning: Camorph needs absolute image Paths. Relative paths are automatically calculated when saving to NeRF
  warnings.warn("Camorph needs absolute image Paths. Relative paths are automatically calculated when saving to NeRF")
Traceback (most recent call last):
  File "D:\.conda\envs\camorph_win\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "D:\.conda\envs\camorph_win\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "d:\.repos\camorph\camorph\__main__.py", line 4, in <module>
    cli.run(run_as_module=True)
  File "D:\.repos\camorph\camorph\cli.py", line 58, in run
    camorph.write_cameras(cargs.output_format, cargs.output, cams, crop=c, scale=s, check_images=cargs.check_images,
  File "d:\.repos\camorph\camorph\camorph.py", line 119, in write_cameras
    dest_inst.write_file(cam_copy, path, file_type)
  File "D:\.repos\camorph\camorph\ext\NeRF\NeRF.py", line 225, in write_file
    with open(output_path, 'w') as f:
PermissionError: [Errno 13] Permission denied: 'D:/.repos/camorph/out'

Documentation typo regarding file types

Dear

Could there be a typo in your documentation regarding file types?

Which value should we use for Nerfs? Should be 'nerf' I guess?

Kind regards
Paulus

Unsupported Camera Model $division

Trying to convert Reality Capture to Colmap.

Get

File "C:\...\site-packages\camorph\ext\COLMAP\util\WriterSyntacticUtil.py", line 168, in _get_model_and_params
    raise Exception(f"Unsupported Camera Model ${cam.model}")
Exception: Unsupported Camera Model $division

Camorph Meshroom to InstantNGP doesn't provide desired outcome

Dear

I tried to use your Camorph script but did not arrive at the expected result. I generated camera postions from 17 pictures in Meshroom via the StructureFromMotion node. I used the cameras.sfm file to then run your script to generate the transforms.json file to use in InstantNGP and arrived at following results.
As example 1 of 17 source pictures:

Results:

The dense point cloud I generated from Meshroom some nodes down the line however seems perfectly fine so I do not really doubt the correctness of the generated camerapositions in Meshroom.

Then I tested to generate the transforms.json file anew via the instantngp-batch procedure described over here:
https://github.com/jonstephens85/instantngp-batch
The procedure first runs colmap to detect camera postions anew and then uses the script colmap2nerf.py to generate the transfroms.json file. This procedure yields significantly better results. See below.

However, I have needed to calculate the camerapositions twice now, once in Meshroom and once via Colmap. I would prefer to only do this once and see promise for your script if it would yield the same results as the Colmap procedure.
I am under the impression that there must be a matrix tranformation bug in the script as I can see from the camera symbols and axis colors on the first Result image that they appear to be flipped along their view direction (blue Z axis). Also I see this because I took the pictures in 2 rows of distance to object and I a know from the pictures that on the nearest row I took 2 pictures each time while on the farthest row I only took 1 picture on each position. From your generated camera symbols I can see that the row order is flipped, which means that the cameras have flipped their viewing direction.
Looking forward to any answer you can give. Please let me know if you need any of my testfiles. I am more than happy to provide those.

For your convenience I have already included both transforms.json files generated by Colmap and by Meshroom and your script:
transforms_FromColmap.json.txt
transforms_FromCamorph.json.txt

Kind regards
Paulus

List of input format commands

Hi, is there a list of input format commands anywhere?

Conversion to LLFF example?

Hi do you have a example of conversion to LLFF ?

I tested Colmap to LLFF but thats missing properties: "near_far_bounds"

I'm wondering what format should work?

Not handling all NERF formats

I get the following error

  File "/home/m/work/aos/camorph/m.py", line 4, in <module>
    cams = camorph.read_cameras('nerf',r'/home/m/work/aos/nerfdata/take15_optimized_cams/transforms.json')
  File "/home/m/work/aos/camorph/camorph/camorph.py", line 29, in read_cameras
    cams = src_inst.read_file(*args, **kwargs)
  File "/home/m/work/aos/camorph/camorph/ext/NeRF/NeRF.py", line 144, in read_file
    cam.focal_length_px = [.5 * cam.resolution[0] / np.tan(.5 * camera_angle_x),
TypeError: unsupported operand type(s) for *: 'float' and 'NoneType'

for code

import camorph.camorph as camorph
cams = camorph.read_cameras('nerf',r'/home/m/work/aos/nerfdata/take15_optimized_cams/transforms.json')
camorph.visualize(cams)

It seems camera_angle_x is not being set. Possibly because transforms.json is using a matrix instead;

{
    "camera_model": "OPENCV",
    "frames": [
        {
            "file_path": "images/frame_00001.jpg",
            "w": 3000,
            "h": 4088,
            "fl_x": 10716.10907946319,
            "fl_y": 10716.10907946319,
            "cx": 1381.2775952841873,
            "cy": 2280.181297884072,
            "k1": 0.0971299736922087,
            "k2": -7.007934572579263,
            "k3": 130.90033135432077,
            "k4": 0.0,
            "p1": -0.0024336328994766076,
            "p2": 0.0065866252174946885,
            "transform_matrix": [
                [
                    -0.3353805106176828,
                    0.3177818535863063,
                    -0.8868678631166393,
                    -6.4681681952398336
                ],
                [
                    -0.9415249266924297,
                    -0.08067195141513692,
                    0.3271434680254041,
                    2.4909429428524836
                ],
                [
                    0.032414896492773564,
                    0.944725743158365,
                    0.326255336660165,
                    4.375235441944789
                ],
                [
                    0.0,
                    -0.0,
                    -0.0,
                    1.0
                ]
            ]
        },

...

Convert to NeRF

Can we convert OMAF JSON file to NeRF transforms.json File??