GithubHelp home page GithubHelp logo

ybayle / scyland3d Goto Github PK

View Code? Open in Web Editor NEW
9.0 4.0 8.0 370 KB

Scyland3D: Processing 3D landmarks

License: MIT License

Python 88.91% TeX 8.77% Shell 2.32%
python landmark landmark3d landmarks shape-analysis geometric-morphometric 3d semilandmark

scyland3d's Introduction

Scyland3D: Processing 3D landmarks

Scyland3D is a Python tool for converting 3D raw landmark and semilandmark coordinates exported from landmark acquisition software (e.g. Landmark Editor 3.6 from IDAV, UC Davis) to a csv format best suited for geometric morphometric analyses. Processing schemes to mirror and reorder these points are provided to address further symmetry issues.

By:

Installation

Install and update using pip:

pip install -U Scyland3D

Dependencies:

  • Python 2 or 3. Scyland3D has been tested with Python 2.7.16 and 3.6.3 and should work with older versions of Python.
  • Numpy >= 1.13.3. Scyland3D has been tested with Numpy 1.13.3 and 1.16.2 and should work with older versions of Numpy.

If you encounter an error with other versions, please submit an issue.

Use-case example

To quickly process landmarks files, you can give Scyland3D the path to the folder containing the .pts files (format defined on page 37 in http://www.idav.ucdavis.edu/research/projects/EvoMorph/supplement/LandmarkDoc_v3_b6.pdf), as shown in the example files provided:

from Scyland3D import Scyland3D
Scyland3D.pts2csv("example/")

This will create a file named landmarks.csv that you can use for statistical analyses in any language/software you want. The example files provided come from a subset of our dataset for studying teeth of the small-spotted catshark Scyliorhinus canicula. We provide 7 landmark and 31 semilandmark coordinates for 2 upper teeth and for 3 lower ones, stored in two folders that describe the specimens they belong to. A landmark is a point set up by hand as opposed to a semilandmark that is interpolated by the computer between two landmarks by following the curvature of the studied form.

Below, the image A is the representation of 7 landmarks for an input right shark tooth and the image B is the output produced by Scyland3D when using the options to mirror and reorder the landmarks. This process can be used to process right teeth in order to superimpose and compare them with left teeth in order to compare analogous elements. More information can be found on the related article.

Landmarks on a tooth

API documentation

The function pts2csv() is the core of Scyland3D and is also called when using the script from the terminal with the following arguments:

  • indir (required)
    • A string containing the input directory where the files are stored. File names can specify the feature modalities separated by _ (e.g. speciesA_ageX_sex1.pts).
    • Example: Scyland3D.pts2csv(indir="path/")
    • Command line: python -m Scyland3D.Scyland3D -i "path/"
  • outdir (optional)
    • A string containing the output directory where the files will be generated.
    • Default: Use the current folder.
    • Example: Scyland3D.pts2csv(indir="path/", outdir="outpath/")
    • Command line: python -m Scyland3D.Scyland3D -i "path/" -o "path/"
  • order (optional)
    • A list of integer indicating the new order to apply to the landmarks. The order_factor argument must also be supplied.
    • Default: None
    • Example: Scyland3D.pts2csv(indir="path/", order_factor="upper", order=[36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 37])
    • Command line: python -m Scyland3D.Scyland3D -i "path/" -f "upper" -r "36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 37"
  • order_factor (optional)
    • A string containing the keyword for items that need their landmarks and semilandmarks to be reordered. The order argument must also be supplied. Warning: no checks are made on the factor, so be sure to avoid any typo.
    • Default: None
    • Example: if the dataset contains lower and upper teeth where only the upper teeth need to be reordered, one can use: Scyland3D.pts2csv(indir="path/", order_factor="upper", order=[36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 37])
    • Command line: python -m Scyland3D.Scyland3D -i "path/" -f "upper" -r "36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 37". This is only valid if the keyword upper is present in the file name.
  • feature_names (optional)
    • A list of string containing the feature names.
    • Default: None
    • Example: Scyland3D.pts2csv(indir="path/", feature_names=["identifier", "species", "location", "length", "sex", "stage", "jaw", "position", "generation"])
    • Command line: python -m Scyland3D.Scyland3D -i "path/" -n "identifier,species,location,length,sex,stage,jaw,position,generation"
  • verbose (optional)
    • A boolean indicating if information should be printed on the console by the process.
    • Default: False
    • Example: Scyland3D.pts2csv(indir="path/", verbose=True) displays information to the console.
    • Command line: python -m Scyland3D.Scyland3D -i "path/" -v displays information to the console.
  • mirror_factor (optional)
    • A string containing the keyword for items to be mirrored in the 3D space. Warning: no checks are made on the factor, so be sure to avoid any typo.
    • Default: None
    • Example: Scyland3D.pts2csv(indir="path/", mirror_factor="upper")
    • Command line: python -m Scyland3D.Scyland3D -i "path/" -m "upper" will mirror in 3D the landmarks stored in .pts files with the keyword upper in their filename. Then, it will export to a .csv file those mirrored landmarks and also the unprocessed landmarks from .pts files that do not contain the keyword upper in their filename. The figure above depicts the mirroring.

How To Contribute

Contributions are welcome! Please refer to the CONTRIBUTING.md file.

License

Scyland3D is licensed under the MIT License as described in the license file. Please use the following citation:

@article{Berio2019,
    author  = {Berio, Fidji and Bayle, Yann},
    title   = {{Scyland3D: Processing 3D landmarks}},
    journal = {{The Journal of Open Source Software}},
    page    = {1153--1155},
    volume  = {4},
    issue   = {33},
    doi     = {10.21105/joss.01153},
    note    = {Review pending}
}

Acknowledgements

  • We acknowledge the contribution of SFR Biosciences (UMS3444/CNRS, US8/Inserm, ENS de Lyon, UCBL) facilities: AniRA-ImmOs and Mathilde Bouchet for her help with X-ray microtomography.
  • We are thankful to @patrikhuber and @r-barnes, the two reviewers from JOSS that helped improved a lot the code and the documentation.

scyland3d's People

Contributors

ybayle avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

scyland3d's Issues

API documentation

The rubric asks

Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?

While the documentation of the API in the read me meets the standard, the documentation within the code itself is lacking. Please include complete documentation within the code itself (see, for instance, this).

CSV output

The export2csv() and write_header() functions make me nervous because they have a lot of handwritten code, which seems unnecessary because python has a core CSV module that seems as though it could handle much of this. Is it possible to make use of this module to simplify the code?

Dependencies

Could you clarify whether this works with Python 2, Python 3, or both? I don't think of support for Python 2 is necessary, but support for Python 3 is.

Mistake / old information in Readme for command-line arguments

I think I found another piece of information in the Readme that doesn't match the actual command-line parameters (perhaps you forgot to update those when you switched to the new cmdline options module).

Readme says:

Command line: python -m Scyland3D.Scyland3D -v False for removing the display of information to the console.

Output:

python -m Scyland3D.Scyland3D -v False
usage: Scyland3D.py [-h] [-i INDIR] [-o OUTDIR] [-m MIRROR_FACTOR] [-r ORDER]
                    [-f ORDER_FACTOR] [-n FEATURE_NAMES] [-v]
Scyland3D.py: error: unrecognized arguments: False

Could you also check all the other examples in the readme please whether they work as expected?

Example usage

I think the current example, as shown in the read me, would benefit from additional pictures such as the one showing in the paper. Could you link up the paper image into the read me?

I think the read me might also benefit from additional images explaining what a semi Landmark is, as well as showing pictorially what the software can do.

private functions

I also note that the code includes several private functions, please begin these function names with an underscore. This is often used to indicate a private function (source).

statement of need

The rubric asks

Do the authors clearly state what problems the software is designed to solve and who the target audience is?

I feel that the paper as written takes a long time to address this. Perhaps it is possible to succinctly describe the need within the abstract?

`AssertionError: Must supply order and order_factor` when supplying -f and -o

I was trying to play around with the command-line, using the following line from the readme (slightly modified - I added -i ...):

python -m Scyland3D.Scyland3D -i scyland3d-review/Scyland3D/Scyland3D/example/ -f "upper" -o "36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 37"

And I get AssertionError: Must supply order and order_factor.. But I did supply both (which should be the -f and -o), so I am a bit confused.

Full output:

$ python -m Scyland3D.Scyland3D -i scyland3d-review/Scyland3D/Scyland3D/example/ -f "upper" -o "36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3, 2, 1, 0, 37"
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/patrik/.virtualenvs/scyland3d/lib/python3.6/site-packages/Scyland3D/Scyland3D.py", line 492, in <module>
    pts2csv(args.indir, args.outdir, args.mirror_factor, args.order, args.order_factor, args.feature_names, args.verbose)
  File "/home/patrik/.virtualenvs/scyland3d/lib/python3.6/site-packages/Scyland3D/Scyland3D.py", line 231, in pts2csv
    ), "Must supply order and order_factor."
AssertionError: Must supply order and order_factor.

Running without arguments fails

Configuration shows a command-line entry point

    entry_points='''
        [console_scripts]
        Scyland3D=Scyland3D:pts2csv
    ''',

Running Scyland3D on the command line raises an error:

rick@sally:/z$ Scyland3D 
Traceback (most recent call last):
  File "/home/rick/.local/bin/Scyland3D", line 11, in <module>
    sys.exit(pts2csv())
TypeError: pts2csv() missing 1 required positional argument: 'indir'

Comments in code

The source code tells the computer what to do, but does not explain what you intended the computer to do. Given the speed with which the open-source world changes, and the lack of incentives to maintain software with academia, I think it's valuable to have comments within code so that people providing patches, new features, or updating the code have an easy way of orientating themselves to the intended functionality of what can otherwise be mysterious code blocks.

Please add additional comments within the code.

AssertionError when running tests

Hi,

When I clone the git repo and run the tests, I'm getting the following error:

(scyland3d-review) patrik:/mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D$ ../test.sh
Testing call from python...
Using default example directory: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/example/
File successfully generated: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/landmarks.csv
Using default example directory: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/example/
File successfully generated: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/landmarks_reordered.csv
Using default example directory: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/example/
File successfully generated: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/landmarks_reversed.csv
Testing default behavior...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/Scyland3D.py", line 486, in test_no_regression
    _validation_against_ref()
  File "/mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/Scyland3D.py", line 391, in _validation_against_ref
    dirn + "landmarks.csv", dirn + "test/landmarks_ref.csv"
  File "/mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/Scyland3D.py", line 383, in _same_file
    assert line_ref == line_test, "Invalid line " + line_ref
AssertionError: Invalid line example/200118G_canicula_mediterranean_40_female_mature_lower_tooth04_4.pts,23.760645,14.822544,26.607994,23.460169,14.72399,26.149445,23.619444,14.667845,26.251095,23.519606,14.463796,25.761469,23.799814,14.451308,26.218939,23.817883,14.327648,26.021515,24.05011,14.500256,26.583118,23.731268,14.851231,26.612024,23.693602,14.879261,26.590559,23.657339,14.899388,26.562233,23.620415,14.917179,26.531328,23.591503,14.916238,26.493502,23.580864,14.887245,26.456419,23.555262,14.877494,26.415607,23.52702,14.865196,26.368336,23.579859,14.780132,26.356867,23.546961,14.772367,26.300602,23.504704,14.757745,26.230452,23.531639,14.690864,26.178947,23.574799,14.58428,26.049805,23.669212,14.445662,26.010353,23.800825,14.386982,26.109592,23.858444,14.317473,26.077599,23.894411,14.332585,26.170313,23.92621,14.366543,26.286236,23.955145,14.364937,26.324955,23.982096,14.327062,26.290644,24.00683,14.300524,26.272785,24.027893,14.302449,26.299009,24.04743,14.317135,26.343693,24.0672,14.343269,26.407444,24.082382,14.387399,26.492682,24.071802,14.447412,26.555712,23.957222,14.536147,26.573864,23.874788,14.572289,26.542034,23.817228,14.613439,26.521942,23.779778,14.664021,26.510342,23.76063,14.742847,26.568264,200118G,canicula,mediterranean,40,female,mature,lower,tooth04,4

Testing call from the command line...
Using default example directory: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/example/
File successfully generated: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/landmarks.csv
Using default example directory: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/example/
Traceback (most recent call last):
  File "Scyland3D.py", line 490, in <module>
    main(sys.argv[1:])
  File "Scyland3D.py", line 374, in main
    verbose=verbose,
  File "Scyland3D.py", line 211, in pts2csv
    ), "Must supply order and order_factor."
AssertionError: Must supply order and order_factor.
Using default example directory: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/example/
File successfully generated: /mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/landmarks_reversed.csv
Testing default behavior...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/Scyland3D.py", line 391, in _validation_against_ref
    dirn + "landmarks.csv", dirn + "test/landmarks_ref.csv"
  File "/mnt/c/Users/Patrik/Projects/Scyland3D-review/Scyland3D/Scyland3D/Scyland3D.py", line 383, in _same_file
    assert line_ref == line_test, "Invalid line " + line_ref
AssertionError: Invalid line example/200118G_canicula_mediterranean_40_female_mature_lower_tooth04_4.pts,23.760645,14.822544,26.607994,23.460169,14.72399,26.149445,23.619444,14.667845,26.251095,23.519606,14.463796,25.761469,23.799814,14.451308,26.218939,23.817883,14.327648,26.021515,24.05011,14.500256,26.583118,23.731268,14.851231,26.612024,23.693602,14.879261,26.590559,23.657339,14.899388,26.562233,23.620415,14.917179,26.531328,23.591503,14.916238,26.493502,23.580864,14.887245,26.456419,23.555262,14.877494,26.415607,23.52702,14.865196,26.368336,23.579859,14.780132,26.356867,23.546961,14.772367,26.300602,23.504704,14.757745,26.230452,23.531639,14.690864,26.178947,23.574799,14.58428,26.049805,23.669212,14.445662,26.010353,23.800825,14.386982,26.109592,23.858444,14.317473,26.077599,23.894411,14.332585,26.170313,23.92621,14.366543,26.286236,23.955145,14.364937,26.324955,23.982096,14.327062,26.290644,24.00683,14.300524,26.272785,24.027893,14.302449,26.299009,24.04743,14.317135,26.343693,24.0672,14.343269,26.407444,24.082382,14.387399,26.492682,24.071802,14.447412,26.555712,23.957222,14.536147,26.573864,23.874788,14.572289,26.542034,23.817228,14.613439,26.521942,23.779778,14.664021,26.510342,23.76063,14.742847,26.568264,200118G,canicula,mediterranean,40,female,mature,lower,tooth04,4

The Invalid line error could in theory be a newline inconsistency since I'm using WSL - just a guess. The AssertionError: Must supply order and order_factor. error looks like something else.

Any idea what I might be doing wrong?

Community guidelines

My review rubric asks:

Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support

The current guidelines (CONTRIBUTING.md)

Contributions are welcome! If you found a bug, please submit an issue here. If you are willing to propose new features, please submit a pull request here.

don't quite seem to cover this. I'd suggest:

Contributions are welcome! If you find a bug or need support, please submit an issue here. If you are willing to propose new features or have bug fixes to contribute, please submit a pull request here.

Having trouble running Scyland3D when installed from pip

Hi!

If I clone the repo from git and go into the Scyland3D\Scyland3D directory on a command-line, I can successfully run the example and it creates the .csv file. However, when I create a new virtualenv and do pip install Scyland3D, then I can still import the module without error, but it doesn't seem to have any functions. help(Scyland3D) then looks like an empty module and trying to call any function fails, e.g. AttributeError: module 'Scyland3D' has no attribute 'pts2csv'.

I tried this on both Windows and on WSL Ubuntu 18.04.

Anything that I might be doing wrong here?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.