GithubHelp home page GithubHelp logo

tudelft3d / val3dity Goto Github PK

View Code? Open in Web Editor NEW
90.0 13.0 22.0 30.2 MB

Validation of 3D primitives according to the international standard ISO19107

License: GNU General Public License v3.0

CMake 2.23% C++ 63.70% Python 26.87% Shell 0.11% Dockerfile 0.58% CSS 0.03% HTML 3.67% JavaScript 2.81%
gis validation iso19107 gml cityjson indoorgml cgal 3d-gis

val3dity's Introduction

GitHub license docs webapp DOI

val3dity

val3dity---pronounced 'val-three-dity'---allows us to validate 3D primitives according to the international standard ISO19107. Think of it as PostGIS ST_IsValid, but for 3D primitives (PostGIS only validates 2D primitives).

In short, it verifies whether a 3D primitive respects the definition as given in ISO19107 and GML.

The validation of the following 3D primitives is fully supported:

  • MultiSurface
  • CompositeSurface
  • Solid
  • MultiSolid
  • CompositeSolid

Unlike many other validation tools in 3D GIS, inner rings in polygons/surfaces are supported and so are cavities in solids (also called voids or inner shells). However, as is the case for many formats used in practice, only planar and linear primitives are allowed: no curves or spheres or other parametrically-modelled primitives are supported. There is no plan to support these geometries.

val3dity accepts as input:

For the CityJSON and IndoorGML formats, extra validations are performed (specific to the format), eg the overlap between different parts of a building, or the validation of the navigation graph in IndoorGML.

Usage

val3dity is a command-line interface (CLI) programme only, with several options.

val3dity cli demo
No demo visible here? View it on asciinema.

Web application

If you don't want to go through the troubles of compiling and/or installing val3dity, we suggest you use the web application. Simply upload your file to our server and get a validation report back. We delete the file as soon as it has been validated (promised!). However, a file is limited to 50MB.

Installation of the command-line tool

macOS

The simplest, if you have Homebrew, is:

$ brew tap tudelft3d/software
$ brew install val3dity

You can also get the code, and compile it yourself. You first need to install the following free libraries:

  1. CGAL v5.4+
  2. Eigen library
  3. GEOS
  4. CMake

Under macOS, we suggest using Homebrew:

$ brew install cgal
$ brew install eigen
$ brew install geos
$ brew install cmake

and then

$ mkdir build
$ cd build
$ cmake ..
$ make

linux

Under Linux (at least Ubuntu), CGAL has to be compiled because apt-get doesn't give you a version with Eigen. Thus, in a nutshell,

  1. install Eigen library: $ sudo apt install libeigen3-dev
  2. install GEOS library: $ sudo apt install libgeos++-dev
  3. install CGAL: $ sudo apt-get install libcgal-dev - if it's not CGAL v5.4+ that is installed, download latest CGAL code and unzip somewhere - set CGAL_DIR to that folder, eg export CGAL_DIR=/home/hledoux/software/CGAL-5.4.2, this will tell your shell to use that version of CGAL (thus more version of CGAL can be installed on the same computer; see that handy manual)

Windows

For Windows, we offer an executable.

It's also possible to compile val3dity yourself with the CMake file we offer, but it's slightly more complex with the dependencies. We suggest you look at how we build it on GitHub Actions to get some inspirate (and help).

Usage

To compile and run val3dity (from the val3dity folder):

$ ./val3dity /data/cityjson/cube.json

The summary of the validation is reported, and you should see that cube.json contains one valid primitive.

Finally, to see all the options possible:

$ ./val3dity --help

Using val3dity as a library

val3dity can be compiled as a library:

$ cmake .. -DLIBRARY=true

There is a simple example in ./demo_lib with instructions to compile it.

Unit tests

To verify that everything went fine during the compilation, run the unit tests (from the root folder of val3dity) (this requires pip install pytest pyyaml):

$ python -m pytest --runfull

You shouldn't get any errors.

Simple visualiser for the errors: viz3dity

In the folder tools/viz3dity/, there is a simple Python script where you load a CityJSON file with geometries and one with the val3dity report, and you can see which objects have specific errors (each error gets a colour).

However, it's not possible to see where in an object the error is (eg which surface is not planar). This tool helps to quickly visualise where the errors are in a large dataset, for instance a city.

Documentation and help

Read the full documentation at https://val3dity.rtfd.io.

The primary channel to communicate with the developers is the Issues section.

If you have a question or came across a bug, please submit an issue there. However we ask you check first whether your problem has already been solved by someone else.

If you use val3dity in a scientific context, please cite these articles:

@article{Ledoux18,
    title = {{val3dity}: validation of {3D GIS} primitives according to the international standards},
    author = {Ledoux, Hugo},
    journal = {Open Geospatial Data, Software and Standards},
    year = {2018},
    volume = {3},
    number = {1},
    pages = {1},
    doi = {http://dx.doi.org/10.1186/s40965-018-0043-x}
}

@article{Ledoux13,
  title = {On the validation of solids represented with the international standards for geographic information},
  author = {Ledoux, Hugo},
  journal = {Computer-Aided Civil and Infrastructure Engineering},
  year = {2013},
  volume = {28},
  number = {9},
  pages = {693--706},
  doi = {http://dx.doi.org/10.1111/mice.12043}
}

Error codes

val3dity's People

Contributors

balazsdukai avatar bmmeijers avatar hugoledoux avatar ipadjen avatar jeffcoukell avatar johnzjq avatar kenohori avatar liberostelios avatar twut avatar vvmruder avatar weixiaogao avatar ylannl avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

val3dity's Issues

potential false positive on 305

On branch v2_test
Run pytest -k "306" -v
Test 306_2 returns 305 instead of 306. It is a unit cube with one extra face inside another face.
The description of 305 says "Polygons that are not connected to the shell should be reported as an error.". In my understanding a face within a face counts as connected. Is that correct?

remove POLY support from the doc

Remove POLY support from the doc, because it is only used for testing ( #49 )

  • remove it from the Accepted input section
  • remove the --ishell option

error 105 reported as 104

On branch v2_test
Test as $ pytest -v -k "test_105".
The test contains a

  • single polygon with 4 vertices and each of them are in line,
  • a unit cube where the top surface contains a polygon of 3 vertices collapsed into a line

Both tests report 104 but I think it should be 105.

snap_tolerance of 0 == crashes

because:

int tol = int(1 / _tol_snap);
std::string s = std::to_string(int64(p->x() * tol)) + std::to_string(int64(p->y() * tol)) + std::to_string(int64(p->z() * tol));

POLY parser stalls on a hole definition with boundary marker

On branch v2_test
Run $ pytest -v -k "test_valid_poly[valid_2.poly]"
The cube has a hole defined in it according the the .poly documentation but val3dity hangs when reading the file.

A sample from the file:

...
8 0.2 0.2 1.0
9 0.5 0.2 1.0
10 0.8 0.2 1.0
6 0
2 1 0           # a facet with two polygons and one hole, no boundary marker
4 4 5 6 7       # polygon 1
4 7 8 9 10      # polygon 2, this is the hole
1 0.7 0.3 1.0   # a point in polygon 2 to mark the hole
1 0
4 0 3 2 1 
...

Furthermore, the files in /data/parse_poly/ confirm that this problem occurs in a case when there is,

  • a cube, not in the case of a single 3D polygon,
  • a hole defined in a facet,
  • the facet definition 2 1 0 also contains a digit for the boundary marker, even though that is 0

validating complex solid

Hi, I have been using val3dity to validate the citygml files that I have generated. It is a very useful tool. However, I am facing trouble validating this file that I have generated (as shown in the figure ). I keep getting the error "WRONG_ORIENTATION_SHELL" from val3dity. As you can see from the figure, the normal of each polygon is already pointing outwards. Thus, I cannot really figure out what is really wrong with my geometry. I have checked numerous time, so I thought I could maybe get some help from val3dity? I have also attached the citygml file here. Thanks.

tower

example3_part.txt

error 304 instead of 303

On branch v2_test
Run pytest -k "303" -v.
It reports 304 although data contains non-manifold vertices. Maybe 303 and 304 could be merged into a NON_MANIFOLD_VERTEX_OR_EDGE category? Similarly as in the case with #52

Implement robustness option, à la Oracle Spatial

Robustness defined in 2D and used in Oracle Spatial should be implemented in 3D:
van Oosterom P, Quak W, and Tijssen T (2004). About invalid, valid and clean polygons. In Fisher PF, editor, Developments in Spatial Data Handling—11th International Symposium on Spatial Data Handling, pages 1–16. Springer.

  1. vertex-vertex distances
  2. vertex-plane distances

CityGML BuildingPart gml:id not found

On v2.
Run val3dity val3dity/data/gml/DenHaag_1BwBP.gml -r DenHaag_1BwBP.xml.
The gml:id of the BuildingParts is not found and the following report is produced:

...
<Building>
<id>GUID_7F91F413-F839-48F6-A36A-E0F2F090C6B0</id>
		<Error>
			<code>601</code>
			<type>BUILDINGPARTS_OVERLAP</type>
			<info> & </info>
			<primitives></primitives>
		</Error>
		<Error>
			<code>601</code>
			<type>BUILDINGPARTS_OVERLAP</type>
			<info> & </info>
			<primitives></primitives>
		</Error>
		<Error>
			<code>601</code>
			<type>BUILDINGPARTS_OVERLAP</type>
			<info> & </info>
			<primitives></primitives>
		</Error>
		<Error>
			<code>601</code>
			<type>BUILDINGPARTS_OVERLAP</type>
			<info> & </info>
			<primitives></primitives>
		</Error>
		<Error>
			<code>601</code>
			<type>BUILDINGPARTS_OVERLAP</type>
			<info> & </info>
			<primitives></primitives>
		</Error>
...

I think each error should reference the gml:id of the BuildingParts (eg GUID_7F91F413-F839-48F6-A36A-E0F2F090C6B0_6).

error 309 not reported for OFF and OBJ

On branch v2_test
Run pytest -k "309" -v
Each of the two files, 309.off and 309_1.obj contain one unused vertex. This should trigger a 309 error but it doesn't.

How to heal Error 203 & 204 types without triangulate of surfaces

We have created realistic roof surfaces from LiDAR point cloud data with planar tolerance of 3 cm, most of the data is turned into Error type: 203 (NON_PLANAR_POLYGON_DISTANCE_PLANE) and 204 (NON_PLANAR_POLYGON_NORMALS_DEVIATION).

We are able to identify above errors with val3dity tool.

But, how to heal these errors {Error type:203 (NON_PLANAR_POLYGON_DISTANCE_PLANE) and 204 (NON_PLANAR_POLYGON_NORMALS_DEVIATION)} types without triangulating of surfaces?

Please anyone can suggest a logic for aforesaid problem.

Regards
Govardhan

describe the .log

Describe that val3dity generates a log file in the working dir and desc. what does it contain.

an issue of false positive errors in Error type: 203 & 204

Hi val3dity team,

Thanks for providing such excellent tool for validating the CityGML 2.0 3D models geometrically.

We would like to request you to please have a look into below issue.

Currently we are using val3dity tool 1.2/1.3.1 version. We are facing an issue of false positive errors for Error type: 203 NON_PLANAR_POLYGON_DISTANCE_PLANE) and 204 (NON_PLANAR_POLYGON_NORMALS_DEVIATION), though these surfaces are in planar.

Herewith we are attaching some sample buildings for your quick reference, which are having planar surfaces. These have been validated through FME planer checks and it is also showing planar in Micro Station software.

Can you please give your valuable comments on this.

We look forward to your quick response.

307 also reports 304

On branch v2_test
Run pytest -k "307" -v
Test 307 reports 304, 307 although the unit cube does not contain non-manifold edge, just a face that is oriented in the opposite direction as the rest.

empty surface / shell ID tags in Error reports

Following the discussion in #42 about the empty <surface_id> tag where @hugoledoux mentioned the possibility to merge the surface_id and info tags in the error description. There are other tags that describe similar information. Would it make sense to keep the info tag and just consolidate the id tags that within the <Error> description? For example:

		<Error>
			<code>501</code>
			<type>INTERSECTION_SOLIDS</type>
			<id>Solid 1, Solid 2</id>
			<info></info>
		</Error>
     # or
		<Error>
			<code>302</code>
			<type>SHELL_NOT_CLOSED</type>
			<id>Surface 1</id>
			<info>Location hole: 0 0 0</info>
		</Error>
    # or
		<Error>
			<code>405</code>
			<type>WRONG_ORIENTATION_SHELL</type>
			<id>Shell i</id>
			<info></info>
		</Error>

Below are a few examples of the current id tags that I came across, are there more?

    # surface_id
		<Error>
			<code>302</code>
			<type>SHELL_NOT_CLOSED</type>
			<surface_id></surface_id>
			<info>Location hole: 0 0 0</info>
		</Error>
     # id of Solid
	<CompositeSolid>
		<id>elvisisalive</id>
		<Error>
			<code>501</code>
			<type>INTERSECTION_SOLIDS</type>
			<id>1 & 2</id>
			<info></info>
		</Error>
    # empty id
	<CompositeSolid>
		<id>elvisisalive</id>
		<Error>
			<code>503</code>
			<type>DISCONNECTED_SOLIDS</type>
			<id></id>
			<info>-->CompositeSolid is formed of 2 parts</info>
		</Error>
     # shell id
	<Solid>
		<Error>
			<code>405</code>
			<type>WRONG_ORIENTATION_SHELL</type>
			<shell>i</shell>
			<info></info>
		</Error>

POLY file could be valid but reports 201

On branch v2_test
Test $ pytest -v -k "test_valid_poly[valid_3.poly]" returns 201 but I think it should be valid, because TetView opens it without complaints and displays it as a closed Solid. It is a cube with the top rectangular surface is composed of 2 polygons.

add CONTRIBUTING

include:

  • communication channels
  • how to contribute issues, questions, comments etc.
  • how to contribute documentation
  • how to contribute tests
  • how to contribute code

Segfaulting with simple input

With simple input file, val3dity on Linux segfaults (although I am not sure what is wrong, the input file is dealt with properly by tetgen 1.5.0).

./val3dity  /tmp/boekie.poly 
Reading 1 file(s).
Reading outer shell:    /tmp/boekie.poly

Validating surface in 2D with GEOS (their projection)
-----all valid

Triangulating outer shell
-----done

Validating shell #0
-----Planarity
    yes
-----Combinatorial consistency
Error 306: some vertices are not referenced by faces
    [shell: #0; face: #-1]
    yes
-----Geometrical consistency
    yes
-----Orientation of normals
Segmentation fault

And the input file contents:

# Generated by 'solids.py' at 2014-01-30 13:01:17.741013
# Delft University of Technology, GIS technology
# Martijn Meijers <[email protected]>

# part 1 - node list
6 3 0 0
1 0 0 0
2 0 1 0
3 1 0 0
4 0 1 1
5 0 0 1
6 1 0 1

# Part 2 - facet list
# facet count, no boundary marker
5 0

# facet 1
1
3    1 2 3

# facet 2
1
3    4 5 6

# facet 3
1
4    1 5 4 2

# facet 4
1
4    1 3 6 5

# facet 5
1
4    3 2 4 6

# Part 3 - hole list
0            # no hole

# Part 4 - region list
0            # no region

POLY invalid hole definition not caugth by val3dity

On branch v2_test
Possibly related to #49
Test $ pytest -v -k "test_poly_hole"and val3dity reports 302, but I think it should be 901, because TetView doesn't open the file and returns Error: Hole 1 in facet 1 has no coordinatesNothing to draw!

closed cube reported as not closed

On branch v2_test.
Run pytest -k "closed_top" -v.
The top surface of the unit cube is composed of two polygons and it is closed, but reported as 302.

rounding of values in validation checks

On branch v2_test.
test_204_valid fails to report valid, with surface normal difference is 0.99971° (using tolerance 1°)
What is a reasonable precision to expect in this case?

flag to ignore error 204

these are caused by vertices very close (like 0.001mm) to each others.

Temporary solution is to put planarity_n option to 180, but I'll add an option soon (in branch v2).

specify location in error 302

On v2.
The file msol3 produces the error:

		<Error>
			<code>302</code>
			<type>SHELL_NOT_CLOSED</type>
			<surface_id></surface_id>
			<info>Location hole: 0 0 0</info>
		</Error>

Describe in docs:error what does Location hole: 0 0 0 refer to? What is 0 0 0?

Compilation on Linux

It does not compile on Linux as is. I will make a fork with minor changes (Linux is a bit more picky on case sensitive folder names, as is the case for CGAL).

poly2off.py doesn't work fully

Only if the input as no holes/inner rings.

Otherwise we need to triangulate, so better to use TetGen for that. v1.4.x had an OFF output that worked fine, now disappeared from v1.5.x

consider using an alternative character instead of `&` in the report

On v2.
Run val3dity /val3dity/data/gml/csol2.gml -r csol2_rep.xml.
The report doesn't open in a web browser (Firefox or Chromium), due to an XML parsing error of the & character. In the report it looks like:

		<Error>
			<code>501</code>
			<type>INTERSECTION_SOLIDS</type>
			<id>1 & 2</id>
			<info></info>
		</Error>

Maybe a , could be used instead?

error 204 reported as 104

On branch v2_test.
Run "pytest -v -k test_204".
Data sets 204_3 and 204_4 have a fold with a vertical side in the top face. Both are reported as 104 although the face has no self-intersection, just the fold.

online val3dity complex solid

Hi, thanks for the really fast fix to the bug. But I just tried the file on the online val3dity, its still giving me the error. Is the online version not updated ?

handle missing primitive IDs in the report's <info> element

On v2.
Run val3dity val3dity/data/gml/DenHaag_1BwBP.gml -r DenHaag_1BwBP.xml
The report contains:

<id>GUID_7F91F413-F839-48F6-A36A-E0F2F090C6B0</id>
		<Error>
			...
			<info> & </info>
			...
		</Error>
	<Solid>
		<id>none</id>
                ...
	</Solid>
	<Solid>
		<id>none</id>
                ...
	</Solid>

Inserting a special keyword (eg none) instead of the empty string into the <info> element could simplify the parsing of the report or prevent errors.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.