osmcode / libosmium Goto Github PK
View Code? Open in Web Editor NEWFast and flexible C++ library for working with OpenStreetMap data.
Home Page: https://osmcode.org/libosmium/
License: Boost Software License 1.0
Fast and flexible C++ library for working with OpenStreetMap data.
Home Page: https://osmcode.org/libosmium/
License: Boost Software License 1.0
The following program creates a Segfault when applied to Geofabrik's Switzerland PBF file ( http://download.geofabrik.de/europe/switzerland-latest.osm.pbf ).
#include <osmium/index/map/dummy.hpp>
#include <osmium/index/map/sparse_table.hpp>
#include <osmium/index/map/stl_map.hpp>
#include <osmium/index/map/mmap_vector_anon.hpp>
#include <osmium/handler/node_locations_for_ways.hpp>
#include <osmium/visitor.hpp>
#include <osmium/io/any_input.hpp>
#include <osmium/handler.hpp>
#include <geos/util/IllegalArgumentException.h>
#include <geos/util/GEOSException.h>
#include <osmium/io/any_input.hpp>
#include <osmium/handler.hpp>
typedef osmium::index::map::Dummy<osmium::unsigned_object_id_type, osmium::Location> index_neg_type;
typedef osmium::index::map::SparseTable<osmium::unsigned_object_id_type, osmium::Location> index_pos_type;
typedef osmium::handler::NodeLocationsForWays<index_pos_type, index_neg_type> location_handler_type;
class MyHandler : public osmium::handler::Handler {
public:
MyHandler() :
osmium::handler::Handler() {
}
}; // class MyHandler
int main(int argc, char* argv[]) {
if (argc != 2) {
std::cerr << "Usage: " << argv[0] << " OSMFILE\n";
exit(1);
}
MyHandler handler;
osmium::io::Reader reader(argv[1]);
index_pos_type index_pos;
index_neg_type index_neg;
location_handler_type location_handler(index_pos, index_neg);
osmium::apply(reader, location_handler, handler);
return 0;
}
Makefile:
#CXXFLAGS += -O3
CXXFLAGS += -g
CXXFLAGS += -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
OS:=$(shell uname -s)
ifeq ($(OS),Darwin)
CXXFLAGS += -stdlib=libc++
LDFLAGS += -stdlib=libc++
endif
CXXFLAGS_WARNINGS := -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast
LIB_EXPAT := -lexpat
LIB_PBF := -pthread -lz -lprotobuf-lite -losmpbf
LIB_BZIP2 := -lbz2
PROGRAMS := segfault
.PHONY: all clean
all: $(PROGRAMS)
segfault: main.cpp
$(CXX) $(CXXFLAGS) $(CXXFLAGS_WARNINGS) -o $@ $< $(LDFLAGS) $(LIB_EXPAT) $(LIB_PBF) $(LIB_BZIP2)
clean:
rm -f *.o core $(PROGRAMS)
gdb Output:
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff675a700 (LWP 10911)]
[New Thread 0x7ffff5dd3700 (LWP 10912)]
[New Thread 0x7ffff55d2700 (LWP 10913)]
[New Thread 0x7ffff4dd1700 (LWP 10914)]
Program received signal SIGSEGV, Segmentation fault.
0x0000000000434deb in google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> >::sparsegroup (this=0x2981acb0, x=...)
at /usr/include/google/sparsetable:795
795 sparsegroup(const sparsegroup& x) : group(0), settings(x.settings) {
(gdb) bt
#0 0x0000000000434deb in google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> >::sparsegroup (this=0x2981acb0,
x=...) at /usr/include/google/sparsetable:795
#1 0x000000000043ac91 in google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > >::construct (this=0x7fffffffdcb0, p=0x2981acb0, val=...) at /usr/include/google/sparsehash/libc_allocator_with_realloc.h:78
#2 0x00000000004392a6 in std::allocator_traits<google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > > >::_S_construct<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> ><google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > const&> >(google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > >&, std::allocator_traits<google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > > >::__construct_helper*, (google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> ><google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > const&>&&)...) (__a=...,
__p=0x2981acb0) at /usr/include/c++/4.8/bits/alloc_traits.h:254
#3 0x0000000000437717 in std::allocator_traits<google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > > >::construct<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> ><google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > const&> >(google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > >&, google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> ><google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > const&>*, (google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> ><google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > const&>&&)...) (__a=..., __p=0x2981acb0) at /usr/include/c++/4.8/bits/alloc_traits.h:393
#4 0x0000000000435031 in std::__uninitialized_fill_n_a<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> >*, unsigned long, google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> >, google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > > > (__first=0x2981acb0, __n=669, __x=..., __alloc=...)
at /usr/include/c++/4.8/bits/stl_uninitialized.h:320
#5 0x0000000000432c1b in std::vector<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> >, google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > > >::_M_fill_insert (this=0x7fffffffdcb0,
__position=..., __n=669, __x=...) at /usr/include/c++/4.8/bits/vector.tcc:488
#6 0x000000000042f680 in std::vector<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> >, google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > > >::insert (this=0x7fffffffdcb0, __position=...,
__n=669, __x=...) at /usr/include/c++/4.8/bits/stl_vector.h:1024
#7 0x000000000042afe9 in std::vector<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> >, google::libc_allocator_with_realloc<google::sparsegroup<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> > > >::resize (this=0x7fffffffdcb0,
__new_size=43523432, __x=...) at /usr/include/c++/4.8/bits/stl_vector.h:687
#8 0x000000000043948c in google::sparsetable<osmium::Location, (unsigned short)48, google::libc_allocator_with_realloc<osmium::Location> >::resize (this=0x7fffffffdcb0,
new_size=2089124703) at /usr/include/google/sparsetable:1317
#9 0x0000000000437917 in osmium::index::map::SparseTable<unsigned long, osmium::Location>::set (this=0x7fffffffdca0, id=2089114703, value=...)
at /usr/include/osmium/index/map/sparse_table.hpp:91
#10 0x000000000043535e in osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long, osmium::Location>, osmium::index::map::Dummy<unsigned long, osmium::Location> >::node (this=0x7fffffffdc80, node=...) at /usr/include/osmium/handler/node_locations_for_ways.hpp:102
---Type <return> to continue, or q <return> to quit---
#11 0x000000000043317c in osmium::visitor::detail::switch_on_type<osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long, osmium::Location>, osmium::index::map::Dummy<unsigned long, osmium::Location> >, osmium::memory::Item> (visitor=..., item=...) at /usr/include/osmium/visitor.hpp:101
#12 0x000000000042f963 in osmium::visitor::detail::apply_item_recurse<osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long, osmium::Location>, osmium::index::map::Dummy<unsigned long, osmium::Location> >, osmium::memory::Item> (item=..., visitor=...) at /usr/include/osmium/visitor.hpp:153
#13 0x000000000042b577 in osmium::visitor::detail::apply_item_recurse<osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long, osmium::Location>, osmium::index::map::Dummy<unsigned long, osmium::Location> >, osmium::memory::Item, MyHandler> (item=..., visitor=...) at /usr/include/osmium/visitor.hpp:158
#14 0x0000000000425309 in osmium::apply<osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>, osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long, osmium::Location>, osmium::index::map::Dummy<unsigned long, osmium::Location> >, MyHandler> (it=..., end=...) at /usr/include/osmium/visitor.hpp:242
#15 0x000000000041ea52 in osmium::apply<osmium::io::Reader, osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long, osmium::Location>, osmium::index::map::Dummy<unsigned long, osmium::Location> >, MyHandler> (source=...) at /usr/include/osmium/visitor.hpp:249
#16 0x00000000004069d6 in main (argc=2, argv=0x7fffffffdf18) at main.cpp:45
(gdb)
A number of the tests fail on a 32 bit platform with errors like this:
c++ -I../include -Iinclude -g -std=c++11 -Werror -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o tests test_main.o t/basic/test_changeset.cpp
--------------------------
In file included from ../include/osmium/memory/collection.hpp:40:0,
from ../include/osmium/osm/changeset.hpp:39,
from t/basic/test_changeset.cpp:3:
../include/osmium/memory/item.hpp: In function 'osmium::memory::item_size_type osmium::memory::padded_length(osmium::memory::item_size_type)':
../include/osmium/memory/item.hpp:59:31: error: redefinition of 'osmium::memory::item_size_type osmium::memory::padded_length(osmium::memory::item_size_type)'
inline item_size_type padded_length(item_size_type length) noexcept {
^
../include/osmium/memory/item.hpp:55:23: error: 'size_t osmium::memory::padded_length(size_t)' previously defined here
inline size_t padded_length(size_t length) noexcept {
^
The problem is that on a 32 bit platform size_t
is uint32_t
and therefore the same as item_size_type
and we wind up with duplicate function definitions.
If one uses the wizard-like make_osmium_project.sh, the resulting code doesn't compile (anymore).
All Osmium objects should get support for writing them to streams (ie with operator<<). (And not only ostream but basic_ostream probably.) This should be in extra header files so that those don't have to be included if you don't need them.
For many objects there are several ways they could be sent to an ostream. Maybe we only need one implementation, maybe we need some way of changing the way objects are formatted in ostreams (like with basic types such as int that can be written in decimal, hex, oct, etc.).
The recent changes for Windows have surfaced a few problems where code accessing the file system was Linux/OSX specific. We have fixed all those issues with the necessary #ifdefs
etc. but those were all ad-hoc fixes here and there and no systematic approach. And it made the code a big uglier and harder to maintain.
There are a few ways we could take:
@alex85k, @springmeyer, @DennisOSRM: Opinions?
The reader test is failing for me seemingly for both PBF and XML input, with gcc 4.8.3 and the git master version of osmpbf:
Checking \033[1mtest/t/io/test_reader.cpp\033[0m ..................... \033[31m[COMPILE ERROR]\033[0m
==========================
c++ -I../include -Iinclude -g -std=c++11 -Werror -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o tests test_main.o t/io/test_reader.cpp -lbz2 -lexpat -lz -pthread -lz -lprotobuf-lite -losmpbf
--------------------------
In file included from ../include/osmium/io/detail/input_format.hpp:43:0,
from ../include/osmium/io/reader.hpp:55,
from ../include/osmium/io/pbf_input.hpp:36,
from ../include/osmium/io/any_input.hpp:38,
from t/io/test_reader.cpp:4:
../include/osmium/io/file.hpp:150:13: error: unused parameter ‘other’ [-Werror=unused-parameter]
File(const File& other) = default;
^
In file included from ../include/osmium/io/reader.hpp:55:0,
from ../include/osmium/io/pbf_input.hpp:36,
from ../include/osmium/io/any_input.hpp:38,
from t/io/test_reader.cpp:4:
../include/osmium/io/detail/input_format.hpp: In constructor ‘osmium::io::detail::InputFormat::InputFormat(const osmium::io::File&, osmium::osm_entity_bits::type, osmium::thread::Queue<std::basic_string<char> >&)’:
../include/osmium/io/detail/input_format.hpp:78:46: note: synthesized method ‘osmium::io::File::File(const osmium::io::File&)’ first required here
m_input_queue(input_queue) {
^
In file included from ../include/osmium/io/xml_input.hpp:37:0,
from ../include/osmium/io/any_input.hpp:39,
from t/io/test_reader.cpp:4:
../include/osmium/io/detail/xml_input_format.hpp: In instantiation of ‘std::__future_base::_Task_state<_Fn, _Alloc, _Res(_Args ...)>::_Impl::_Impl(_Fn2&&, const _Alloc&) [with _Fn2 = osmium::io::detail::XMLParser; _Fn = osmium::io::detail::XMLParser; _Alloc = std::allocator<int>; _Res = bool; _Args = {}]’:
/usr/include/c++/4.8.3/future:1275:41: required from ‘std::__future_base::_Task_state<_Fn, _Alloc, _Res(_Args ...)>::_Task_state(_Fn2&&, const _Alloc&) [with _Fn2 = osmium::io::detail::XMLParser; _Fn = osmium::io::detail::XMLParser; _Alloc = std::allocator<int>; _Res = bool; _Args = {}]’
/usr/include/c++/4.8.3/ext/new_allocator.h:120:4: required from ‘void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = std::__future_base::_Task_state<osmium::io::detail::XMLParser, std::allocator<int>, bool()>; _Args = {osmium::io::detail::XMLParser, const std::allocator<int>&}; _Tp = int]’
/usr/include/c++/4.8.3/bits/alloc_traits.h:254:4: required from ‘static typename std::enable_if<std::allocator_traits<_Alloc>::__construct_helper<_Tp, _Args>::value, void>::type std::allocator_traits<_Alloc>::_S_construct(_Alloc&, _Tp*, _Args&& ...) [with _Tp = std::__future_base::_Task_state<osmium::io::detail::XMLParser, std::allocator<int>, bool()>; _Args = {osmium::io::detail::XMLParser, const std::allocator<int>&}; _Alloc = std::allocator<int>; typename std::enable_if<std::allocator_traits<_Alloc>::__construct_helper<_Tp, _Args>::value, void>::type = void]’
/usr/include/c++/4.8.3/bits/alloc_traits.h:393:57: required from ‘static decltype (_S_construct(__a, __p, (forward<_Args>)(std::allocator_traits::construct::__args)...)) std::allocator_traits<_Alloc>::construct(_Alloc&, _Tp*, _Args&& ...) [with _Tp = std::__future_base::_Task_state<osmium::io::detail::XMLParser, std::allocator<int>, bool()>; _Args = {osmium::io::detail::XMLParser, const std::allocator<int>&}; _Alloc = std::allocator<int>; decltype (_S_construct(__a, __p, (forward<_Args>)(std::allocator_traits::construct::__args)...)) = <type error>]’
/usr/include/c++/4.8.3/bits/shared_ptr_base.h:400:38: required from ‘std::_Sp_counted_ptr_inplace<_Tp, _Alloc, _Lp>::_Sp_counted_ptr_inplace(_Alloc, _Args&& ...) [with _Args = {osmium::io::detail::XMLParser, const std::allocator<int>&}; _Tp = std::__future_base::_Task_state<osmium::io::detail::XMLParser, std::allocator<int>, bool()>; _Alloc = std::allocator<int>; __gnu_cxx::_Lock_policy _Lp = (__gnu_cxx::_Lock_policy)2u]’
/usr/include/c++/4.8.3/ext/new_allocator.h:120:4: [ skipping 6 instantiation contexts, use -ftemplate-backtrace-limit=0 to disable ]
/usr/include/c++/4.8.3/bits/shared_ptr.h:598:39: required from ‘std::shared_ptr<_Tp1> std::allocate_shared(const _Alloc&, _Args&& ...) [with _Tp = std::__future_base::_Task_state<osmium::io::detail::XMLParser, std::allocator<int>, bool()>; _Alloc = std::allocator<int>; _Args = {osmium::io::detail::XMLParser, const std::allocator<int>&}]’
/usr/include/c++/4.8.3/future:1318:76: required from ‘std::shared_ptr<std::__future_base::_Task_state_base<_Signature> > std::__create_task_state(_Fn&&, const _Alloc&) [with _Signature = bool(); _Fn = osmium::io::detail::XMLParser; _Alloc = std::allocator<int>]’
/usr/include/c++/4.8.3/future:1370:35: required from ‘std::packaged_task<_Res(_ArgTypes ...)>::packaged_task(std::allocator_arg_t, const _Alloc&, _Fn&&) [with _Fn = osmium::io::detail::XMLParser; _Alloc = std::allocator<int>; <template-parameter-2-3> = void; _Res = bool; _ArgTypes = {}]’
/usr/include/c++/4.8.3/future:1360:27: required from ‘std::packaged_task<_Res(_ArgTypes ...)>::packaged_task(_Fn&&) [with _Fn = osmium::io::detail::XMLParser; <template-parameter-2-2> = void; _Res = bool; _ArgTypes = {}]’
../include/osmium/thread/checked_task.hpp:55:85: required from ‘osmium::thread::CheckedTask<T>::CheckedTask(TArgs&& ...) [with TArgs = {osmium::thread::Queue<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >&, osmium::thread::Queue<osmium::memory::Buffer>&, std::promise<osmium::io::Header>&, osmium::osm_entity_bits::type&, std::atomic<bool>&}; T = osmium::io::detail::XMLParser]’
../include/osmium/io/detail/xml_input_format.hpp:686:102: required from here
../include/osmium/io/detail/xml_input_format.hpp:305:17: error: unused parameter ‘other’ [-Werror=unused-parameter]
XMLParser(XMLParser&& other) = default;
^
In file included from ../include/osmium/thread/checked_task.hpp:37:0,
from ../include/osmium/io/reader.hpp:62,
from ../include/osmium/io/pbf_input.hpp:36,
from ../include/osmium/io/any_input.hpp:38,
from t/io/test_reader.cpp:4:
/usr/include/c++/4.8.3/future:1307:49: note: synthesized method ‘osmium::io::detail::XMLParser::XMLParser(osmium::io::detail::XMLParser&&)’ first required here
: _Alloc(__a), _M_fn(std::forward<_Fn2>(__fn)) { }
^
cc1plus: all warnings being treated as errors
==========================
Currently input and output of osm.gz and osm.bz2 use the external programs gzip and bzip2. When we didn't have multithreading this wasn't such a bad idea, because at least the compressing and umcompressing was done in a separate process speeding things up. But it makes everything a little more brittle (what happens if it can't find that executable?) and, more importantly, it doesn't work on Windows, where we can't just fork the process.
Now that we have multithreading in Osmium, compressing and uncompressing should be done in extra threads using the respective libraries.
A similiar issue is the use of the "curl" executable for downloading OSM files from a URL. This should also be done in a thread using the curl or some other http library.
c++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.9.sdk/usr/include/libxml2 -o osmium_convert osmium_convert.cpp -stdlib=libc++ -lexpat -pthread -lz -lprotobuf-lite -losmpbf -lxml2
In file included from osmium_convert.cpp:14:
In file included from ../include/osmium/io/any_output.hpp:37:
../include/osmium/io/pbf_output.hpp:287:29: error: assigning to
'Bytef *' (aka 'unsigned char *') from incompatible type
'const unsigned char *'
...= reinterpret_cast<const unsigned char*>(in.c_str());
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
make: *** [osmium_convert] Error 1
The cmake build for all includes currently builds them all in one file. The make build builds them each separately. The separate build is intentional so that we see that they work on their own, have all needed include directives etc. The cmake build should probably work the same.
This issue can be reproduced by setting limit in collector.hpp:499 to a low value. Then purge_removed() is called while assembling multipolygons.
0 osmium::memory::Item::type item.hpp 158 0x5a6bf6
1 osmium::memory::ItemIteratorosmium::OSMEntity::advance_to_next_item_of_right_type item_iterator.hpp 108 0x60a969
2 osmium::memory::ItemIteratorosmium::OSMEntity::operator++ item_iterator.hpp 130 0x624826
3 osmium::memory::Buffer::purge_removedosmium::relations::Collector<osmium::area::MultipolygonCollector<osmium::area::Assembler, false, true, false>> buffer.hpp 507 0x62359d
4 osmium::relations::Collectorosmium::area::MultipolygonCollector<osmium::area::Assembler, false, true, false>::possibly_purge_removed_members collector.hpp 501 0x621c86
5 osmium::relations::Collectorosmium::area::MultipolygonCollector<osmium::area::Assembler, false, true, false>::HandlerPass2::find_and_add_object collector.hpp 177 0x61fc6c
6 osmium::relations::Collectorosmium::area::MultipolygonCollector<osmium::area::Assembler, false, true, false>::HandlerPass2::way collector.hpp 201 0x61df61
7 osmium::detail::apply_item_recurseosmium::relations::Collector<osmium::area::MultipolygonCollector<osmium::area::Assembler, false, true, false>::HandlerPass2, osmium::memory::Item> visitor.hpp 72 0x61b7d3
8 osmium::detail::apply_item_recurse<m1::osm::osmium::clipping::ClipCropped::Pass2, osmium::memory::Item, osmium::relations::Collector<osmium::area::MultipolygonCollectorosmium::area::Assembler, false, true, false>::HandlerPass2> visitor.hpp 209 0x616f33
9 osmium::apply<osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>, m1::osm::osmium::clipping::ClipCropped::Pass2, osmium::relations::Collectorosmium::area::MultipolygonCollector<osmium::area::Assembler, false, true, false>::HandlerPass2> visitor.hpp 238 0x6108f2
10 osmium::apply<osmium::io::Reader, m1::osm::osmium::clipping::ClipCropped::Pass2, osmium::relations::Collector<osmium::area::MultipolygonCollectorosmium::area::Assembler, false, true, false>::HandlerPass2> visitor.hpp 245 0x60a732
....
File formats (XML, PBF, ...) are currently detected based on file name suffixes. Additionally or instead we could detect formats by looking at the first few bytes of the file.
From a recent coverity scan:
osmium/memory/item.hpp: 126 in osmium::memory::Item::Item(unsigned int, osmium::item_type)()
120 protected:
121
122 explicit Item(item_size_type size=0, item_type type=item_type()) noexcept :
123 m_size(size),
124 m_type(type),
125 m_removed(false) {
CID 1255394: Uninitialized scalar field (UNINIT_CTOR)
Non-static class member "m_padding" is not initialized in this constructor nor in any functions that it calls.
126 }
127
128 Item(const Item&) = delete;
129 Item(Item&&) = delete;
130
131 Item& operator=(const Item&) = delete;
Seeing that clang++
is working with master branch but not g++, on travis (https://travis-ci.org/osmcode/libosmium/jobs/18206306#L581).
g++-4.8 -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o osmium_mpdump osmium_mpdump.cpp -lexpat -pthread -lz -lprotobuf-lite -losmpbf -lz -lbz2
In file included from ../include/osmium/relations/collector.hpp:51:0,
from ../include/osmium/area/collector.hpp:44,
from osmium_mpdump.cpp:17:
../include/osmium/relations/detail/member_meta.hpp: In function ‘constexpr bool osmium::relations::operator<(const osmium::relations::MemberMeta&, const osmium::relations::MemberMeta&)’:
../include/osmium/relations/detail/member_meta.hpp:102:32: error: call to non-constexpr function ‘osmium::object_id_type osmium::relations::MemberMeta::member_id() const’
strftime
is accounting for many calls. This profiling output is from OS X when running the https://github.com/springmeyer/node-osmium tests:
I am on OS X 10.9.2, aka as Mavericks and I am using Macports
a) What I installed following the list of prerequisites (package @Version):
boost @1.55.0 # there are no other boost packages, no boost-program-options, no libboost-test
zlib @1.2.8
bzip2 @1.0.6
gdal # There is a Macport package of gdal, but as a serious GIS worker, I have gdal from trunk, of course ;-)
expat @2.1.0
geos @3.4.2
sparsehash @2.0.2
protobuf-cpp @2.5.0
Doxygen # who wants to read documentation ?
libboost-test # see boost
OSMPBF from github
b) Macports install into /opt/local, so I had to point to /opt/local/lib and /opt/local/include in various places
c) For some reason which I read months ago and now forgot: one has to add the suffix "-mt" in 2 places:
examples/Makefile: LIB_PRGOPT := -lboost_program_options-mt
test/run_tests.sh: PARAMS= ... -lboost_unit_test_framework-mt"
d) To make it really work I also have to remove a suffix: -lprotobuf instead of -lprotobuf-lite
examples/Makefile: LIB_PBF := -pthread -lz -lprotobuf -losmpbf
Outcome #1:
[libosmium]> make test
(cd test && ./run_tests.sh)
Checking t/area/test_node_ref_segment.cpp ...
c++ -I../include -I. -g -std=c++11 -stdlib=libc++ -Wall -Wextra -Wredundant-decls -Wdisabled-optimization -pedantic -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -o tests -L/Users/peifer/local/lib -L/opt/local/lib test_main.cpp test_utils.cpp t/area/test_node_ref_segment.cpp -DBOOST_TEST_DYN_LINK -lboost_unit_test_framework-mt
Running 5 test cases...
No errors detected
Then more tests and more of "No errors detected"
Outcome #2: some warnings which don't seem to harm
[examples]> CPLUS_INCLUDE_PATH=/opt/local/include make
clang++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o osmium_convert osmium_convert.cpp -L/opt/local/lib -L/Users/peifer/local/lib -stdlib=libc++ -lexpat -pthread -lz -lprotobuf -losmpbf -lz -lbz2
clang++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o osmium_debug osmium_debug.cpp -L/opt/local/lib -L/Users/peifer/local/lib -stdlib=libc++ -lexpat -pthread -lz -lprotobuf -losmpbf -lz -lbz2
clang++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o osmium_index osmium_index.cpp -L/opt/local/lib -L/Users/peifer/local/lib -stdlib=libc++ -lboost_program_options-mt
clang++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o osmium_mpdump osmium_mpdump.cpp -L/opt/local/lib -L/Users/peifer/local/lib -stdlib=libc++ -lexpat -pthread -lz -lprotobuf -losmpbf -lz -lbz2
In file included from osmium_mpdump.cpp:17:
In file included from ../include/osmium/area/collector.hpp:44:
../include/osmium/relations/collector.hpp:163:57: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned long')
[-Wsign-compare]
assert(member_meta.member_pos() < m_collector.get_relation(relation_meta).members().size());
~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(builtin_expect(!(e), 0) ? __assert_rtn(__func, FILE, LINE, #e) : (void)0)
^
../include/osmium/relations/collector.hpp:198:31: note: in instantiation of member function
'osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2::find_and_add_object'
requested here
if (! find_and_add_object(node)) {
^
../include/osmium/visitor.hpp:111:33: note: in instantiation of member function
'osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2::node' requested here
visitor.node(static_cast<MaybeConst<TItem, osmium::Node>&>(item));
^
../include/osmium/visitor.hpp:172:17: note: in instantiation of function template specialization
'osmium::visitor::detail::switch_on_typeosmium::relations::Collector<osmium::area::Collector<osmium::area::Assembler, false, true,
false>::HandlerPass2, osmium::memory::Item>' requested here
switch_on_type(visitor, item, std::is_base_of<osmium::handler::Handler, TVisitor>());
^
../include/osmium/visitor.hpp:178:17: note: in instantiation of function template specialization
'osmium::visitor::detail::apply_item_recurseosmium::relations::Collector<osmium::area::Collector<osmium::area::Assembler, false, true,
false>::HandlerPass2, osmium::memory::Item>' requested here
apply_item_recurse(item, more...);
^
../include/osmium/visitor.hpp:261:38: note: in instantiation of function template specialization
'osmium::visitor::detail::apply_item_recurse<osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long long,
osmium::Location>, osmium::index::map::Dummy<unsigned long long, osmium::Location> >, osmium::memory::Item,
osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2>' requested here
osmium::visitor::detail::apply_item_recurse(_it, visitors...);
^
../include/osmium/visitor.hpp:268:9: note: in instantiation of function template specialization
'osmium::apply<osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>,
osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long long, osmium::Location>,
osmium::index::map::Dummy<unsigned long long, osmium::Location> >,
osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2>' requested here
apply(osmium::io::InputIterator {source},
^
osmium_mpdump.cpp:70:13: note: in instantiation of function template specialization 'osmium::apply<osmium::io::Reader,
osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long long, osmium::Location>,
osmium::index::map::Dummy<unsigned long long, osmium::Location> >,
osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2>' requested here
osmium::apply(reader2, location_handler, collector.handler());
^
1 warning generated.
clang++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o osmium_read osmium_read.cpp -L/opt/local/lib -L/Users/peifer/local/lib -stdlib=libc++ -lexpat -pthread -lz -lprotobuf -losmpbf -lz -lbz2
clang++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o osmium_serdump osmium_serdump.cpp -L/opt/local/lib -L/Users/peifer/local/lib -stdlib=libc++ -lexpat -pthread -lz -lprotobuf -losmpbf -lz -lbz2
clang++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -I/Users/peifer/local/include -o osmium_toogr osmium_toogr.cpp -L/opt/local/lib -L/Users/peifer/local/lib -stdlib=libc++ -lexpat -pthread -lz -lprotobuf -losmpbf -lz -lbz2 -L/Users/peifer/local/lib -lgdal
clang++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -I/Users/peifer/local/include -o osmium_toogr2 osmium_toogr2.cpp -L/opt/local/lib -L/Users/peifer/local/lib -stdlib=libc++ -lexpat -pthread -lz -lprotobuf -losmpbf -lz -lbz2 -L/Users/peifer/local/lib -lgdal
In file included from osmium_toogr2.cpp:31:
In file included from ../include/osmium/area/collector.hpp:44:
../include/osmium/relations/collector.hpp:163:57: warning: comparison of integers of different signs: 'int' and 'size_t' (aka 'unsigned long')
[-Wsign-compare]
assert(member_meta.member_pos() < m_collector.get_relation(relation_meta).members().size());
~~~~~~~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/assert.h:93:25: note: expanded from macro 'assert'
(builtin_expect(!(e), 0) ? __assert_rtn(__func, FILE, LINE, #e) : (void)0)
^
../include/osmium/relations/collector.hpp:198:31: note: in instantiation of member function
'osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2::find_and_add_object'
requested here
if (! find_and_add_object(node)) {
^
../include/osmium/visitor.hpp:111:33: note: in instantiation of member function
'osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2::node' requested here
visitor.node(static_cast<MaybeConst<TItem, osmium::Node>&>(item));
^
../include/osmium/visitor.hpp:172:17: note: in instantiation of function template specialization
'osmium::visitor::detail::switch_on_typeosmium::relations::Collector<osmium::area::Collector<osmium::area::Assembler, false, true,
false>::HandlerPass2, osmium::memory::Item>' requested here
switch_on_type(visitor, item, std::is_base_of<osmium::handler::Handler, TVisitor>());
^
../include/osmium/visitor.hpp:178:17: note: in instantiation of function template specialization
'osmium::visitor::detail::apply_item_recurseosmium::relations::Collector<osmium::area::Collector<osmium::area::Assembler, false, true,
false>::HandlerPass2, osmium::memory::Item>' requested here
apply_item_recurse(item, more...);
^
../include/osmium/visitor.hpp:178:17: note: in instantiation of function template specialization
'osmium::visitor::detail::apply_item_recurse<MyOGRHandler, osmium::memory::Item,
osmium::relations::Collector<osmium::area::Collectorosmium::area::Assembler, false, true, false>::HandlerPass2>' requested here
apply_item_recurse(item, more...);
^
../include/osmium/visitor.hpp:261:38: note: in instantiation of function template specialization
'osmium::visitor::detail::apply_item_recurse<osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long long,
osmium::Location>, osmium::index::map::Dummy<unsigned long long, osmium::Location> >, osmium::memory::Item, MyOGRHandler,
osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2>' requested here
osmium::visitor::detail::apply_item_recurse(_it, visitors...);
^
../include/osmium/visitor.hpp:268:9: note: in instantiation of function template specialization
'osmium::apply<osmium::io::InputIterator<osmium::io::Reader, osmium::memory::Item>,
osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long long, osmium::Location>,
osmium::index::map::Dummy<unsigned long long, osmium::Location> >, MyOGRHandler,
osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2>' requested here
apply(osmium::io::InputIterator {source},
^
osmium_toogr2.cpp:315:13: note: in instantiation of function template specialization 'osmium::apply<osmium::io::Reader,
osmium::handler::NodeLocationsForWays<osmium::index::map::SparseTable<unsigned long long, osmium::Location>,
osmium::index::map::Dummy<unsigned long long, osmium::Location> >, MyOGRHandler,
osmium::relations::Collectorosmium::area::Collector<osmium::area::Assembler, false, true, false>::HandlerPass2>' requested here
osmium::apply(reader2, location_handler, ogr_handler, collector.handler());
^
1 warning generated.
This comparison uses a ruby program, a shell script and sqlite. We'll need some other way of doing this.
Several errors occur while compiling tests:
C:\Program Files (x86)\Microsoft Visual C++ Compiler Nov 2013 CTP\include\utility(53): error C2678: binary '=' : no operator found which takes a left-hand operand of type 'const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>' (or there is no acceptable conversion) [D:\build_d\libosmium7\build-cmake\test\test_geojson.vcxproj]
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(948): could be 'std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>> &std::vector<_Ty,std::allocator<_Ty>>::operator =(const std::vector<_Ty,std::allocator<_Ty>> &)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(926): or 'std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>> &std::vector<_Ty,std::allocator<_Ty>>::operator =(std::initializer_list<std::pair<osmium::object_id_type,osmium::Location>>)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(822): or 'std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>> &std::vector<_Ty,std::allocator<_Ty>>::operator =(std::vector<_Ty,std::allocator<_Ty>> &&)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
while trying to match the argument list '(const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>, const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual C++ Compiler Nov 2013 CTP\include\utility(61) : see reference to function template instantiation 'void std::swap<_Ty>(_Ty &,_Ty &)' being compiled
with
[
_Ty=const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<std::pair<osmium::object_id_type,osmium::Location>>>
]
C:\Program Files (x86)\Microsoft Visual C++ Compiler Nov 2013 CTP\include\utility(189) : see reference to function template instantiation 'void std::_Swap_adl<_Ty2>(_Ty &,_Ty &)' being compiled
with
[
_Ty2=const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<std::pair<osmium::object_id_type,osmium::Location>>>
, _Ty=const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<std::pair<osmium::object_id_type,osmium::Location>>>
]
C:\Program Files (x86)\Microsoft Visual C++ Compiler Nov 2013 CTP\include\utility(185) : while compiling class template member function 'void std::pair<bool,const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>>::swap(std::pair<bool,const std::vector<_Ty,std::allocator<_Ty>>> &)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual C++ Compiler Nov 2013 CTP\include\utility(211) : see reference to function template instantiation 'void std::pair<bool,const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>>::swap(std::pair<bool,const std::vector<_Ty,std::allocator<_Ty>>> &)' being compiled
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
d:\build_d\libosmium7\test\t\geom\../basic/helper.hpp(73) : see reference to class template instantiation 'std::pair<bool,const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>>' being compiled
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual C++ Compiler Nov 2013 CTP\include\utility(54): error C2678: binary '=' : no operator found which takes a left-hand operand of type 'const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>' (or there is no acceptable conversion) [D:\build_d\libosmium7\build-cmake\test\test_geojson.vcxproj]
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(948): could be 'std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>> &std::vector<_Ty,std::allocator<_Ty>>::operator =(const std::vector<_Ty,std::allocator<_Ty>> &)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(926): or 'std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>> &std::vector<_Ty,std::allocator<_Ty>>::operator =(std::initializer_list<std::pair<osmium::object_id_type,osmium::Location>>)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(822): or 'std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>> &std::vector<_Ty,std::allocator<_Ty>>::operator =(std::vector<_Ty,std::allocator<_Ty>> &&)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
while trying to match the argument list '(const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>, const std::vector<std::pair<osmium::object_id_type,osmium::Location>,std::allocator<_Ty>>)'
with
[
_Ty=std::pair<osmium::object_id_type,osmium::Location>
]
test_geos.cpp
Some vector-pair-template-powered assignment that is melting my old-style Java-C brain.std::pair
I'm using Osmium for the Open Postcode Finder and it's working great. I've recently started porting over to the new libosmium as a prototype but I'm finding a strange bug.
When running over the Geofabrik West Midlands UK PBF extract (from a few weeks ago), I come to node 1925746696. If I do something like:
QString postcode = node.tags().get_value_by_key("addr:postcode");
if(postcode.startsWith("B13 8DD")) //This should be the whole value
{
const char* value = node.tags().get_value_by_key("addr:postcode");
std::cout << int(value[0]) << std::endl;
std::cout << int(value[1]) << std::endl;
std::cout << int(value[2]) << std::endl;
std::cout << int(value[3]) << std::endl;
std::cout << int(value[4]) << std::endl;
std::cout << int(value[5]) << std::endl;
std::cout << int(value[6]) << std::endl;
std::cout << int(value[7]) << std::endl;
std::cout << int(value[8]) << std::endl;
std::cout << int(value[9]) << std::endl;
std::cout << int(value[10]) << std::endl;
}
I would expect this to output:
66
49
51
32
56
68
68
0
<something>
<something>
<something>
where <something>
could be anything since it's past the end of the array (even segfault I guess). however, what is actually printed is:
66
49
51
32
56
68
68
-30
-128
-114
0
The first 7 entries match exactly "B13 8DD" but then, rather than being NULL terminated (as I would assume the char*
is), it has three negative values before it has a zero.
I've found a good number of other objects in the UK with similar behaviour and it's consistently the same objects. I suppose that it's possible that it's actually what's in the OSM database but I have no way to check other that with Osmium.
I'm on openSUSE 12.3 (x86_64) using GCC 4.8.1 with the latest master of libosmium.
Thanks for the great tool.
The classes osmium::detail::mmap_vector_base/anon/file currently only implement that part of the STL vector interface that we need. It would be nice to implement that interface fully and add tests for it.
The output of OSM data into files should use multithreading.
There is a helper class in utils/timestamp.hpp. But it might be better to have a real Timestamp class and use it in Object. Internally it would just contain a uint32_t, too. But this would allow us to add extra methods to it.
They are stored in a packed_task and need to be copyable for that.
To compile libosmium on mingw64 (on Windows or cross-platform), few changes are needed
(except building all the libraries with Mingw):
alex85k@8fb03c3
Tested on mingw-builds 4.9.1 (posix threads, seh exceptions), the full set of buld scripts (MSVC and Msys2+Mingw64) is here: https://github.com/alex85k/winbuilds
The command:
make doc
depends on doxygen. This is not listed on the readme. Would it be a good idea to list the dependency there?
libosmium➟ make install
install -m 755 -g root -o root -d /usr/include
install: root: Invalid argument
make: *** [install] Error 67
Seems that this is the issue that blocks installing currently on OSX
In file included from t/geom/test_geos_wkb.cpp:5:
../include/osmium/geom/wkb.hpp:44:11: fatal error: 'endian.h' file not found
# include <endian.h>
In file included from t/basic/test_node.cpp:1:
include/catch.hpp:831:34: error: comparison of integers of different signs: 'int' and 'unsigned long long' [-Werror,-Wsign-compare]
return opCast( lhs ) == opCast( rhs );
~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~
include/catch.hpp:876:39: note: in instantiation of member function 'Catch::Internal::Evaluator<int, unsigned long long, 0>::evaluate' requested here
return Evaluator<T1, T2, Op>::evaluate( lhs, rhs );
^
include/catch.hpp:1293:39: note: in instantiation of function template specialization 'Catch::Internal::compare<0, int, unsigned long long>' requested here
.setResultType( Internal::compare<Op>( m_lhs, rhs ) )
^
include/catch.hpp:1236:16: note: in instantiation of function template specialization 'Catch::ExpressionLhs<const int &>::captureExpression<0, unsigned long long>' requested here
return captureExpression<Internal::IsEqualTo>( rhs );
^
t/basic/test_node.cpp:47:15: note: in instantiation of function template specialization 'Catch::ExpressionLhs<const int &>::operator==<unsigned long long>' requested here
REQUIRE(0 == node.positive_id());
^
In file included from osmium_convert.cpp:14:
In file included from ../include/osmium/io/any_output.hpp:38:
In file included from ../include/osmium/io/opl_output.hpp:37:
../include/osmium/io/detail/opl_output_format.hpp:109:25: error: unused variable 'len' [-Werror,-Wunused-variable]
int len = snprintf(m_tmp_buffer, tmp_buffer_size, format, std::forward<TArgs>(args)...);
^
In file included from osmium_convert.cpp:14:
In file included from ../include/osmium/io/any_output.hpp:40:
In file included from ../include/osmium/io/xml_output.hpp:37:
../include/osmium/io/detail/xml_output_format.hpp:100:25: error: unused variable 'len' [-Werror,-Wunused-variable]
int len = snprintf(buffer, max_size, format, value);
^
2 errors generated.
make: *** [osmium_convert] Error 1
Running make doc
in a clean checkout fails with:
make: *** No rule to make target `doc/xml/classosmium_1_1*', needed by `doc/classes.txt'. Stop.
running it a second time completes successfully.
I think the problem is that the wildcard is expanded early, when the files don't yet exist. On the second run they do exist so the wildcard can match.
c++ -O3 -std=c++11 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I../include -stdlib=libc++ -Wall -Wextra -pedantic -Wredundant-decls -Wdisabled-optimization -Wctor-dtor-privacy -Wnon-virtual-dtor -Woverloaded-virtual -Wsign-promo -Wold-style-cast -o osmium_index osmium_index.cpp -stdlib=libc++ -Wl,-Bstatic -lboost_program_options -Wl,-Bdynamic
ld: unknown option: -Bstatic
clang: error: linker command failed with exit code 1 (use -v to see invocation)
with c++:
examples➟ c++ -v
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
Target: x86_64-apple-darwin13.0.0
Thread model: posix
The exception handling in Osmium is a bit of a mess. In some parts the usual exceptions (such as std::runtime_error) are used, some parts use special Osmium exceptions. This has to be straightened out. We have to look at all cases from the perspective of the library user and decide what the best kind of exception is in each case and what extra data is needed in the exception.
Everything that can't throw an exception should get a "noexcept".
The error is trivial:
..\..\test\t\index\test_typed_mmap.cpp(52): error C3861: 'mkstemp': identifier not found [D:\build_d\libosmium7\build-cmake\test\test_typed_mmap.vcxproj]
The solutions can be completely different (from avoiding mkstemp to adding its implementation from mingw into headers).
/home/dennis/Coding/Project-OSRM/ThirdParty/osmium/io/file_compression.hpp: In function ‘const char* osmium::io::as_string(osmium::io::file_compression)’:
/home/dennis/Coding/Project-OSRM/ThirdParty/osmium/io/file_compression.hpp:57:9: warning: control reaches end of non-void function [-Wreturn-type]
The tests in test/osm-testdata/testcases should use Catch instead of just having asserts.
After adding advanced FindOsmium to osmium-contrib, I noticed that it can also be included to main libosmium distribution. This will allow users to simply take FindOsmium.cmake from it sources and also make build scripts easier to maintain.
@joto Could you have a look? (and check on Linux)
alex85k@39f1d20
Unlike the make build, the cmake build doesn't build/run the tests in test/osm-testdata.
I have these errors and can not fix them myself:
D:\build_d\libosmium\include\osmium/index/map/vector.hpp(112): error C2976: 'std::vector' : too few template arguments [D:\build_d\libosmium\build-cmake\examples\osmium_serdump.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(650) : see declaration of 'std::vector'
..\..\examples\osmium_serdump.cpp(104) : see reference to class template instantiation 'osmium::index::map::VectorBasedSparseMap<osmium::unsigned_object_id_type,size_t,std::vector>' being compiled
D:\build_d\libosmium\include\osmium/index/map/vector.hpp(113): error C2955: 'std::vector' : use of class template requires template argument list [D:\build_d\libosmium\build-cmake\examples\osmium_serdump.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(650) : see declaration of 'std::vector'
D:\build_d\libosmium\include\osmium/index/map/vector.hpp(114): error C2955: 'std::vector' : use of class template requires template argument list [D:\build_d\libosmium\build-cmake\examples\osmium_serdump.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(650) : see declaration of 'std::vector'
D:\build_d\libosmium\include\osmium/index/map/vector.hpp(200): error C2282: 'osmium::index::map::VectorBasedSparseMap<osmium::unsigned_object_id_type,size_t,std::vector>::~VectorBasedSparseMap' cannot override 'osmium::index::map::Map<osmium::unsigned_object_id_type,size_t>::~Map' [D:\build_d\libosmium\build-cmake\examples\osmium_serdump.vcxproj]
D:\build_d\libosmium\include\osmium/index/map.hpp(104) : 'osmium::index::map::Map<osmium::unsigned_object_id_type,size_t>::~Map' is not deleted
D:\build_d\libosmium\include\osmium/index/multimap/vector.hpp(55): error C2976: 'std::vector' : too few template arguments [D:\build_d\libosmium\build-cmake\examples\osmium_serdump.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(650) : see declaration of 'std::vector'
..\..\examples\osmium_serdump.cpp(110) : see reference to class template instantiation 'osmium::index::multimap::VectorBasedSparseMultimap<osmium::unsigned_object_id_type,osmium::unsigned_object_id_type,std::vector>' being compiled
D:\build_d\libosmium\include\osmium/index/multimap/vector.hpp(56): error C2955: 'std::vector' : use of class template requires template argument list [D:\build_d\libosmium\build-cmake\examples\osmium_serdump.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(650) : see declaration of 'std::vector'
D:\build_d\libosmium\include\osmium/index/multimap/vector.hpp(57): error C2955: 'std::vector' : use of class template requires template argument list [D:\build_d\libosmium\build-cmake\examples\osmium_serdump.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\vector(650) : see declaration of 'std::vector'
My half-fixed branch (with cmake-building of examples by build.bat
):
https://github.com/alex85k/libosmium
Maybe this is related:
http://stackoverflow.com/questions/24017466/is-it-required-to-explicitly-list-default-parameters-when-using-template-templat
Windows is as always a treat. This time, the str(n)casecmp
are available under a different name only. The following diff fixes compilation.
diff --git a/third_party/osmium/util/config.hpp b/third_party/osmium/util/config.hpp
index c30f02e..6c86d68 100644
--- a/third_party/osmium/util/config.hpp
+++ b/third_party/osmium/util/config.hpp
@@ -36,6 +36,11 @@ DEALINGS IN THE SOFTWARE.
#include <cstdlib>
#include <cstring>
+#ifdef _MSC_VER
+#define strncasecmp _strnicmp
+#define strcasecmp _stricmp
+#endif
+
namespace osmium {
namespace config {
There is a bit of overlap between what OSMFile and Meta classes are doing. And it is unclear what the relationship between OSMFile and the input and output (or Reader and Writer classes, respectively) is. The Reader class can already be instantiated with just a filename.
The meta class also might need some more data in it. It is already a bit of a hodge-podge class, but we have to put the data from the PBF header block and XML file somewhere.
This all needs to be straightened out.
Hey @joto,
I am seeing this issue on Travis (link to log):
In file included from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/pbf_input.hpp:37:0,
from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_input.hpp:38,
from /home/travis/build/Project-OSRM/osrm-backend/Extractor/Extractor.cpp:46:
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/pbf_input_format.hpp: In member function ‘void osmium::io::detail::HeaderBlobParser::handle_blob(const string&)’:
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/pbf_input_format.hpp:480:42: error: ‘class OSMPBF::HeaderBlock’ has no member named ‘has_osmosis_replication_timestamp’
if (pbf_header_block.has_osmosis_replication_timestamp()) {
^
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/pbf_input_format.hpp:481:106: error: ‘class OSMPBF::HeaderBlock’ has no member named ‘osmosis_replication_timestamp’
m_header.set("osmosis_replication_timestamp", osmium::Timestamp(pbf_header_block.osmosis_replication_timestamp()).to_iso());
^
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/pbf_input_format.hpp:484:42: error: ‘class OSMPBF::HeaderBlock’ has no member named ‘has_osmosis_replication_sequence_number’
if (pbf_header_block.has_osmosis_replication_sequence_number()) {
^
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/pbf_input_format.hpp:485:109: error: ‘class OSMPBF::HeaderBlock’ has no member named ‘osmosis_replication_sequence_number’
m_header.set("osmosis_replication_sequence_number", std::to_string(pbf_header_block.osmosis_replication_sequence_number()));
^
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/pbf_input_format.hpp:488:42: error: ‘class OSMPBF::HeaderBlock’ has no member named ‘has_osmosis_replication_base_url’
if (pbf_header_block.has_osmosis_replication_base_url()) {
^
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/pbf_input_format.hpp:489:87: error: ‘class OSMPBF::HeaderBlock’ has no member named ‘osmosis_replication_base_url’
m_header.set("osmosis_replication_base_url", pbf_header_block.osmosis_replication_base_url());
^
In file included from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/bzip2_compression.hpp:45:0,
from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_compression.hpp:36,
from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_input.hpp:36,
from /home/travis/build/Project-OSRM/osrm-backend/Extractor/Extractor.cpp:46:
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/compression.hpp: At global scope:
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/compression.hpp:224:24: warning: ‘osmium::io::{anonymous}::registered_no_compression’ defined but not used [-Wunused-variable]
const bool registered_no_compression = osmium::io::CompressionFactory::instance().register_compression(osmium::io::file_compression::none,
^
In file included from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_compression.hpp:36:0,
from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_input.hpp:36,
from /home/travis/build/Project-OSRM/osrm-backend/Extractor/Extractor.cpp:46:
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/bzip2_compression.hpp:182:24: warning: ‘osmium::io::{anonymous}::registered_bzip2_compression’ defined but not used [-Wunused-variable]
const bool registered_bzip2_compression = osmium::io::CompressionFactory::instance().register_compression(osmium::io::file_compression::bzip2,
^
In file included from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_compression.hpp:37:0,
from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_input.hpp:36,
from /home/travis/build/Project-OSRM/osrm-backend/Extractor/Extractor.cpp:46:
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/gzip_compression.hpp:121:24: warning: ‘osmium::io::{anonymous}::registered_gzip_compression’ defined but not used [-Wunused-variable]
const bool registered_gzip_compression = osmium::io::CompressionFactory::instance().register_compression(osmium::io::file_compression::gzip,
^
In file included from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/pbf_input.hpp:37:0,
from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_input.hpp:38,
from /home/travis/build/Project-OSRM/osrm-backend/Extractor/Extractor.cpp:46:
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/pbf_input_format.hpp:664:28: warning: ‘osmium::io::detail::{anonymous}::registered_pbf_input’ defined but not used [-Wunused-variable]
const bool registered_pbf_input = osmium::io::detail::InputFormatFactory::instance().register_input_format(osmium::io::file_format::pbf,
^
In file included from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/xml_input.hpp:37:0,
from /home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/any_input.hpp:39,
from /home/travis/build/Project-OSRM/osrm-backend/Extractor/Extractor.cpp:46:
/home/travis/build/Project-OSRM/osrm-backend/ThirdParty/osmium/io/detail/xml_input_format.hpp:627:28: warning: ‘osmium::io::detail::{anonymous}::registered_xml_input’ defined but not used [-Wunused-variable]
const bool registered_xml_input = osmium::io::detail::InputFormatFactory::instance().register_input_format(osmium::io::file_format::xml,
^
Looks like the PBF definitions are too old?
I have added an appveyor config, but it is only a start and it doesn't work yet. See appveyor. I don't understand enough about windows builds and how to use the nuget stuff so that cmake will find the libraries. The config is just bits and pieces stolen from other configs so maybe I am doing it all wrong.
@alex85k: Maybe you can take a look.
Struct packing in MSVC is different than with GCC and clang. (Also see #30.)
@alex85k, @DennisOSRM: Can you try changing https://github.com/osmcode/libosmium/blob/master/include/osmium/memory/item.hpp#L104 ff. to:
item_type m_removed : 1;
item_type m_padding : 15;
and see whether the static_assert on https://github.com/osmcode/libosmium/blob/master/include/osmium/memory/item.hpp#L171 is okay then?
(Good explanation is here: http://randomascii.wordpress.com/2010/06/06/bit-field-packing-with-visual-c/)
I have these errors and can not fix them myself:
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(1014): error C2182: '_Get_value' : illegal use of type 'void' [D:\build_d\libosmium\build-cmake\examples\osmium_area_test.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(1466) : see reference to class template instantiation 'std::_State_manager<_Ty>' being compiled
with
[
_Ty=void
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(1747) : see reference to class template instantiation 'std::_Promise<_Ret>' being compiled
with
[
_Ret=void
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(1747) : while compiling class template member function 'std::packaged_task<void (void)>::packaged_task(std::packaged_task<void (void)> &&) throw()'
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\type_traits(1545) : see reference to function template instantiation 'std::packaged_task<void (void)>::packaged_task(std::packaged_task<void (void)> &&) throw()' being compiled
D:\build_d\libosmium\include\osmium/thread/checked_task.hpp(55) : see reference to class template instantiation 'std::packaged_task<void (void)>' being compiled
D:\build_d\libosmium\include\osmium/io/reader.hpp(176) : see reference to function template instantiation 'osmium::thread::CheckedTask<osmium::io::detail::ReadThread>::CheckedTask<osmium::thread::Queue<std::string>&,osmium::io::Decompressor*,std::atomic<bool>&>(osmium::thread::Queue<std::string> &,osmium::io::Decompressor *&&,std::atomic<bool> &)' being compiled
D:\build_d\libosmium\include\osmium/io/reader.hpp(176) : see reference to function template instantiation 'osmium::thread::CheckedTask<osmium::io::detail::ReadThread>::CheckedTask<osmium::thread::Queue<std::string>&,osmium::io::Decompressor*,std::atomic<bool>&>(osmium::thread::Queue<std::string> &,osmium::io::Decompressor *&&,std::atomic<bool> &)' being compiled
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(1021): error C2182: '_Val' : illegal use of type 'void' [D:\build_d\libosmium\build-cmake\examples\osmium_area_test.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(1029): error C2182: '_Val' : illegal use of type 'void' [D:\build_d\libosmium\build-cmake\examples\osmium_area_test.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(1747): error C2665: 'std::forward' : none of the 2 overloads could convert all the argument types [D:\build_d\libosmium\build-cmake\examples\osmium_area_test.vcxproj]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\type_traits(1509): could be '_Ty &&std::forward<std::_Promise<_Ret>>(std::_Promise<_Ret> &&) throw()'
with
[
_Ty=std::_Promise<void>
, _Ret=void
]
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\type_traits(1503): or '_Ty &&std::forward<std::_Promise<_Ret>>(std::_Promise<_Ret> &)'
with
[
_Ty=std::_Promise<void>
, _Ret=void
]
while trying to match the argument list '(std::_Promise<int>)'
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\include\future(1749): error C2512: 'std::_Promise<int>' : no appropriate default constructor available [D:\build_d\libosmium\build-cmake\examples\osmium_area_test.vcxproj]
osmium_convert.cpp
This is known to be an error or standard unclearness about packaged_task<void()>:
http://stackoverflow.com/questions/14744588/why-is-stdpackaged-taskvoid-not-valid
We had to add a workaround for this in OSRM:
Project-OSRM/osrm-backend@42d3ee9
Project-OSRM/osrm-backend#998 (comment)
in libosmium it seems much harder...
My half-fixed branch (with cmake-building of examples by build.bat
):
https://github.com/alex85k/libosmium
The new Osmium doesn't support multipolygons yet.
I had multiple errors like this when compiling the tests
..\..\test\t\tags\test_filter.cpp(37): error C2668: 'osmium::builder::build_tag_list' : ambiguous call to overloaded function [D:\build_d\libosmium7\build-cmake\test\test_filter.vcxproj]
D:\build_d\libosmium7\include\osmium/builder/builder_helper.hpp(89): could be 'const osmium::TagList &osmium::builder::build_tag_list(osmium::memory::Buffer &,std::function<void (osmium::builder::TagListBuilder &)>)'
D:\build_d\libosmium7\include\osmium/builder/builder_helper.hpp(77): or 'const osmium::TagList &osmium::builder::build_tag_list(osmium::memory::Buffer &,const std::map<const char *,const char *,std::less<_Kty>,std::allocator<std::pair<const _Kty,_Ty>>> &)'
with
[
_Kty=const char *
, _Ty=const char *
]
D:\build_d\libosmium7\include\osmium/builder/builder_helper.hpp(65): or 'const osmium::TagList &osmium::builder::build_tag_list(osmium::memory::Buffer &,std::initializer_list<std::pair<const char *,const char *>>)'
while trying to match the argument list '(osmium::memory::Buffer, initializer-list)'
it is in
const osmium::TagList& tag_list = osmium::builder::build_tag_list(buffer, {
{ "highway", "primary" }, // match
{ "name", "Main Street" }, // no match
{ "source", "GPS" } // no match
});
and I have no idea how the compiler can think that {...} is similar to std::function...
There is the osmium::handler::Handler class and its derived classes and the is osmium::osm::Visitor. They do slightly different things and have different interfaces, but essentially they are the same thing. This needs to be sorted out.
It seems MSVC does not support constexpr
properly. Instead of littering the code with ifdefs, does it make sense to define a 'OSMIUM_CONSTEXPR' once depending on compiler and use that? Are all cases of constexpr problematic, or only some of them?
@alex85k: Opinions?
There are lots of changes needed for Windows support. Smaller changes can just be added and tracked in this issue, for more complicated changes a new issue should be opened.
Changes in alex85k/libosmium/msvc:
Other issues concerning windows are labled 'windows'.
The classes in osmium::index::map/multimap are just simple wrapper classes for different data structures. Most of their functions are really simple and good candidates for inlining. This would work if thoses classes are simple template classes. But in some cases we also want to use them polymorphically, ie we want to decide at runtime which implementation to use depending on the kind of OSM data we work with (mainly whole planet vs. small extract) and how much memory we have available. For that they have to be virtual derived classes.
This becomes worse for the multimap classes, because their get_all() functions returns two iterators through which we can access the results. But the type of those iterators is different depending on the implementation of the multimap class. To make this work polymorphically, the iterators would have to be polymorphic, too. Of course all of this slows down the use case where you already know at compile time what class you want.
The currently situation is a strange mix. The multimap classes are virtual derived classes, but some of their functions (for instance get_all()) are not virtual and they have different return types in the different derived classes. This is not only ugly, it is error prone and hard to work with and will not work polymorphically.
One option would be to split the functionality into one class hierarchy that can not be used polymorphically (so you have to decide at compile time) and another class hierarchy on top of that that is usable polymorphically.
And maybe all these complications are not really problematic, because the cost of not inlining but having those virtual function calls isn't all that great compared to what else Osmium is doing. Maybe we should benchmark that... Unfortunately we have to implement both variants to do that...
To decide on the right design, we might have to look at more use cases.
The struct packing on Windows is different to what GCC/Clang do. Checking struct sizes will thus fail.
Happening here: https://github.com/osmcode/libosmium/blob/master/include/osmium/memory/item.hpp#L171
On Windows only fields of the same underlying type will be packed. While this is inconvenient, the standard says packing is implementation defined.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.