OpenSplice examples are linked against Connext about ros2

[esteve]

I don't really know what repository (or repositories) contains the bug, so I filed this ticket against ros2/ros2

[esteve]

ldd output for talker__rmw_connext_cpp:

esteve@osrf-esteve:~/Projects/ros2/ros2_ws$ ldd install/bin/talker__rmw_connext_cpp 
        linux-vdso.so.1 =>  (0x00007fffceda0000)
        librmw_connext_cpp.so => /home/esteve/Projects/ros2/ros2_ws/install/lib/librmw_connext_cpp.so (0x00007fbb281ea000)
        libsimple_msgs__dds_connext_cpp.so => /home/esteve/Projects/ros2/ros2_ws/install/lib/libsimple_msgs__dds_connext_cpp.so (0x00007fbb27d68000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fbb27b1d000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fbb27819000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fbb27602000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fbb2723d000)
        libnddscpp.so => /home/esteve/Projects/rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3/libnddscpp.so (0x00007fbb26efd000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fbb283f5000)
        libbuiltin_msgs__dds_connext_cpp.so => /home/esteve/Projects/ros2/ros2_ws/install/lib/libbuiltin_msgs__dds_connext_cpp.so (0x00007fbb26cc7000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbb269c1000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fbb267b8000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fbb265b4000)
        libnddsc.so => /home/esteve/Projects/rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3/libnddsc.so (0x00007fbb25f74000)
        libnddscore.so => /home/esteve/Projects/rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3/libnddscore.so (0x00007fbb25873000)

[esteve]

ldd output for talker__rmw_opensplice_cpp:

esteve@osrf-esteve:~/Projects/ros2/ros2_ws$ ldd install/bin/talker__rmw_opensplice_cpp 
        linux-vdso.so.1 =>  (0x00007fff35ff0000)
        librmw_opensplice_cpp.so => /home/esteve/Projects/ros2/ros2_ws/install/lib/librmw_opensplice_cpp.so (0x00007fa489e73000)
        libsimple_msgs__dds_connext_cpp.so => /home/esteve/Projects/ros2/ros2_ws/install/lib/libsimple_msgs__dds_connext_cpp.so (0x00007fa4899f1000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa4897a6000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa4894a2000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa48928b000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa488ec6000)
        libdcpssacpp.so => /usr/lib/libdcpssacpp.so (0x00007fa488ba8000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fa48a07c000)
        libbuiltin_msgs__dds_connext_cpp.so => /home/esteve/Projects/ros2/ros2_ws/install/lib/libbuiltin_msgs__dds_connext_cpp.so (0x00007fa488972000)
        libnddscpp.so => /home/esteve/Projects/rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3/libnddscpp.so (0x00007fa488632000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa48832c000)
        libdcpsgapi.so => /usr/lib/libdcpsgapi.so (0x00007fa4880c3000)
        libddsdatabase.so => /usr/lib/libddsdatabase.so (0x00007fa487e87000)
        libddsos.so => /usr/lib/libddsos.so (0x00007fa487c65000)
        libddskernel.so => /usr/lib/libddskernel.so (0x00007fa4879d1000)
        libddsuser.so => /usr/lib/libddsuser.so (0x00007fa487793000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa48758a000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa487386000)
        libnddsc.so => /home/esteve/Projects/rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3/libnddsc.so (0x00007fa486d46000)
        libnddscore.so => /home/esteve/Projects/rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3/libnddscore.so (0x00007fa486645000)
        libddsutil.so => /usr/lib/libddsutil.so (0x00007fa48643b000)
        libddsserialization.so => /usr/lib/libddsserialization.so (0x00007fa486211000)
        libddsconf.so => /usr/lib/libddsconf.so (0x00007fa48600d000)
        libddsconfparser.so => /usr/lib/libddsconfparser.so (0x00007fa485e00000)

[wjwwood]

Have you looked at the linker flags directly, al la VERBOSE=1? Are both libbuiltin_msgs__dds_connext_cpp and libbuiltin_msgs__dds_opensplice_cpp being passed to the linker for talker__rmw_opensplice_cpp/talker__rmw_connext_cpp?

[esteve]

I added -DCMAKE_VERBOSE_MAKEFILE=1 to ament and yeah, both libraries are passed to the linker when linking both executables:

/usr/local/bin/g++    -std=c++11 -Wall -g    
CMakeFiles/talker__rmw_connext_cpp.dir/src/ros1_like/talker.cpp.o  -o 
talker__rmw_connext_cpp  
-L/home/esteve/Projects/rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3 
-rdynamic
/home/esteve/Projects/ros2/ros2_ws/install/lib/librmw_connext_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libsimple_msgs__typesupport_introspection_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libsimple_msgs__dds_connext_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libsimple_msgs__dds_opensplice_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libbuiltin_msgs__typesupport_introspection_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libbuiltin_msgs__dds_connext_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libuserland_msgs__typesupport_introspection_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libuserland_msgs__dds_connext_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libuserland_msgs__dds_opensplice_cpp.so
-lnddscpp -lrticonnextmsgcpp -lcmxml 
-lcommonserv -ldcpsgapi -ldcpssac -lddsconfparser -lddsconf -lddsdatabase 
-lddsi2 -lddskernel -lddsosnet -lddsos -lddsserialization -lddsuser -lddsutil 
-ldurability -lspliced -ldcpsisocpp -ldcpssacpp -lpthread -ldl 
-Wl,-rpath,/home/esteve/Projects/ros2/ros2_ws/install/lib:/home/esteve/Projects/
rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3

/usr/local/bin/g++    -std=c++11 -Wall -g    
CMakeFiles/talker__rmw_opensplice_cpp.dir/src/ros1_like/talker.cpp.o  -o 
talker__rmw_opensplice_cpp  
-L/home/esteve/Projects/rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3 
-rdynamic 
/home/esteve/Projects/ros2/ros2_ws/install/lib/librmw_opensplice_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libsimple_msgs__typesupport_introspection_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libsimple_msgs__dds_connext_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libsimple_msgs__dds_opensplice_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libbuiltin_msgs__typesupport_introspection_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libbuiltin_msgs__dds_connext_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libuserland_msgs__typesupport_introspection_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libuserland_msgs__dds_connext_cpp.so 
/home/esteve/Projects/ros2/ros2_ws/install/lib/libuserland_msgs__dds_opensplice_cpp.so
-lnddscpp -lrticonnextmsgcpp -lcmxml 
-lcommonserv -ldcpsgapi -ldcpssac -lddsconfparser -lddsconf -lddsdatabase 
-lddsi2 -lddskernel -lddsosnet -lddsos -lddsserialization -lddsuser -lddsutil 
-ldurability -lspliced -ldcpsisocpp -ldcpssacpp -lpthread -ldl 
-Wl,-rpath,/home/esteve/Projects/ros2/ros2_ws/install/lib:/home/esteve/Projects/
rti/usbstick/ubuntu/ndds.5.1.0/lib/x64Linux3.xgcc4.6.3

[wjwwood]

Is that the core issue? or do we expect it to select the correct library somehow? @dirk-thomas do you remember how we intended this to work?

[esteve]

@wjwwood and I talked a bit about this. It seems that for some reason, the linker can't pick the right library (probably because there are no symbols that uniquely identify each), so maybe because connext comes first alphabetically, the linker picks it.

One solution would be to make find_package(simple_msgs) accept a COMPONENT that lets the user point to the RMW implementation they want to use. This is more explicit than letting the linker decide, but requires heavy changes to the current infrastructure.

[wjwwood]

Another option would be to provide infrastructure in each message package like what is in rmw_implementation, where by default <msg_package>_LIBRARIES contains the library for the default middleware implementation, but then there are CMake macros to get libraries specific to a message library and middleware implementation.

Both that option and the one @esteve described make the choice to not over link executables and not depend on the linker to pick the right one, but we would could go back to that strategy and fix the libraries so the linker can tell the difference again. But we both agreed that seemed potentially fragile and if we want to continue doing that we should find some precedence for it or some documentation that explicitly says it will work on different linkers, to make sure it's a reasonable thing to do.

[dirk-thomas]

Message packages always export the libraries for every type support.

Before the refactoring to C the linker was able to pick "the right" library. Now that doesn't work anymore - we assume it is since they don't contain any specific symbols anymore.

Selecting the "the right" libraries in CMake is challenging - especially since this libraries are often passed over several dependency levels. At the moment I am not convinced that this would be feasible / a good idea.

I would rather try to get back to the previously working approach by introducing a unique symbol again. At least until we find a solid approach without over linking.

[wjwwood]

It's hard for me to give a +1 to the changes attached to this. Basically the code changes look fine, but the real question is: does this work with on Linux and Windows after you apply the windows branches?

So we either have to do one of two things:

• Merge the windows knowing this issue still exists and rebase these pull requests, testing everything again and iterating on this pull request if needed.
• Merge these pull requests, rebasing all windows branches and then checking to see if it still works, iterating if needed in a new pull request or on the windows branches.

What should we do?

[esteve]

I already merged the windows branches onto windows-disambiguate-linking so we can test if these changes work.

[wjwwood]

I propose we review and merge these pull requests so I can rebase the windows branches on top, it's getting to be too difficult to wrangle three sets of branches all at once.

@dirk-thomas @tfoote can you guys review the related pull requests if you haven't already?

@esteve can you merge these pull requests when they're ready?

We've already shown it works on Linux, not it's just about fixing up windows again.

[esteve]

I'm going to merge these branches, I need them for the services branches too.

[wjwwood]

Ok, thanks, I'll rebase the windows stuff now.