Question: how to use with CMake? about qtusb

[petrmanek]

@fpoussin Yes, importing works at the moment -- feel free to try cloning the fork yourself. 🙂

Once I get the testing and static linking to work, I will file a PR and then we can test it more rigorously.

[petrmanek]

I should probably clarify that the Qt-recommended way does not work at the moment. Putting the following directive in CMakeLists.txt:

find_package(Qt6
  CONFIG
  REQUIRED
  COMPONENTS Usb
  )

....produces the following error:

-- Could NOT find Qt6Usb (missing: Qt6Usb_DIR)
CMake Error at CMakeLists.txt:50 (find_package):
  Found package configuration file:

    /usr/lib64/cmake/Qt6/Qt6Config.cmake

  but it set Qt6_FOUND to FALSE so package "Qt6" is considered to be NOT
  FOUND.  Reason given by package:

  Failed to find Qt component "Usb" config file at ""

This suggests that there may be some package metadata missing.

[fpoussin]

Hi,

CMake and/or Qt6 aren't supported at the moment.

[petrmanek]

@fpoussin Thanks for letting me know. I can confirm now that QtUsb works without any problems using Qt6's qmake. As far as CMake support is concerned, I will likely be patching that myself in the future. If you would consider a PR, I would be happy to make my work available.

[fpoussin]

Glad to know it works with Qt6!
A PR would be fantastic, thanks :)

[petrmanek]

@fpoussin Splendid. In the meantime, I have made some discoveries:

• If you compile QtUsb with Qt5 and QMake, both CMake & pkg-config metadata are generated in the lib/ directory as a part of the build process.
• If you compile QtUsb with Qt6 and QMake, only pkg-config metadata are generated. The create_cmake.prf mkspec has no effect.

I suspect this is because Qt6 uses CMake internally as its default build system for all its modules (and their corresponding CMake metadata), and in Qt6 the QMake command is only a compatibility wrapper for CMake, which appears to ignore the most of .prf files. For that reason, it seems that the only way to get the build system to generate CMake metadata in Qt6 is to fully transition from QMake to CMake.

I have taken the liberty of forking the repository and converting its build system from QMake to CMake according to the official guidelines. I also used the official pro2cmake.py helper script, which is recommended by the Qt developers, and patched some of the scope-related issues manually.

The current state is as follows:

• I can get QtUsb to build and link with Qt6 on Linux. All metadata (and other install files) appear to be generated correctly. Have not tried Windows and macOS yet, but they should work theoretically. There may be some problems with Android, however I am unable to verify that since I do not have an Android device myself.
• Test targets compile as well but I have to pass -DQT_BUILD_STANDALONE_TESTS=ON option to CMake, otherwise they are not generated. There is an additional issue -- with this option enabled, only test targets are generated and nothing else, so the tests build but then produce linking errors because there is no QtUsb module to link with. This leads me to believe that flipping the QT_BUILD_STANDALONE_TESTS flag may not be the proper way to enable QTest in Qt6. Unfortunately, I have no way to verify as Qt6's CMake macro documentation is virtually non-existent thus far.
• Static build is almost fully ported. Instead of using -static target, you just need now to pass -DQTUSB_STATIC=ON flag to CMake at the configuration stage. I also implemented the correct generation of qusbglobal.h in static/dynamic scopes but there are again some problems with the Qt6's extension macros. I will look into that in the near future.
• I verified that the generated CMake metadata can work for user projects. I generated and installed local package and tried to include it in a hello world executable using find_package(Qt6 REQUIRED COMPONENTS Usb). This worked nicely! The only catch was that I had to version-match the entire project's version with my Qt installation. Otherwise this line in the generated metadata file points to a non-existing file (that is usually provided by the Qt6 installation). At the moment, I am aware of no workaround.

[fpoussin]

Great!
Can the project still be imported into another one? (what the .pri did)
This is one of the most importants aspects.

[fpoussin]

Hi @petrmanek,
Have you made any progress?

[petrmanek]

@fpoussin I have made some progress, yes.

Here's what's new:

• The project now builds without errors in all configurations on Linux x86_64, Intel-based macOS and Windows x86_64. No idea about Android, though, as I am unable to test it.
• I had to enable linking with on-board libusb and hidapi on macOS, as it turns out that they are not available on vanilla macOS installations (albeit they can be downloaded with 3rd party package managers like Homebrew/Macports).
• Both static and dynamic builds now work properly, and are toggled by CMake's standard BUILD_SHARED_LIBS option.
• I corrected some errors & warnings arising from using slots instead of the Q_SLOTS macro in the code.
• I had some troubles incorporating QtUsb into CMake projects via ExternalProject. Seems like Qt's internal macros do not like Debug targets and tend to reconfigure the entire parent project's cache, which is incredibly inconvenient. To avoid it, I added an option to build QtUsb as a Qt library rather than a module. When enabled by setting QTUSB_MODULE=OFF, this produces a target called ExtUsb instead of QtUsb (the name change is because of windeployqt, which confuses any DLL that begins with Qt with a Qt module, and crashes hard if it cannot be found in %QT_DIR%).

I would be interested in your feedback. The fork is publicly available for you to try. 🙂

[petrmanek]

@fpoussin Have you had a chance to review my work?

[petrmanek]

One off-topic improvement I just made while testing the library with my (Linux) laptop: I discovered & corrected a bug that disrupts control transfers. As far as I can tell, the bug has been in QtUsb long before I came, and affects all control transfers initiated through QUsbEndpoint.

The main symptom is an infinite loop of the background thread after initiating a control transfer (note that it may only be visible if you increase QtUsb's verbosity to debug level). The first transfer technically succeeds, but the infinite loop between libusb and a callback in QtUsb unnecessarily eats memory, CPU time, and effectively prevents any further control transfers from taking place.

The bug appears due to mismatch between the length and actual_length fields of libusb_transfer. While usually these fields should match values for most completed transfers, control transfers are special snowflakes. Their data is divided into two parts:

1. A fixed setup stage of size LIBUSB_CONTROL_SETUP_SIZE (8 bytes),
2. An optional data stage of arbitrary size.

Since the setup stage is always present, libusb does not bother to include its size in actual_length. On the other hand, the length field is always set to include the setup stage size when a control transfer is initiated. This implies that even successful and fully completed control transfers will appear as if they were missing 8 bytes of data, which will lead the callback to spin indefinitely back & forth between libusb and QtUsb. I corrected the problem by simply adding an exception for control transfers that takes these extra 8 bytes into account. Having tested the library with a USB serial modem of CDC ACM class, I can confirm that control transfers are now fully operational.

[fpoussin]

Nice catch, thanks!

I've had a look, everything looks good so far :)

[petrmanek]

One more random catch. During testing with macOS I discovered that a call to QUsb::devices leads to non-deterministic crashes with the following backtraces:

* thread #21, name = 'QThread', stop reason = signal SIGABRT
  * frame #0: 0x00007fff20340792 libsystem_kernel.dylib`__abort_with_payload + 10
    frame #1: 0x00007fff203421d9 libsystem_kernel.dylib`abort_with_payload_wrapper_internal + 80
    frame #2: 0x00007fff2034220b libsystem_kernel.dylib`abort_with_payload + 9
    frame #3: 0x00007fff202a571b libsystem_c.dylib`_os_crash_fmt.cold.1 + 55
    frame #4: 0x00007fff20238105 libsystem_c.dylib`_os_crash_fmt + 154
    frame #5: 0x00007fff22b65cdd IOKit`IOHIDManagerScheduleWithRunLoop + 302
    frame #6: 0x0000000113762c56 libExtUsb.dylib`hid_enumerate [inlined] init_hid_manager at hid.c:382:3 [opt]
    frame #7: 0x0000000113762c05 libExtUsb.dylib`hid_enumerate [inlined] hid_init at hid.c:406 [opt]
    frame #8: 0x0000000113762bd1 libExtUsb.dylib`hid_enumerate(vendor_id=0, product_id=0) at hid.c:557 [opt]
    frame #9: 0x00000001137564f3 libExtUsb.dylib`QUsb::devices() at qusb.cpp:332:16 [opt]
    frame #10: 0x0000000113756027 libExtUsb.dylib`QUsb::QUsb(this=0x0000000103ed7070, parent=<unavailable>) at qusb.cpp:237:21 [opt]
    frame #11: 0x00000001132b7227 libusb_tools.dylib`UsbDiscoveryWorker::beginMonitoring(this=0x0000000103ed58d0) at discovery_worker.cpp:15:14
    frame #12: 0x00000001132b6e45 libusb_tools.dylib`UsbDiscoveryWorker::qt_static_metacall(_o=0x0000000103ed58d0, _c=InvokeMetaMethod, _id=2, _a=0x0000000103ed5f98) at moc_discovery_worker.cpp:100:21
[...]

Turns out that the issue was caused by multi-threading (in my program, two threads could sometimes independently call QUsb::devices roughly around the same time. I found that internally the function calls hidapi's hid_enumerate, which according to this comment is not thread-safe in macOS. For this reason, I added an application-wide mutex in the areas that use hid_enumerate and verified that it indeed prevents the crash.

Hi, I'm currently trying to use this library with CMake build system, does integration with this build system still in roadmap ?

Thank you for all your work !

[petrmanek]

@leger50 Feel free to check out my fork. The last time I used it the CMake integration worked just fine with Qt6. As to the other issues (re: multi-threading), I can report no progress.

[evan-swinney]

@petrmanek Did your CMake changes ever get merged into this repo? I tried adding this to a new Qt 6.5.0 build and it never could find the headers for the library by doing a simple add_subdirectory(QtUsb)

[petrmanek]

@evan-swinney Yes, they did.

I am currently using QtUsb under Qt 6.4.3 and Qt 6.5.0 without any issues. Here is how I do it: To remove the need for installing QtUsb as a full Qt module ahead of time (and thus polluting my Qt installation), I use the QTUSB_MODULE=OFF option, which turns QtUsb into a conventional Qt library called ExtUsb (because QtXXX prefix is reserved only for Qt components and windeployqt tends to complain about that). Then I simply link my program with it using an alias target.

Here is my minimal example:

# (omitting standard CMake code for finding Qt)

set(QTUSB_MODULE OFF CACHE BOOL "" FORCE)
add_subdirectory(qtusb) # this assumes that QtUsb is cloned into ./qtusb
add_library(Qt::Usb ALIAS ExtUsb)

add_executable(my_program main.cpp)
# (omitting linking with Qt and other libraries)
target_link_libraries(my_program PUBLIC Qt::Usb)

In main.cpp, I can then use QtUsb as desired, e.g.:

#include <QUsbDevice>
#include <QUsbEndpoint>

int main(int argc, char* argv[]) {
  auto device = new QUsbDevice();
  // this should compile
}

Hope this helps!