Plans for Julia 1.7 about cxx.jl

[Gnimuc]

Just to share some updates on this.

I reimplemented some basic Cxx utilities on top of ClangCompiler.jl in the rebuild-clang-compiler branch. (still WIP...)

Basically, it reworked some of the utilities in initialization.jl on top of ClangCompiler.jl with which you can directly create a Clang compiler instance from a real .cpp file(it's more like Clang.jl's compiler initialization).

Helper functions like addHeaderDir and cxxinclude are not implemented because we can directly write the code in the .cpp file with the new initialization API. They can be easily implemented in ClangCompiler.jl if there is any need in the future. As a result, the major functionalities of the cxx macro can be provided by directly writing C++ code in this .cpp file.

As for type translation(i.e. mapping between a Julia Datatype and a Clang QualType), we still use the method in cxxtypes.jl as the type hierarchy on the Julia side. The functionalities related to Clang types are partially moved into ClangCompiler.jl.

The next step is implementing an LLVM level of type translation(stripmodifier&resolvemodifier in the codegen.jl), so Julia types and Clang types can match at the LLVM IR level.

To lower a Clang type to LLVM type, we need to use some methods from the CodeGenTypes class which is not in the public headers of Clang. This adds another wrinkle to the build process: as Julia maintains a patched version of LLVM/Clang source code, to maximize the compatibility, we need to use patched source code when building our wrapper library. But at this moment, the patched code can not be easily accessed, yet(see JuliaPackaging/Yggdrasil#692 (comment)). (UPDATE 7/10/2021: we now have everything we need in the upstream, but I don't have time working on this until the end of this year.)

Then, we need some wrapper functions for creating Clang function decls, manipulating arguments, parsing the function body, and emitting the LLVM IR. The emitted LLVM IR for the C++ function can be inlined(cloned) into a new LLVM::Function created via LLVM.jl, so we can call it by using Julia's llvmcall.

The last step is to resurrect the icxx string macro. The interpolation feature needs some source code editing utilities like automatically generating Julia variables(symbols) and then handle them on the Clang side via sema(the semantic analysis module).

As for the @cxx macro, it's a bit convoluted than the icxx macro and I don't know much about what needs to be done to resurrect it.

[oschulz]

As for the @cxx macro, it's a bit convoluted than the icxx macro

Just speaking for myself, I'd be happy having Cxx.jl back on current Julia versions even if it's without the @cxx macro for now.

[Gnimuc]

We should add support for Julia 1.6 firstly, as it's probably the next LTS version. I started working on it two weeks ago following the way described in #486 (comment). If you'd like to help and don't know where to start, you can find me in the #clang channel on Slack.

[00sapo]

Fine, I have just joined the #clang channel. I don't have so much time in this period, but I could help you as soon as I'll have a bit more time (mainly during we).

Also, I'm not expert about LLVM/clang, but I would like to learn!

[Gnimuc]

I just changed my mind due to some issues with 1.6(LLVM11), see JuliaPackaging/Yggdrasil#3315 (comment).

[PallHaraldsson]

With 1.7 now out, does it seem like it could be supported, or are there any known showstoppers to it getting supported? Would such support still likely break in any later version, or has there been some new development already to stop that?

Since Julia 1.0 is no longer LTS, it's not likely package authors are going to keep it as a lower bound, and may then go for 1.6, or 1.7. Unless [we make it] known it should be 1.3. Are there any other reasons, than this package to do that? And many cons to do that?

[Gnimuc]

The biggest problem is we lack active developers.

[00sapo]

Hello, do you have any news about the development status?

[alhirzel]

Hey @Gnimuc, I really appreciate your work on this package over the years. I wonder what steps could be taken by community members to increase participation? Do you have any recommendations or requests that we could help with?

[Gnimuc]

A good starting point is to read the comments in the source code to get a big picture. The next step is probably to get familiar with those LLVM/Clang APIs used in the project. If you're new to LLVM/Clang, it's highly recommended to compile the LLVM/Clang source code and browse the code using an IDE because you might need to reference the implementation of certain APIs to understand why some codes in the bootstrap.cpp are implemented in that way. A basic understanding of how Julia codegen works is also needed. Then, you can pick up some test cases and additional bug fixes.

[Gnimuc]

I forgot to mention how to compile the bootstrap library libcxxffi. You could take https://github.com/Gnimuc/libcxxffi as an example. It will be much easier for new contributors to get involved in the development of the package if #496 gets merged.

[Gnimuc]

Feel free to ask any questions in the Slack #clang channel.

[Gnimuc]

I'm closing this. See #496 for what feature has been resurrected and what is not.