C language bindings for DataFusion
Home Page: https://datafusion-contrib.github.io/datafusion-c/latest/
License: Apache License 2.0
C language bindings for DataFusion.
See https://datafusion-contrib.github.io/datafusion-c/ for details.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Oh...
We already have the release document: https://datafusion-contrib.github.io/datafusion-c/latest/developer/release.html
dev/release/README.md is duplicated.
Can this be used to transform sql -> datafusion logical query plan in C++?
Like Apache Arrow C++.
We can reuse build system in apache/arrow: https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/package-task.rb
We can use INPUT in Doxyfile to specify built datafusion.h.
Here are features provided by implementing a normal GLib bindings library on the current C API:
datafusion.hIf Red Arrow is released but system package manager doesn't support it yet, we get the following error:
checking for arrow-glib version (>= 15.0.1)... no (15.0.0
installing 'apache-arrow-glib' native package... succeeded
checking for arrow-glib version (>= 15.0.1)... no (15.0.0
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers. Check the mkmf.log file for more details. You may
need configuration options.
We can use the latest "available" Red Arrow in CI.
We can use some caches for the followings to improve CI speed:
AFAIK there is no bindings for saving a data frame as a parquet file. (is that correct?)
pip3 install --upgrade meson
...
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
error: casting raw pointers to the same type and constness is unnecessary (`*mut capi::DFArrowArray` -> `*mut capi::DFArrowArray`)
Error: --> src/capi.rs:651:21
|
651 | (*c_abi_record_batch as *mut DFArrowArray) as *mut FFI_ArrowArray,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try: `*c_abi_record_batch`
|
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast
= note: `-D clippy::unnecessary-cast` implied by `-D clippy::all`
We need some scripts to release a new version.
#8 557.9 g++ -I. -I/usr/local/share/gems/gems/glib2-4.2.0/ext/glib2/../../lib -I/usr/local/share/gems/gems/glib2-4.2.0/ext/glib2 -I/usr/include -I/usr/include/ruby/backward -I/usr/include -I. -I/usr/local/share/gems/gems/extpp-0.1.1/include -I./.gem.20230911-6322-l56nvn -I/usr/lib64/ruby/vendor_ruby -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIC -O2 -g -pipe -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs\=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs\=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mtune\=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -std\=gnu\+\+17 -Wall -Wp,-D_FORTIFY_SOURCE\=2 -Wp,-D_GLIBCXX_ASSERTIONS -m64 -o arrow.o -c arrow.cpp
#8 557.9 g++ -I. -I/usr/local/share/gems/gems/glib2-4.2.0/ext/glib2/../../lib -I/usr/local/share/gems/gems/glib2-4.2.0/ext/glib2 -I/usr/include -I/usr/include/ruby/backward -I/usr/include -I. -I/usr/local/share/gems/gems/extpp-0.1.1/include -I./.gem.20230911-6322-l56nvn -I/usr/lib64/ruby/vendor_ruby -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include -fPIC -O2 -g -pipe -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs\=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs\=/usr/lib/rpm/redhat/redhat-annobin-cc1 -mtune\=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -std\=gnu\+\+17 -Wall -Wp,-D_FORTIFY_SOURCE\=2 -Wp,-D_GLIBCXX_ASSERTIONS -m64 -o converters.o -c converters.cpp
#8 557.9 In file included from /usr/include/arrow-glib/arrow-glib.hpp:29,
#8 557.9 from red-arrow.hpp:32,
#8 557.9 from arrow.cpp:20:
#8 557.9 /usr/include/arrow-glib/compute.hpp:23:10: fatal error: arrow/acero/exec_plan.h: No such file or directory
#8 557.9 #include <arrow/acero/exec_plan.h>
#8 557.9 ^~~~~~~~~~~~~~~~~~~~~~~~~
#8 557.9 In file included from /usr/include/arrow-glib/arrow-glib.hpp:29,
#8 557.9 from red-arrow.hpp:32,
#8 557.9 from converters.hpp:20,
#8 557.9 from converters.cpp:20:
#8 557.9 /usr/include/arrow-glib/compute.hpp:23:10: fatal error: arrow/acero/exec_plan.h: No such file or directory
#8 557.9 #include <arrow/acero/exec_plan.h>
#8 557.9 ^~~~~~~~~~~~~~~~~~~~~~~~~
It's for apache/arrow#33672.
Like Apache Arrow C++.
We can reuse build system in apache/arrow: https://github.com/apache/arrow/blob/master/dev/tasks/linux-packages/package-task.rb
Step 5/8 : RUN sed -i'' -e 's/main$/main contrib non-free/g' /etc/apt/sources.list
---> Running in ef794518e89e
sed: can't read /etc/apt/sources.list: No such file or directory
==> Pouring [email protected]_1.monterey.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/2to3
Target /usr/local/bin/2to3
already exists. You may want to remove it:
rm '/usr/local/bin/2to3'
To force the link and overwrite all conflicting files:
brew link --overwrite [email protected]
To list all files that would be deleted:
brew link --overwrite [email protected] --dry-run
Possible conflicting files are:
/usr/local/bin/2to3 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/2to3
/usr/local/bin/2to3-3.12 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/2to3-3.12
/usr/local/bin/idle3 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/idle3
/usr/local/bin/idle3.12 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/idle3.12
/usr/local/bin/pydoc3 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/pydoc3
/usr/local/bin/pydoc3.12 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/pydoc3.12
/usr/local/bin/python3 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/python3
/usr/local/bin/python3-config -> /Library/Frameworks/Python.framework/Versions/3.12/bin/python3-config
/usr/local/bin/python3.12 -> /Library/Frameworks/Python.framework/Versions/3.12/bin/python3.12
/usr/local/bin/python3.12-config -> /Library/Frameworks/Python.framework/Versions/3.12/bin/python3.12-config
/usr/local/share/man/man1/python3.1 -> /usr/local/Cellar/[email protected]/3.11.7_1/share/man/man1/python3.1
/usr/local/lib/pkgconfig/python3-embed.pc -> /usr/local/Cellar/[email protected]/3.11.7_1/lib/pkgconfig/python3-embed.pc
/usr/local/lib/pkgconfig/python3.pc -> /usr/local/Cellar/[email protected]/3.11.7_1/lib/pkgconfig/python3.pc
/usr/local/Frameworks/Python.framework/Headers -> /usr/local/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Headers
/usr/local/Frameworks/Python.framework/Python -> /usr/local/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Python
/usr/local/Frameworks/Python.framework/Resources -> /usr/local/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Resources
/usr/local/Frameworks/Python.framework/Versions/Current -> /usr/local/Cellar/[email protected]/3.11.7_1/Frameworks/Python.framework/Versions/Current
This is caused by conflicting system Python.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.