The rust-blosc-src from mulimoen

Use `-sys` crates for lz4/zstd/slib

The libraries linked might collide when linking if using the sys crates. Need to investigate and use the relevant wrappers instead, and propagate e.g. static builds

README and basic documentation

This crate is great, and seems to be the most easily usable Blosc crate available. However, it looks pretty sketchy, and that almost scared me away from trying it. A README and some basic documentation would go a long way to make it friendlier-looking.

Investigate c-blosc-2

@maiteko has started working on this in [0], should investigate how to help with this effort and maybe use the crate directly with COMPAT_MODE set

[0] https://github.com/maiteko/blosc2-src-rs

Cross compilation is broken

I'm trying to cross compile a larger application from linux (x86_64) to linux (aarch64) that depends on blosc-src. I get the following error:

The following warnings were emitted during compilation:

warning: [email protected]: aarch64-linux-gnu-gcc: error: unrecognized command-line option ‘-msse2’

error: failed to run custom build command for `blosc-src v0.3.1`

Caused by:
  process didn't exit successfully: `/tmp/test_blosc/target/debug/build/blosc-src-8213a8b070384c50/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=build.rs
  TARGET = Some("aarch64-unknown-linux-gnu")
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-unknown-linux-gnu")
  cargo:rerun-if-env-changed=CC_aarch64-unknown-linux-gnu
  CC_aarch64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CC_aarch64_unknown_linux_gnu
  CC_aarch64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=TARGET_CC
  TARGET_CC = None
  cargo:rerun-if-env-changed=CC
  CC = None
  RUSTC_LINKER = None
  cargo:rerun-if-env-changed=CROSS_COMPILE
  CROSS_COMPILE = None
  cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
  cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("neon")
  cargo:rerun-if-env-changed=CFLAGS_aarch64-unknown-linux-gnu
  CFLAGS_aarch64-unknown-linux-gnu = None
  cargo:rerun-if-env-changed=CFLAGS_aarch64_unknown_linux_gnu
  CFLAGS_aarch64_unknown_linux_gnu = None
  cargo:rerun-if-env-changed=TARGET_CFLAGS
  TARGET_CFLAGS = None
  cargo:rerun-if-env-changed=CFLAGS
  CFLAGS = None
  cargo:warning=aarch64-linux-gnu-gcc: error: unrecognized command-line option ‘-msse2’

From looking at the relevant source code the problem seems to be this part:

rust-blosc-src/build.rs

Lines 24 to 41 in 0bdc021

 if cfg!(target_feature = "sse2") { 

 build.define("SHUFFLE_SSE2_ENABLED", "1"); 

 if cfg!(target_env = "msvc") { 

 if cfg!(target_pointer_width = "32") { 

 build.flag("/arch:SSE2"); 

 } 

 } else { 

 build.flag("-msse2"); 

 } 

 } 

 if cfg!(target_feature = "avx2") { 

 build.define("SHUFFLE_AVX2_ENABLED", "1"); 

 if cfg!(target_env = "msvc") { 

 build.flag("/arch:AVX2"); 

 } else { 

 build.flag("-mavx2"); 

 } 

 }

It uses the cfg! macro which is evaluated at compile time and therefore refers to the host target_features in this case. Instead the script needs to read and evaluate the CARGO_CFG_TARGET_FEATURE set by cargo for build scripts.

(There is another issue with the build script with the cfg!(target_env = "msvc") a few lines above that can be resolved in a similar way).

mulimoen / rust-blosc-src Goto Github PK

rust-blosc-src's People

Contributors

Stargazers

Watchers

Forkers

rust-blosc-src's Issues

Use `-sys` crates for lz4/zstd/slib

README and basic documentation

Investigate c-blosc-2

Cross compilation is broken

Include SNAPPY codec

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Jobs

	if cfg!(target_feature = "sse2") {
	build.define("SHUFFLE_SSE2_ENABLED", "1");
	if cfg!(target_env = "msvc") {
	if cfg!(target_pointer_width = "32") {
	build.flag("/arch:SSE2");
	}
	} else {
	build.flag("-msse2");
	}
	}
	if cfg!(target_feature = "avx2") {
	build.define("SHUFFLE_AVX2_ENABLED", "1");
	if cfg!(target_env = "msvc") {
	build.flag("/arch:AVX2");
	} else {
	build.flag("-mavx2");
	}
	}