klauspost / crc32 Goto Github PK
View Code? Open in Web Editor NEWCRC32 hash with x64 optimizations
License: BSD 3-Clause "New" or "Revised" License
CRC32 hash with x64 optimizations
License: BSD 3-Clause "New" or "Revised" License
I'm not using this directly, but via compress/gzip
, and I get this error when compiling for use on App Engine. It currently is using Go 1.4.2 and I believe it can't compile assembly. Would it be possible to add a build flag to support it?
Compile failed:
2015/12/14 16:39:45 go-app-builder: build timing: 77×6g (10.91s total), 1×6l (1.511s total)
2015/12/14 16:39:45 go-app-builder: failed running 6l: exit status 1
github.com/klauspost/crc32.init: github.com/klauspost/crc32.haveSSE42: not defined
github.com/klauspost/crc32.init: github.com/klauspost/crc32.haveCLMUL: not defined
github.com/klauspost/crc32.init: github.com/klauspost/crc32.haveSSE41: not defined
github.com/klauspost/crc32.init: undefined: github.com/klauspost/crc32.haveSSE42
github.com/klauspost/crc32.init: undefined: github.com/klauspost/crc32.haveCLMUL
github.com/klauspost/crc32.init: undefined: github.com/klauspost/crc32.haveSSE41
While building my software on CircleCI, I get spurious crashes like this:
=== RUN TestDownload
unexpected fault address 0xc820800000
fatal error: fault
[signal 0xb code=0x1 addr=0xc820800000 pc=0x7fded1]
goroutine 28 [running]:
runtime.throw(0x972920, 0x5)
/tmp/workdir/go/src/runtime/panic.go:527 +0x90 fp=0xc8200abad0 sp=0xc8200abab8
runtime.sigpanic()
/tmp/workdir/go/src/runtime/sigpanic_unix.go:27 +0x2ab fp=0xc8200abb20 sp=0xc8200abad0
dboard/vendor/github.com/klauspost/compress/flate.crc32sseAll(0xc8207fff9d, 0x63, 0x63, 0xc820328134, 0x60, 0x106, 0x1, 0x1, 0x0, 0x10000, ...)
/home/ubuntu/.go_project/src/dboard/vendor/github.com/klauspost/compress/flate/crc32_amd64.s:38 +0x31 fp=0xc8200abb28 sp=0xc8200abb20
dboard/vendor/github.com/klauspost/compress/flate.(*compressor).deflateLazy(0xc820328000)
/home/ubuntu/.go_project/src/dboard/vendor/github.com/klauspost/compress/flate/deflate.go:568 +0x11bc fp=0xc8200abcc0 sp=0xc8200abb28
dboard/vendor/github.com/klauspost/compress/flate.(*compressor).syncFlush(0xc820328000, 0x0, 0x0)
/home/ubuntu/.go_project/src/dboard/vendor/github.com/klauspost/compress/flate/deflate.go:726 +0xd9 fp=0xc8200abce0 sp=0xc8200abcc0
dboard/vendor/github.com/klauspost/compress/flate.(*Writer).Flush(0xc820328000, 0x0, 0x0)
/home/ubuntu/.go_project/src/dboard/vendor/github.com/klauspost/compress/flate/deflate.go:890 +0x42 fp=0xc8200abd10 sp=0xc8200abce0
dboard/vendor/github.com/klauspost/pgzip.compressBlock(0xc82045c000, 0x10000, 0x4c4b4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
/home/ubuntu/.go_project/src/dboard/vendor/github.com/klauspost/pgzip/gzip.go:406 +0x2bd fp=0xc8200abe50 sp=0xc8200abd10
runtime.goexit()
/tmp/workdir/go/src/runtime/asm_amd64.s:1696 +0x1 fp=0xc8200abe58 sp=0xc8200abe50
created by dboard/vendor/github.com/klauspost/pgzip.(*Writer).compressCurrent
/home/ubuntu/.go_project/src/dboard/vendor/github.com/klauspost/pgzip/gzip.go:244 +0x515
Hi Klaus – I was looking at this again and wondered what are IEEE tables. Also what are "IEEE optimizations"?
Thanks,
Joe
I was playing with this repo mainly as a way to teach myself code and including optional assembly versions of some routines. Very nice.
So I extended your benchmarks to test the Castagnoli version as well, and found that it wasn't any faster than the system hash, and sure enough the go 1.5 version includes the SSE 4.2 code to use the new crc32c opcode. The IEEE crc32 is still slow.
It might be a good idea to note that in the README
I installed gccgo (from gcc 5, on ubuntu). The crc32_amd64.s source file doesn't compile, since that's cgo specific asm language:
Godeps/_workspace/src/github.com/klauspost/crc32/crc32_amd64.s:9: Error: no such instruction: `text ·castagnoliSSE42(SB),NOSPLIT,$0'
....
There should be some kind of guard on the .s files (and/or the alternative pure go files) that detect when gccgo is in use and not compile the .s file(s).
It seems that including both this and the Go 1.6 crc32 (with the same code) creates a symbol conflict.
C:\Go\pkg\tool\windows_amd64\link.exe: running gcc failed: exit status 1
e:\temp\wintemp\go-link-561199559/go.o:(.text+0x242fd0): multiple definition of `rupoly'
e:\temp\wintemp\go-link-561199559/go.o:(.text+0x242fe0): first defined here
e:\temp\wintemp\go-link-561199559/go.o:(.text+0x2422b8): multiple definition of `r5'
e:\temp\wintemp\go-link-561199559/go.o:(.text+0x2422c0): first defined here
e:\temp\wintemp\go-link-561199559/go.o:(.text+0x242eb0): multiple definition of `r4r3'
e:\temp\wintemp\go-link-561199559/go.o:(.text+0x242ec0): first defined here
e:\temp\wintemp\go-link-561199559/go.o:(.text+0x242e90): multiple definition of `r2r1'
e:\temp\wintemp\go-link-561199559/go.o:(.text+0x242ea0): first defined here
collect2.exe: error: ld returned 1 exit status
We should probably rename our globals to avoid this collision.
Hello,
Can you please tag and version this project?
I am the Debian Maintainer for crc32 and versioning would help Debian keep up with development.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.