Comments (24)
What does this mean exactly? Is it aligning to 8 and overwriting things?
from vector.
It means the items aren't aligned in memory as they should be, which means peeking and pokeing may produce undefined behaviour.
Note that the peek and poke functions might require properly aligned addresses to function correctly. This is architecture dependent; thus, portable code should ensure that when peeking or poking values of some type a, the alignment constraint for a, as given by the function alignment is fulfilled.
from vector.
Could you share your storable instance code?
On Fri, Feb 6, 2015, 1:56 PM Daniel Waterworth [email protected]
wrote:
It means the items aren't aligned in memory as they should be, which means
peeking and pokeing may produce undefined behaviour.—
Reply to this email directly or view it on GitHub
#75 (comment).
from vector.
I guess it could happen if sizeof a
is not multiple of alignment
. Vector uses peek/pokeElemOff
which calculates offset in array as n * sizeof a
so alignment isn't be respected in this case. Storable's documentation doesn't say much about alinment so I think this is main problem
from vector.
Here you go:
data Word512 =
Word512
{-# UNPACK #-} !Word64
{-# UNPACK #-} !Word64
{-# UNPACK #-} !Word64
{-# UNPACK #-} !Word64
{-# UNPACK #-} !Word64
{-# UNPACK #-} !Word64
{-# UNPACK #-} !Word64
{-# UNPACK #-} !Word64
deriving Show
instance Storable Word512 where
sizeOf _ = 64
alignment _ = 64
peek ptr =
let
ptr' = castPtr ptr
in
Word512 <$>
peek ptr' <*>
peek (plusPtr ptr' 8) <*>
peek (plusPtr ptr' 16) <*>
peek (plusPtr ptr' 24) <*>
peek (plusPtr ptr' 32) <*>
peek (plusPtr ptr' 40) <*>
peek (plusPtr ptr' 48) <*>
peek (plusPtr ptr' 56)
poke ptr (Word512 a b c d e f g h) = do
let ptr' = castPtr ptr
poke ptr' a
poke (plusPtr ptr' 8) b
poke (plusPtr ptr' 16) c
poke (plusPtr ptr' 24) d
poke (plusPtr ptr' 32) e
poke (plusPtr ptr' 40) f
poke (plusPtr ptr' 48) g
poke (plusPtr ptr' 56) h
from vector.
https://ghc.haskell.org/trac/ghc/ticket/9806 is a (possibly) related ghc ticket.
is the allocation we do using storable vectors using the affected operations? @dolio @ekmett
from vector.
looks like storable vector doesnt do the right thing
{-# INLINE mallocVector #-}
mallocVector :: Storable a => Int -> IO (ForeignPtr a)
mallocVector =
#if __GLASGOW_HASKELL__ >= 605
doMalloc undefined
where
doMalloc :: Storable b => b -> Int -> IO (ForeignPtr b)
doMalloc dummy size = mallocPlainForeignPtrBytes (size * sizeOf dummy)
#else
mallocForeignPtrArray
#endif
from vector.
ok, i think the solution is that we need to use
-- | This function is similar to 'mallocForeignPtrBytes', except that the
-- size and alignment of the memory required is given explicitly as numbers of
-- bytes.
mallocForeignPtrAlignedBytes :: Int -> Int -> IO (ForeignPtr a)
mallocForeignPtrAlignedBytes size _align | size < 0 =
error "mallocForeignPtrAlignedBytes: size must be >= 0"
mallocForeignPtrAlignedBytes (I# size) (I# align) = do
r <- newIORef NoFinalizers
IO $ \s ->
case newAlignedPinnedByteArray# size align s of { (# s', mbarr# #) ->
(# s', ForeignPtr (byteArrayContents# (unsafeCoerce# mbarr#))
(MallocPtr mbarr# r) #)
}
which is defined in GHC.ForeignPtr
Next question: what range of ghc versions support this?
from vector.
@dolio whatever fix we do for this in master, we should /probably also do a patchlevel bug fix for 0.10
if it doesn't change the supported ghc version range, right?
from vector.
https://github.com/haskell/vector/blob/master/Data/Vector/Storable/Mutable.hs is where the allocation for storable vectors is defined
from vector.
@hvr has pointed me to https://ghc.haskell.org/trac/ghc/ticket/7067
so we can only directly use the provided operation in GHC >= 7.6,
and have to provide some sort of (less efficient) shim that provides the target alignment in < 7.6
from vector.
ok, we've had newAlignedPinnedByteArray# :: Int# -> Int# -> State# s -> (#State# s, MutableByteArray# s#)
since at least 7.0, possibly earlier!
from vector.
ok, 6.12 too. I dont think 6.10 had it. but >=6.12 should be a large enough supported range :)
from vector.
Wow @cartazio, eight consecutive comments. I've got to say, I'm impressed.
from vector.
@danielwaterworth combo-breaker! ;-)
from vector.
ok, i think given some of the other changes slated for vector 0.11, newAlignedPinnedByteArray# should be on the table for storable vectors in 0.11 release
from vector.
Hi, I was trying to fix this issue but couldn't find a backwards-compatible way to construct ForeignPtr from MutableByteArray# type returned by newAlignedPinnedByteArray#.
There's a way to obtain Addr# from the mutable array, but it cannot be straightforwardly converted into ForeignPtr. The reason is that only starting from 7.6.3 the Ghc.ForeignPtr module started exporting ForeignPtrContents datatype required to apply ForeignPtr constructor - see https://downloads.haskell.org/~ghc/7.6.3/docs/html/libraries/base-4.6.0.1/src/GHC-ForeignPtr.html (compare with ghc 7.4.2 where it's not exported https://downloads.haskell.org/~ghc/7.4.2/docs/html/libraries/base-4.5.1.0/src/GHC-ForeignPtr.html).
Another possibility is to convert Addr# to Ptr and then wrap Ptr into ForeignPtr. But the only way to obtain Ptr from Addr# seems to be casting via integers. I'm not sure whether this is a good idea.
I should mention in passing, maybe it's ok to use mallocPlainForeignPtrAlignedBytes on newer ghcs and keep old behavior on older ones?
from vector.
Ptr
is defined as is:
data Ptr = Ptr Addr#
so to convert an Addr# to Ptr you just need the Ptr constructor which is available in GHC.Ptr
.
from vector.
did we fix this with pr #96 ?
from vector.
I'm going to close this. I think it should be fixed by #96.
For reference, in case this comes back up, I'm somewhat skeptical of the example given. Word512
there is defined to be made up of several Word64
values. I don't think there should be a hardware reason why the type needs to be 64-byte aligned. It should be possible to be 8-byte aligned, because Word64#
is what would impose the hardware constraints (because all the peeks and pokes happen on 8-byte values). I think there would need to be a 512-bit hardware primitive to require 512-bit alignment for the usual reasons.
Of course, it wasn't even enforcing 8-byte alignment before, which was wrong.
from vector.
I don't think there should be a hardware reason why the type needs to be 64-byte aligned.
There is. It means that the whole thing fits on one cache line.
from vector.
Actually, I do have a use for it, which is the same one that @danielwaterworth mentions. I have code that needs stuff to align to cache line sizes to avoid cache-line contention a lot of places. This is the same behavior as the cache_aligned_allocator in intel's thread building blocks in C++.
Otherwise I have to double the size of everything yet again to avoid false sharing.
from vector.
This sort of alignment is only possible currently for storable vectors
correct?
On Thursday, August 11, 2016, Edward Kmett [email protected] wrote:
Actually, I do have a use for it, which is the same one that
@danielwaterworth https://github.com/danielwaterworth mentions. I have
code that needs stuff to align to cache line sizes to avoid cache-line
contention a lot of places. This is the same behavior as the
cache_aligned_allocator in intel's thread building blocks in C++.Otherwise I have to double the size of everything yet again to avoid false
sharing.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#75 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAQwkJ_OqApqgB9yAI3T-M3CWFGzKaRks5qewTZgaJpZM4DczBF
.
from vector.
Yes, as they are the only thing using, you know, Storable
. :)
from vector.
Related Issues (20)
- Adding `intersperse` to the API HOT 2
- When vector-0.13 will be released? HOT 3
- tests don't compile on ghc 9.2.2 HOT 2
- `Prim (Weak a)` instance? HOT 1
- Not fusing unless monadic HOT 2
- `fromList` is memory inefficient HOT 2
- Improve documentation and refactor `copy` + `move` in mutable APIs HOT 1
- Please concretize maintainership HOT 1
- Take over `vector-binary-instances` HOT 17
- High level documentation? HOT 1
- overlaps for Data.Vector.Mutable behaves oddly for empty vectors HOT 2
- Performance issues with `foldl'` and large tuples HOT 3
- Add a variant unfoldrExactN, returning the resulting generator HOT 3
- Vector tests take more than 4GB memory to build HOT 3
- `Data.Vector.Fusion.Bundle.Size.Size` fields are not strict HOT 1
- Why is `Data.Vector.Generic.New.New` not a newtype? HOT 2
- Question: Is there a reason why `Unbox a => Maybe (Unbox a)` doesn't exist? HOT 1
- Is `vector` susceptible to the same fusion problems as `text`? HOT 2
- Improve compilation time HOT 3
- Split generic tests into separate package HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vector.