mkloczko / derive-storable

Deriving Storable instances using GHC.Generics

Introduction

The derive-storable package allows you to automatically generate Storable instances for your datatypes. It uses GHC.Generics, which allows the coders to derive certain instances automatically. To derive a (G)Storable instance, the data-type has to:

• have only one constructor There is now a sumtypes option for data-types with multiple constructors. See Sum types section for more.
• all fields of the constructor need to be GStorable.
• implement a Generic instance (derive (Generic))

Sum types

To enable support for sum types, add a -f sumtypes option to cabal new-build or cabal new-configure. The library discerns between sum and non-sum types. Non-sum types have the same memory layout as C structs, while sum types correspond to tagged unions:

struct datatype {
    unsigned char tag;
    union { 
        constructor1 a;
        constructor2 b;
        ...
    } val;
};

Note - while it is possible to have an instance for a self/mutually recursive data-type, using methods for the data-type will result in infinite loop. So there is no support for recursion in data-types.

Note on performance

There are some problems with performance of derived Storable instances. For now there exists a solution in form of GHC Core plugin - derive-storable-plugin.

Usage

Here's an example:

{-# LANGUAGE DeriveGeneric #-}

import Foreign.Storable
import Foreign.Storable.Generic
import Foreign.Ptr
import Foreign.Marshal.Alloc

import Generics.Deriving

data Position = Position {
   x :: Double, 
   y :: Double
} deriving (Show,Read, Generic)

instance GStorable Position

updatePosition :: Ptr Position -> Position -> IO ()
updatePosition ptr pos = poke ptr pos


main = do
    let val = Position 0.0 10.0
    ptr <- malloc :: IO (Ptr Position)      
    putStrLn "Created a ptr with value of"
    putStrLn =<< show <$> peek ptr
    updatePosition ptr val
    putStrLn "And now the value of ptr is:"   
    putStrLn =<< show <$> peek ptr

derive-storable's Issues

Preprocessing src/Foreign/Storable/Generic/Instances.hs fails for stack+OSX+ghc-8.0.1

derive-storable-0.1.0.3: configure
derive-storable-0.1.0.3: build
Progress: 1/2
--  While building package derive-storable-0.1.0.3 using:
      /Users/ian/.stack/setup-exe-cache/x86_64-osx/setup-Simple-Cabal-1.24.0.0-ghc-8.0.1 --builddir=.stack-work/dist/x86_64-osx/Cabal-1.24.0.0 build --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1
    Logs have been written to: .stack-work/logs/derive-storable-0.1.0.3.log

    Configuring derive-storable-0.1.0.3...
    Building derive-storable-0.1.0.3...
    Preprocessing library derive-storable-0.1.0.3...
    [1 of 4] Compiling Foreign.Storable.Generic.Tools ( src/Foreign/Storable/Generic/Tools.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.0.0/build/Foreign/Storable/Generic/Tools.o )
    [2 of 4] Compiling Foreign.Storable.Generic.Internal ( src/Foreign/Storable/Generic/Internal.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.0.0/build/Foreign/Storable/Generic/Internal.o )
    [3 of 4] Compiling Foreign.Storable.Generic.Instances ( src/Foreign/Storable/Generic/Instances.hs, .stack-work/dist/x86_64-osx/Cabal-1.24.0.0/build/Foreign/Storable/Generic/Instances.o )

    /private/var/folders/v6/w9s25grj1xg_9tj3dg07cflc0000gn/T/stack2438/derive-storable-0.1.0.3/src/Foreign/Storable/Generic/Instances.hs:55:1: error:
        Parse error: naked expression at top level
        Perhaps you intended to use TemplateHaskell

Unable to build on Windows.

derive-storable defines GStorable instances for POSIX types which don't exist on Windows.

[3 of 4] Compiling Foreign.Storable.Generic.Instances ( src\Foreign\Storable\Generic\Instances.hs, .stack-work\dist\e53504d9\build\Foreign\Storable\Generic\Instances.o )

    C:\Users\alex\AppData\Local\Temp\stack8620\derive-storable-0.1.1.0\src\Foreign\Storable\Generic\Instances.hs:90:20: error:
        Not in scope: type constructor or class `CRLim'
       |
    90 | MakeGStorable(CRLim)
       |                    ^^^^^

    C:\Users\alex\AppData\Local\Temp\stack8620\derive-storable-0.1.1.0\src\Foreign\Storable\Generic\Instances.hs:91:20: error:
        Not in scope: type constructor or class `CTcflag'
       |
    91 | MakeGStorable(CTcflag)
       |                    ^^^^^^^

    C:\Users\alex\AppData\Local\Temp\stack8620\derive-storable-0.1.1.0\src\Foreign\Storable\Generic\Instances.hs:92:20: error:
        Not in scope: type constructor or class `CSpeed'
       |
    92 | MakeGStorable(CSpeed)
       |                    ^^^^^^

    C:\Users\alex\AppData\Local\Temp\stack8620\derive-storable-0.1.1.0\src\Foreign\Storable\Generic\Instances.hs:93:20: error:
        Not in scope: type constructor or class `CCc'
       |
    93 | MakeGStorable(CCc)
       |                    ^^^

    C:\Users\alex\AppData\Local\Temp\stack8620\derive-storable-0.1.1.0\src\Foreign\Storable\Generic\Instances.hs:94:20: error:
        Not in scope: type constructor or class `CUid'
        Perhaps you meant `CPid' (imported from System.Posix.Types)
       |
    94 | MakeGStorable(CUid)
       |                    ^^^^

    C:\Users\alex\AppData\Local\Temp\stack8620\derive-storable-0.1.1.0\src\Foreign\Storable\Generic\Instances.hs:95:20: error:
        Not in scope: type constructor or class `CNlink'
       |
    95 | MakeGStorable(CNlink)
       |                    ^^^^^^

    C:\Users\alex\AppData\Local\Temp\stack8620\derive-storable-0.1.1.0\src\Foreign\Storable\Generic\Instances.hs:96:20: error:
        Not in scope: type constructor or class `CGid'
        Perhaps you meant `CPid' (imported from System.Posix.Types)
       |
    96 | MakeGStorable(CGid)
       |                    ^^^^

Alternative to the overlapping instance

Not sure if this is of interest:

newtype GStored a = GStored a
instance (Generic a, GStorable' (Rep a), KnownNat (NoFields (Rep a)))
  => GStorable (GStored a) where
  gsizeOf _ = internalSizeOf (undefined :: Rep a p)
  galignment _ = internalAlignment (undefined :: Rep a p)
  gpeekByteOff ptr offset = GStored . G.to <$> internalPeekByteOff ptr offset
  gpokeByteOff ptr offset (GStored x) =
    internalPokeByteOff ptr offset (G.from x)

One can use -XDerivingStrategies to derive using this newtype

For example:

data Sphere = Sphere
  { spherePos   :: V4 Float
  , sphereColor :: V4 Float
  }
  deriving stock Generic
  deriving Storable via GStored Sphere

Packed data

What would it take to add support for packed/unpadded data?

I assume GStorable generic instances may be cloned with padding removed. Would that still be supported by derive-storable-plugin?

0.3.0.1 is missing from Hackage

I had to inspect outputs one more time and found that getFilling doesn't appear in the most recent Hackage version.
Apparently I just used git versions when I needed that before 😄

Why does this depend on GStorable for member types rather than Storable

For example if I have the following:

data Bar = Bar
instance Storable Bar where

data Foo = Foo Bar
  deriving (Generic, GStorable)

this fails with No instance for (GStorable Bar), however I would expect this to use the Storable instance available.

Deriving Via

Right now we have 3 typeclasses: GStorable' which acts on Rep a, Storable which is already defined in base and there's GStorable which is basically Storable but with a default signature and a default implementation. I think it might be beneficial to use the DerivingVia to simplify the situation.

We could remove the typeclass GStorable altogether and rename GStorable' into GStorable and create a dummy newtype Generically which implements the Storable class using GStorable. This newtype can be used to derive Storable for any datatype that implements GStorable with the help of DerivingVia. (This is what generic-data is doing)

newtype Generically a = Generically { unGenerically :: a }

instance GStorable a => Storable (Generically a) where
  ...

data MyData = MyData 
  { ...
  }
  deriving Storable via Generically MyData

Q: Deriving sum types?

Are there any workarounds to derive Storable for sum types?