An extension to Data.Serialize with built-in version control
SafeCopy extends the parsing and serialization capabilities of
Data.Serialize to include nested
version control. Nested version control means that you can change the
definition and binary format of a type nested deep within other types without
problems.
ghc-8.4.1 comes with template-haskell-2.13.0.0 but safecopy-0.9.4 depends on template-haskell < 2.13. I installed successfully with cabal install --allow-newer=template-haskell but I haven't run the test-suite to be sure all is ok.
It would be very nice if safecopy supported interchangable backends - I'd like to use safecopy and it's versioning, but actually store the data in a human-readable format like json, xml, oder whatever.
Upper bound on template-haskell is problematic
Preprocessing test suite 'instances' for safecopy-0.9.1...
[1 of 1] Compiling Main ( test/instances.hs, dist/build/instances/instances-tmp/Main.o )
test/instances.hs:64:1: error:
Duplicate instance declarations:
instance Show UniversalTime -- Defined at test/instances.hs:64:1
instance [safe] Show UniversalTime
-- Defined in ‘time-1.6.0.1:Data.Time.LocalTime.LocalTime’
Blocked by:
(The template-haskell bounds are too tight.)
Without these you can't derive SafeCopy for types with no constructor, e.g. Data.Void.Void.
Blocked by:
Take the following program:
{-# LANGUAGE
TemplateHaskell,
TypeFamilies,
OverloadedStrings,
ScopedTypeVariables
#-}
import Data.SafeCopy
import Data.Acid
newtype X = X String
data Foo = Foo X X
deriveSafeCopy 0 'base ''X
deriveSafeCopy 0 'base ''Foo
makeAcidic ''Foo []
main = do
db :: AcidState Foo <- openLocalStateFrom "acid-hs/" (Foo (X "a") (X "b"))
putStrLn "loaded the database successfully"
createCheckpoint db
putStrLn "created checkpoint"
closeAcidState db
putStrLn "closed db"Introduce a type Y and try to go from Foo X X to Foo X Y:
{-# LANGUAGE
TemplateHaskell,
TypeFamilies,
OverloadedStrings,
ScopedTypeVariables
#-}
import Data.SafeCopy
import Data.Acid
newtype X = X String
newtype Y = Y String -- new
data Foo = Foo X Y -- was Foo X X
deriveSafeCopy 0 'base ''X
deriveSafeCopy 1 'extension ''Y
deriveSafeCopy 0 'base ''Foo
instance Migrate Y where -- migration from X to Y
type MigrateFrom Y = X
migrate (X a) = Y a
makeAcidic ''Foo []
main = do
db :: AcidState Foo <- openLocalStateFrom "acid-hs/" undefined
putStrLn "loaded the database successfully"
createCheckpoint db
putStrLn "created checkpoint"
closeAcidState db
putStrLn "closed db"Try to run the 1st version and then load the checkpoint in the 2nd version:
$ runghc acid1.hs
loaded the database successfully
created checkpoint
closed db
$ runghc acid2.hs
acid2.hs: Could not parse saved checkpoint due to the following error:
Failed reading: safecopy: Char: Cannot find getter associated with this version number:
Version {unVersion = 1627389952}
From: Main.Foo:
Main.X:
Ouch.
I think the reason it happens is that for Foo X X we generate the following safePut:
putCopy (Foo arg_a9dA arg_a9dB)
= contain
(do { safePut_X_a9dC <- getSafePut;
safePut_X_a9dC arg_a9dA;
safePut_X_a9dC arg_a9dB;
return () })It will write the version tag for X only once (since writing the tag is done in getSafePut). However, when reading Foo X Y we will try to read the version tag twice:
getCopy
= contain
(Data.Serialize.Get.label
"Main.Foo:"
(do { safeGet_X_a9fJ <- getSafeGet;
safeGet_Y_a9fK <- getSafeGet;
(((return Foo) <*> safeGet_X_a9fJ) <*> safeGet_Y_a9fK) }))Ouch.
(My actual usecase, by the way, involves migrating from something like Foo X X to Foo (X Int) (X Bool), where Int and Bool are phantom (i.e. X is defined as newtype X a = X Text).)
Minimal code to reproduce:
{-# LANGUAGE TemplateHaskell #-}
import Data.SafeCopy
newtype IntPair = IntPair (Int, Int)
deriveSafeCopy 0 'base ''IntPairIt fails to compile with:
Main.hs:7:1:
Illegal variable name: ‘safePut_(,)IntInt’
When splicing a TH declaration:
instance Data.SafeCopy.SafeCopy.SafeCopy Main.IntPair
where Data.SafeCopy.SafeCopy.putCopy (Main.IntPair arg_0) = Data.SafeCopy.SafeCopy.contain (do {safePut_(,)IntInt_1 <- Data.SafeCopy.SafeCopy.getSafePut;
safePut_(,)IntInt_1 arg_0;
GHC.Base.return ()})
Data.SafeCopy.SafeCopy.getCopy = Data.SafeCopy.SafeCopy.contain (Data.Serialize.Get.label "Main.IntPair:" (do {safeGet_(,)IntInt_2 <- Data.SafeCopy.SafeCopy.getSafeGet;
GHC.Base.return Main.IntPair GHC.Base.<*> safeGet_(,)IntInt_2}))
Data.SafeCopy.SafeCopy.version = 0
Data.SafeCopy.SafeCopy.kind = Data.SafeCopy.SafeCopy.base
Data.SafeCopy.SafeCopy.errorTypeName _ = "Main.IntPair"
It fails with a message like this:
test/instances.hs:102:35:
Not in scope: data constructor ‘ClassP’
Perhaps you meant one of these:
‘ClassI’ (imported from Language.Haskell.TH.Syntax),
‘ClassD’ (imported from Language.Haskell.TH.Syntax),
variable ‘classP’ (imported from Language.Haskell.TH)
The following snippet works fine with GHC 7.8.4 and safecopy-0.8.3
{-# LANGUAGE TemplateHaskell #-}
import Data.SafeCopy
newtype Log a = Exp { ln :: a } deriving (Eq,Ord)
deriveSafeCopy 1 'base ''Logbut with GHC 7.10 it fails to compile with
test.hs:7:1:
Expecting one more argument to ‘Log’
The first argument of ‘SafeCopy’ should have kind ‘*’, but ‘Log’ has kind ‘* -> *’
In the instance declaration for ‘SafeCopy Log’
The Proxy type in Data.Proxy looks like
data Proxy t = Proxy
It has a type parameter t, but no actual data of that type. However, the SafeCopy instance that deriveSafeCopy generates has the unnecessary constraint
instance SafeCopy t => SafeCopy (Proxy t) where...
I'm looking at this error message:
ErrorCall (Failed reading: safecopy: Ratio: Cannot find getter associated with this version number: Version {unVersion = 1}
It would be very helpful to know where Ratio is coming from. Keeping a list of the types that have been traversed would be helpful to get a sense of where these problems are occurring.
Hej,
I noticed this today when trying to build my blog with GHC 7.8.2
Minimal complete example:
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
import Data.SafeCopy
newtype Foo = Foo Integer
deriving (SafeCopy)Output:
~/s/acid-test> ghc Broken.hs
[1 of 1] Compiling Main ( Broken.hs, Broken.o )
Broken.hs:6:23:
Could not coerce from ‘Kind Integer’ to ‘Kind Foo’
because the first type argument of ‘Kind’ has role Nominal,
but the arguments ‘Integer’ and ‘Foo’ differ
arising from the coercion of the method ‘kind’ from type
‘Kind Integer’ to type ‘Kind Foo’
Possible fix:
use a standalone 'deriving instance' declaration,
so you can specify the instance context yourself
When deriving the instance for (SafeCopy Foo)
Using the TH call deriveSafeCopy works fine, but seems to generate instances incompatible with earlier instances derived automatically. Fails with an error: Could not parse saved checkpoint due to the following error: Failed reading: safecopy: Integer: Cannot find getter associated with this version number: Version {unVersion = 5131390}
Any ideas?
//V
Line 39 in af59f92
There are circumstances where more context is needed on the type parameter when generating a SafeCopy instance from T1 typ. For example, http://lpaste.net/135048 is a module where the IxSet types create a need for Data, Typeable, and Ord. I'm not yet sure what the correct solution is to this issue, but for the moment I've added these superclasses to our version of safecopy.
The current instance SafeCopy a => SafeCopy [a] has this implementation of getCopy:
getCopy = contain $
do n <- get
getSafeGet >>= replicateM n
This doesn't force evaluation of the list elements, which leads to thunks being left around in the deserialized state. Something like this (based on getListOf from cereal) is probably better:
getCopy = contain $ do
n <- get
g <- getSafeGet
go g [] n
where
go :: Get a -> [a] -> Int -> Get [a]
go _ as 0 = return (reverse as)
go g as i = do x <- g
x `seq` go g (x:as) (i - 1)
See https://matrix.hackage.haskell.org/#/package/generic-data
safecopy however says
base >=4.5 && <5,
which should be bumped to >=4.9
Also there are conditionals like
if !impl(ghc > 8.0)
Build-Depends: semigroups >= 0.18 && < 0.19
if(impl(ghc >= 7.2.1))
cpp-options: -DDEFAULT_SIGNATURES
if(impl(ghc >= 7.1))
cpp-options: -DSAFE_HASKELL
which could be cleaned up
Using DefaultSignatures, the errorTypeName method could easily be given the implementation:
default errorTypeName :: (Typeable a) => Proxy a -> String
errorTypeName prxy = show (typeRep prxy)Since all types are Typeable in recent GHC, this would be trivially satisfiable for all types.
My program just crashed at run-time with
ghc: Duplicate version tags: [2,2,1]
because I wrote
deriveSafeCopy 2 'extension ''Mytype_v2
deriveSafeCopy 2 'extension ''Mytype_v3when I meant deriveSafeCopy 3 'extension ''Mytype_v3.
Since I use deriveSafeCopy which is TemplateHaskell, it would be great if this mistake could be pointed out to me at compile time.
src/Data/SafeCopy/Instances.hs:379:31:
Ambiguous occurrence ‘Proxy’
It could refer to either ‘Data.SafeCopy.SafeCopy.Proxy’,
imported from ‘Data.SafeCopy.SafeCopy’ at src/Data/SafeCopy/Instances.hs:5:1-29
(and originally defined at src/Data/SafeCopy/SafeCopy.hs:391:1-20)
or ‘Data.Typeable.Proxy’,
imported from ‘Data.Typeable’ at src/Data/SafeCopy/Instances.hs:34:1-30
(and originally defined in ‘Data.Proxy’)If fixed it with:
import Data.Typeable hiding (Proxy)We probably need a conditional to support previous versions.
Hi,
I am upgrading SafeCopy to version 0.10.4.1 and discovered following
unpleased behavior. Mostly I concerned with stumbling error message.
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE DeriveAnyClass #-}
module Main where
import Data.SafeCopy
import Data.Serialize
import GHC.Generics
import Type.Reflection
import Prelude
newtype IntKey r = IntKey { intKey :: Int } deriving (Show, Eq, Generic, Serialize, Typeable)
instance Typeable r => SafeCopy (IntKey r)
The snipped above is just fine until I enable PolyKinds extension.
Error message makes impression like GHC is incapable to do the obvious thing!
• Could not deduce (Typeable k)
arising from the superclasses of an instance declaration
from the context: Typeable r
bound by the instance declaration
at Main.hs:31:10-43
• In the instance declaration for ‘SafeCopy (IntKey r)’
|
31 | instance Typeable r => SafeCopy (IntKey r)
test/instances.hs:102:35:
Not in scope: data constructor ‘ClassP’
Perhaps you meant one of these:
‘ClassI’ (imported from Language.Haskell.TH.Syntax),
‘ClassD’ (imported from Language.Haskell.TH.Syntax),
variable ‘classP’ (imported from Language.Haskell.TH)src/Data/SafeCopy/Derive.hs:222:14: error:
• Expecting one more argument to ‘TyVarBndr’
Expected a type, but ‘TyVarBndr’ has kind ‘* -> *’
• In the type signature: tyVarName :: TyVarBndr -> Name
|
222 | tyVarName :: TyVarBndr -> Name
| ^^^^^^^^^
I used following patch
diff --git a/src/Data/SafeCopy/Derive.hs b/src/Data/SafeCopy/Derive.hs
index 34e29fc..30494c3 100644
--- a/src/Data/SafeCopy/Derive.hs
+++ b/src/Data/SafeCopy/Derive.hs
@@ -219,9 +219,15 @@ forceTag :: DeriveType -> Bool
forceTag HappstackData = True
forceTag _ = False
+#if MIN_VERSION_template_haskell(2,17,0)
+tyVarName :: TyVarBndr s -> Name
+tyVarName (PlainTV n _) = n
+tyVarName (KindedTV n _ _) = n
+#else
tyVarName :: TyVarBndr -> Name
tyVarName (PlainTV n) = n
tyVarName (KindedTV n _) = n
+#endif
internalDeriveSafeCopy :: DeriveType -> Version a -> Name -> Name -> Q [Dec]
internalDeriveSafeCopy deriveType versionId kindName tyName = do
diff --git a/test/generic.hs b/test/generic.hs
index b3eb6fb..d777328 100644
--- a/test/generic.hs
+++ b/test/generic.hs
@@ -217,9 +217,9 @@ t2 = (T2 'b')
t3 = (T3 'c')
t4 = T4 100 200 300
-$(deriveSafeCopy 3 'base ''T1)
$(deriveSafeCopy 4 'base ''T2)
$(deriveSafeCopy 5 'base ''T3)
+$(deriveSafeCopy 3 'base ''T1)
$(deriveSafeCopy 6 'base ''T4)
data T1G = T1G Char T2G T3G deriving (Generic, Show)
I cannot compile the test suite on Linux with GHC 7.8.4. A complete build log is available at http://hydra.cryp.to/build/357301/nixlog/1/raw.
Hi.
I am studying safecopy library and trying to execute hello world.
I followed the example from haddock and cannot get deserialization working for a new type.
Something is missing in example, which is not obvious.
GHCi, version 8.8.3
{-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE TypeFamilies #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE DeriveAnyClass #-}
{-# LANGUAGE ScopedTypeVariables #-}
module Main where
import GHC.Generics
import Data.SafeCopy
import Data.Serialize
data BaseT = BaseT Int deriving (Generic, Serialize, Show, Eq)
instance SafeCopy BaseT where
version = 0
kind = base
putCopy (BaseT s) = contain $ safePut s
getCopy = contain $ BaseT <$> safeGet
data NextT = NextT Int Int deriving (Generic, Serialize, Show, Eq)
instance SafeCopy NextT where
version = 1
kind = extension
putCopy (NextT a b) = contain $ do safePut a ; safePut b
getCopy = contain $ NextT <$> safeGet <*> safeGet
instance Migrate NextT where
type MigrateFrom NextT = BaseT
migrate (BaseT s) = NextT s s
*Main> (runGet safeGet (encode $ BaseT 888)) :: Either String NextT
Left "Failed reading: safecopy: Int: Cannot find getter associated with this version number: Version {unVersion = 888}\nEmpty call stack\n"
Self explanatory.
I don't get from the documentation how to use Data.Serialize.encode/decode. I think the introduction example should include the few lines necessary to work with ByteStrings, not only the SafeCopy typeclass.
--allow-failure argument from haskell-ci invocation)src/Data/SafeCopy/Derive.hs:246:61: error:
• Couldn't match expected type ‘[Type]’ with actual type ‘Type’
• In the second argument of ‘map’, namely ‘ty’
In the third argument of ‘foldl’, namely ‘(map return ty)’
In the first argument of ‘worker'’, namely
‘(foldl appT (conT tyName) (map return ty))’
|
246 | worker' (foldl appT (conT tyName) (map return ty)) context [] (zip [0..] cons)
| ^^
src/Data/SafeCopy/Derive.hs:249:61: error:
• Couldn't match expected type ‘[Type]’ with actual type ‘Type’
• In the second argument of ‘map’, namely ‘ty’
In the third argument of ‘foldl’, namely ‘(map return ty)’
In the first argument of ‘worker'’, namely
‘(foldl appT (conT tyName) (map return ty))’
|
249 | worker' (foldl appT (conT tyName) (map return ty)) context [] [(0, con)]
| ^^
src/Data/SafeCopy/Derive.hs:280:21: error:
• Couldn't match expected type ‘Type’ with actual type ‘[Type]’
• In the second argument of ‘(==)’, namely ‘tyIndex’
In the expression: ty == tyIndex
In a stmt of a pattern guard for
a case alternative:
ty == tyIndex
|
280 | | ty == tyIndex ->
| ^^^^^^^
src/Data/SafeCopy/Derive.hs:281:61: error:
• Couldn't match expected type ‘[Type]’ with actual type ‘Type’
• In the second argument of ‘map’, namely ‘ty’
In the third argument of ‘foldl’, namely ‘(map return ty)’
In the first argument of ‘worker'’, namely
‘(foldl appT (conT tyName) (map return ty))’
|
281 | worker' (foldl appT (conT tyName) (map return ty)) context [] (zip [0..] cons)
| ^^
src/Data/SafeCopy/Derive.hs:286:21: error:
• Couldn't match expected type ‘Type’ with actual type ‘[Type]’
• In the second argument of ‘(==)’, namely ‘tyIndex’
In the expression: ty == tyIndex
In a stmt of a pattern guard for
a case alternative:
ty == tyIndex
|
286 | | ty == tyIndex ->
| ^^^^^^^
src/Data/SafeCopy/Derive.hs:287:61: error:
• Couldn't match expected type ‘[Type]’ with actual type ‘Type’
• In the second argument of ‘map’, namely ‘ty’
In the third argument of ‘foldl’, namely ‘(map return ty)’
In the first argument of ‘worker'’, namely
‘(foldl appT (conT tyName) (map return ty))’
|
287 | worker' (foldl appT (conT tyName) (map return ty)) context [] [(0, con)]
|
I tried this myself a while ago:
http://hub.darcs.net/dag/safecopy/patch/20121018075208-6eb02
It worked but didn't generate very efficient instances, and certainly not the same instances as deriveSafeCopy. The problem was with my code being too stupid, and probably not Generic not being powerful enough or anything like that. It should be perfectly doable.
Filing this here in case anyone wants to take a stab at it, since I wasn't smart enough to do it. I might try my hand at it again though some time.
I see that the github repo already loosened the upper bound on vector; I'm just requesting a release to hackage.
Can you add it?
instance SafeCopy a => SafeCopy (Vector a) where
getCopy = contain $ fromList <$> safeGet
putCopy = contain . safePut . toList
Thanks
It would be nice to be able to add SafeCopy instances in libraries without getting inferred Unsafe, even if it means writing the instances by hand (until we have generics).
Is there anything exported from Data.SafeCopy that actually is unsafe? I don't think the TH code is a problem in itself because a downstream consumer would just be unable to use TH with -XSafe anyway.
If there's no objections I'll fire up a pull request adding a conditional Trustworthy claim.
Hi folks, I've just installed GHC 9.2.4 (now that is the recommended versio) and tried to compile one of my projects that is working fine in GHC 8.10.7.
To rule out many details, let's say that I have to define two mutual recursive types TypeA and Type B. With 9.2.4 I get errors that can be abstracted as
Could not deduce (SafeCopy TypeB) ... arising from a use of ‘getSafePut’ ... in
$(deriveSafeCopy 0 'base ''TypeA)
and if I try to swap the order of definitions of TypeA/TypeB I get the dual error message.
Am I missing something new that has to be done with 9.2.4 or is it a bug in deriveSafeCopy?
Thanks in advance for any help.
Should I make a pull request or is it enough to post it here.
time is currently
time < 1.6, but is safe to change it to time < 1.7, since its [changelog](time < 1.6,) doesn't mention something that could harm the library.
PS: I would also like to mention that the changes in #37 are merged here but not yet uploaded to hackage and it's the last step to have it working with GHC 8
Seems like copy and paste mistake.
https://github.com/acid-state/safecopy/blob/master/src/Data/SafeCopy/SafeCopy.hs#L215
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.