jberryman / directory-tree Goto Github PK
View Code? Open in Web Editor NEWA simple directory-like tree datatype, with useful IO functions, for Haskell
License: Other
A simple directory-like tree datatype, with useful IO functions, for Haskell
License: Other
I'm not sure what's going on in this failure. The test suite works correctly from my machine, but fails from the Stackage Docker image. I'm temporarily disabling the test suite in future Docker builds.
Test suite test: RUNNING...
-- The following tests will either fail with an error
-- message or with an 'undefined' error
OK
OK
OK
OK
test: No match in record selector err
Test suite test: FAIL
I've been playing with this library to create some sort of filemanager. The idea was to read the directories lazily and stuff all possible information I could ever need into the a
field of DirTree a
.
The problem is, that this doesn't really allow me to save information for the directories. So I came up with my own data type to extend it:
data AnchoredDirTree a b =
(:/) { anchor :: FilePath, dirTree :: DirTree a b }
deriving (Show, Ord, Eq)
data DirTree a b =
Failed {
name :: FileName
, err :: IOException
}
| Dir {
name :: FileName
, contents :: [DirTree a b]
, dir :: a
}
| File {
name :: FileName
, file :: b
} deriving Show
And then wrote a few Bifunctor instances and adjusted the rest of the functions, which seemed fairly straight forward, see this gist: https://gist.github.com/hasufell/f15de337f763570b2adf
Would it make sense to provide this sort of data structure in this library, maybe under a separate module?
Lets concider following code:
import System.Directory.Tree
import qualified Data.Foldable as F
import qualified Data.Traversable as T
main = do
dir <- readDirectoryWithL readFile "directory"
let
a:xs = contents $ dirTree dir
x = "the file content:" ++ file a
print $ name a
_ <- getLine -- change the file content manually
print $ x
In the directory "directory" there is only one file "a.txt" containing "TEST_1"
When the program executes "_ <- getLine" we change the content of that file to "TEST_2", the output of the program is:
"the file content:TEST_1".
The problem is, that file content is read when we are checking the file name with "print $ name a". When the line is commented, the output of this program is (while changing the file content manually as described above):
"the file content:TEST_2".
I thing this is bug, because files should not be read until we need to - when we are checking the filename, this should not imply, we read them.
I came to the package with a much simpler expectation - namely to collect the filenames recursively in a directory and build a tree structure (but without the content). I could not find anything else close. Collecting the file content is overkill for me, however I would love to have a ToJSON instance....
thank you!
Deep within the code there is this:
-- using unsafePerformIO to get "lazy" traversal:
buildLazilyUnsafe' :: Builder a
buildLazilyUnsafe' f p = handleDT n $
do isFile <- doesFileExist p
if isFile
then File n <$> f p
-- HERE IS THE UNSAFE CODE:
else Dir n . fmap (rec . combine p) <$> getDirsFiles p
-- TODO: this should really be unsafeInterleaveIO
where rec = unsafePerformIO . buildLazilyUnsafe' f
n = topDir p
There is a way to do this using unsafeInterleaveIO
instead, and I can think of at least one benefit of the switch: unsafeInterleaveIO
guarantees that the IO
action it wraps is evaluated at most once (and saved for future references, very much how lazy thunks work in Haskell). Any action wrapped in unsafePerformIO
can be run multiple times and give different results if the filesystem changed underneath you in between the two pure evaluations.
This isn't an issue that affects me but somebody brought this my attention at a Hackathon and asked me about the unsafePerformIO
and unsafeInterleaveIO
distinction and they also wanted a bug report opened for this, so I wrote one up for them.
When loading EXAMPLES/Examples.hs I get:
*System.Directory.Tree> :l EXAMPLES/Examples.hs
[1 of 2] Compiling System.Directory.Tree ( System/Directory/Tree.hs, interpreted )
[2 of 2] Compiling Main ( EXAMPLES/Examples.hs, interpreted )EXAMPLES/Examples.hs:9:1: error:
Failed to load interface for `Data.Digest.Pure.MD5'
Use -v to see a list of the files searched for.EXAMPLES/Examples.hs:10:1: error:
Failed to load interface forData.ByteString.Lazy' It is a member of the hidden package
bytestring-0.10.8.1'.
Perhaps you need to add `bytestring' to the build-depends in your .cabal file.
Use -v to see a list of the files searched for.
Failed, modules loaded: System.Directory.Tree.
# ls /dev/fd
0 1 2 3
# file /dev/fd
/dev/fd: symbolic link to /proc/self/fd
# ls /proc/self/fd
0 1 2 3
But when reading it via readDirectoryWithL
I get more files:
*System.Directory.Tree> aa <- readDirectoryWithL (\x -> return x) "/dev/fd"
*System.Directory.Tree> aa
(:/) {anchor = "/dev", dirTree = Dir {name = "fd", contents = [
File {name = "0", file = "/dev/fd/0"},
File {name = "1", file = "/dev/fd/1"},File {name = "2", file = "/dev/fd/2"},
File {name = "3", file = "/dev/fd/3"},File {name = "4", file = "/dev/fd/4"},
File {name = "5", file = "/dev/fd/5"},File {name = "6", file = "/dev/fd/6"},
File {name = "7", file = "/dev/fd/7"},File {name = "8", file = "/dev/fd/8"},
File {name = "9", file = "/dev/fd/9"},File {name = "10", file = "/dev/fd/10"}]}}
Surely those directories are special and it seems laziness causes weird behavior here. Do these directories change frequently or what is the thing here?
For me goes about 35 levels deep and then gives an IO error:
readDirectoryWithL return "/usr/bin/X11"
I hacked it for my specific use case by treating any symlink that doesn't point to a git-annex repo as a file, but I haven't thought of a solution that would be useful to others.
Also thanks for making this module! I've been using it pretty heavily and this is the first error.
It would be great if there were a free variable on the Dir constructor as well--could be the same type as the one on File or a different type. I will need this for something I am working on--I might do it and send you a pull request.
I'd love to see a new relase to hackage and latest GHC support ! Any chance?
So I went with the bifunctor thing from #7 for now and also noticed that the AnchoredDirTree
is not suitable for all use cases. One might like to have an actual zipper. So I came up with this module: https://github.com/hasufell/hsfm/blob/master/src/Data/DirTree/Zipper.hs
It is still lazy and also allows to build the parent dirs/breadcrumbs lazily if you drop into a specific directory via zipLazy, which is sort of the counterpart to readDirectoryWithL
.
I just throw this out for the discussion. It would be easy to convert it to the current directory-tree API.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.