Document how to work with directories about zipzap

[pixelglow]

The zip file format is a flat sequence of entries, not an actual hierarchy. zipzap simply reflects this.

To "populate" a directory, simply add additional entries prefixed with the directory name and forward slash e.g. by using -[NSString stringByAppendingPathComponent:].

To "find out its contents" simply search for entries that are prefixed with the directory name and forward slash e.g. by using -[NSString hasPrefix:].

[mikeabdullah]

That makes sense.

In which case, what is +archiveEntryWithDirectoryName: for — purely as a marker for empty directories? Or is it required to note the presence of all directories?

[pixelglow]

I see that most zip files have directory entries preceding the actual directory file entries, but the spec doesn't appear to need them. I haven't tried what would happen if a zip file didn't have a directory entry but had directory file entries -- presumably a robust implementation would create the directory anyway but with default settings.

[mikeabdullah]

Thanks, I think I understand now. From my diggings, directory entries are basically the same as regular files, but report themselves to be uncompressed and of zero length.

I'm happy to add these details to the README file.

Do you happen to know how unarchivers cope with intermingled entries? e.g. something like this:

directory/
directory/file1
some-random-file
directory/file2

I'm mostly curious because if handled properly, it would allow for some interesting optimisation when updating an existing archive.

But also, as I understand it, clients of zipzap are currently responsible for ensuring directory structures are created properly themselves. It would seem a lot nicer if the library abstracted that away and presented a proper directory hierarchy to clients.

[pixelglow]

Do you happen to know how unarchivers cope with intermingled entries?

Not having a test suite of unarchivers available :-), the answer is: no. Sounds like a case for stackoverflow.

But also, as I understand it, clients of zipzap are currently responsible for ensuring directory structures are created properly themselves. It would seem a lot nicer if the library abstracted that away and presented a proper directory hierarchy to clients.

Clients of zipzap are currently responsible for ensuring directory structures.

I created zipzap to scratch an open-source itch for my Instaviz app. I didn't really need a fully hierarchical system at the time, and so I designed ZZArchive to be minimal but fully functional.

I would see any fully hierarchical system as a layer over ZZArchive, perhaps as a ZZHierarchicalArchive or ZZArchiveSystem, that would need to wrap the entries as well and provide for seamless reading and writing.

OTOH, it could be possible to create something around directory entries without the overhead of another layer, but that would have to handle reading and writing "sub-entries" pretty seamlessly. If you have some ideas about a possible design, I'd welcome the input.

[mikeabdullah]

My first draft of an NSFileWrapper-based API is at https://github.com/mikeabdullah/zipzap/blob/nsfilewrapper/zipzap/ZZArchiveWrapper.h

One nicety is it lets you work with a proper directory structure, including stopping you from accidentally naming two files the same

[pixelglow]

Excellent work on ZZArchiveWrapper !!

I only had a brief overview, but you might want to consider the following:

• Rather than trying to subsume all of the NSFileWrapper functionality e.g. allowing initRegularFileWithContents: to do something sensible with non-archives, it would probably be best to have a single initialiser that always returns or assumes a zip archive.
• You might have to need to have both ZZArchiveWrapper and a ZZArchiveEntryWrapper implementations, although you might be able to get away with just having ZZArchiveWrapper public. The other implementation might make the code paths a little cleaner.
• Be aware that archive.entries are not guaranteed to be valid past a reload or a setEntries:. In particular, if you are squirrelling them away in your own array or references, then call those particular methods, the old entries will be non-nil objects but reading their properties or streaming may crash. (In particular, if you want to keep a reference, it should be a weak reference.) This is an optimisation so that we don't have to constantly copy memory around.
  • The preferred way of doing this is to keep a reference to the filename in the array, and using that to reference off archive.entries whenever you need to. Or you could keep NSData for the contents of individual entries etc.
  • Alternatively, if you are doing a reload or setEntries: in your own code, to trigger some kind of refresh of your system to clear out any cached entries.
• You could keep an instance field for a NSMutableData and use that for output via archiveWithData:. That might be more consonant with the saving model of NSFileWrapper i.e. atomic updating. I haven't fully thought about the consequences of using a file-based archive with NSFileWrapper, in particular what would happen if various add/remove methods are called in one thread and the writing is done is another thread.
• Unit tests!

[mikeabdullah]

I just noticed the problem of ZZArchiveEntrys becoming invalid yesterday. Have a fix available in pull request #17

[mikeabdullah]

in particular what would happen if various add/remove methods are called in one thread and the writing is done is another thread

Like most Cocoa classes, NSFileWrapper expects to only be used on a single thread at a time. Anything else is programmer error

[pixelglow]

Like most Cocoa classes, NSFileWrapper expects to only be used on a single thread at a time.

The issue is somewhat more subtle than that.

A single NSFileWrapper is often used by multiple threads but as you say, it is meant to be used by a single thread at a time. For example, when you use NSFileWrapper in the UIDocument architecture, one part of the API accesses the file wrapper in your main thread, then it is handed over to the document I/O thread -- this is multiple threads using the wrapper one thread at a time.

For objects that are fairly well self-contained this is fine. However if you create an ZZArchiveEntry, the blocks may then capture your own objects in one thread. Then if you use ZZArchiveEntry in another thread at the same time as the original objects are used in the original thread, you unexpectedly set up a multi-threaded race condition.

I don't know if your archive wrapper has these issues. However, I would advise to keep the use of an ZZArchiveEntry fully within one thread, and not to inadvertently share them by persisting them in your class. Share or persist the NSData or whatever other self-contained objects there are instead, and create a ZZArchiveEntry at the moment you want to write things out. This is generally the approach I've taken for Instaviz 2.0.