marschall / memoryfilesystem Goto Github PK

e.g. somepath.getFileSystem().provider.setAttribute( file, "basic:lastModifiedTime", sometime );

throws java.lang.UnsupportedOperationException: file attribute viewinterface java.nio.file.attribute.PosixFileAttributeView not supported

found in 0.6.2
worked in 0.5.4

Note: Test ran on win 7

Once you retrieve FileAttributes the values in the object should not change with the underlying file.

see: https://github.com/openCage/niotest/blob/master/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest3FileIT.java test testFileAttributesAreImmutable

see test in https://github.com/openCage/niotest/blob/16ee86ed6d855ea129799c3185464b011049fe26/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest9WrongProviderIT.java

If I create a symlink to a file, then attempt to call Files.readAllLines() on the symlink, I get the following crash:

  at com.github.marschall.memoryfilesystem.MemoryFileSystem$1.value(MemoryFileSystem.java:332)
  at com.github.marschall.memoryfilesystem.MemoryFileSystem$1.value(MemoryFileSystem.java:299)
  at com.github.marschall.memoryfilesystem.MemoryFileSystem$11.value(MemoryFileSystem.java:592)
  at com.github.marschall.memoryfilesystem.MemoryFileSystem.withLockDo(MemoryFileSystem.java:612)
  at com.github.marschall.memoryfilesystem.MemoryFileSystem.withWriteLockOnLastDo(MemoryFileSystem.java:585)
  at com.github.marschall.memoryfilesystem.MemoryFileSystem.getFile(MemoryFileSystem.java:299)
  at com.github.marschall.memoryfilesystem.MemoryFileSystem.newInputStream(MemoryFileSystem.java:260)
  at com.github.marschall.memoryfilesystem.MemoryFileSystemProvider.newInputStream(MemoryFileSystemProvider.java:247)
  at java.nio.file.Files.newInputStream(Files.java:152)
  at java.nio.file.Files.newBufferedReader(Files.java:2781)
  at java.nio.file.Files.readAllLines(Files.java:3199)

Here is a reproduction:

private static void test() throws IOException {
    final FileSystem fileSystem = MemoryFileSystemBuilder.newLinux().build("1");

    final List<String> lines = new ArrayList<>();
    lines.add("Hello world");

    final Path filePath = fileSystem.getPath("/").resolve("file");
    Files.write(filePath, lines, StandardCharsets.UTF_8);

    final Path linkPath = filePath.resolveSibling("link");
    Files.createSymbolicLink(linkPath, filePath);

    final List<String> lines1 = Files.readAllLines(linkPath, StandardCharsets.UTF_8);
    System.out.println(lines1);
}

I believe the problem is here, where there is a third case of MemorySymbolicLink that is not being handled:

      if (storedEntry instanceof MemoryFile) {
        return (MemoryFile) storedEntry;
      } else {
        throw new IOException("file is a directory");
      }

see: https://github.com/openCage/niotest/blob/16ee86ed6d855ea129799c3185464b011049fe26/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest4CopyIT.java
testMoveIntoItself

Hello.

The documentation for Files.getFileAttributeView states that the method returns null for unsupported views:

http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#getFileAttributeView%28java.nio.file.Path,%20java.lang.Class,%20java.nio.file.LinkOption...%29

So, if I create a memory filesystem and don't specify a PosixFileAttributeView to the builder, I would expect the above method to return null if I attempt:

    final PosixFileAttributeView posix_view = Files.getFileAttributeView(
      root, PosixFileAttributeView.class, LinkOption.NOFOLLOW_LINKS);

However, I get UnsupportedOperationException instead. This makes it somewhat less than useful for checking that code properly handles non-posix filesystems during testing.

java.lang.UnsupportedOperationException: file attribute viewinterface java.nio.file.attribute.PosixFileAttributeView not supported
    at com.github.marschall.memoryfilesystem.MemoryEntry.getFileAttributeView(MemoryEntry.java:256)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem$8.value(MemoryFileSystem.java:633)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem$8.value(MemoryFileSystem.java:629)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem.withLockDo(MemoryFileSystem.java:766)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem.withReadLockDo(MemoryFileSystem.java:725)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem.accessFile(MemoryFileSystem.java:675)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem.accessFileReading(MemoryFileSystem.java:663)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem.getFileAttributeView(MemoryFileSystem.java:629)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem$LazyFileAttributeView.getView(MemoryFileSystem.java:1415)
    at com.github.marschall.memoryfilesystem.MemoryFileSystem$LazyFileAttributeView.invoke(MemoryFileSystem.java:1403)

Views on not existing files should probably work.

'.' normalize vs ./s.txt normalize

The following C:folder\file.txt is relative folder unlike C:\folder\file.txt

http://blog.travis-ci.com/2013-11-26-test-your-java-libraries-on-java-8/

MemoryFileSystemTest#readSymlinkLoop() is failing

Profiling suggests that creating a collator is a performance bottleneck. Consider introducing a cache.

Opening a directory for reading should work on Unix but fail on Windows.

see: https://github.com/openCage/niotest/blob/16ee86ed6d855ea129799c3185464b011049fe26/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest3FileIT.java
testTruncateToNegativeSizeThrows

Hello,

Using the following code:

  @Test
  public void test() throws IOException {
    try (FileSystem fs = MemoryFileSystemBuilder.newEmpty()
        .addFileAttributeView(DosFileAttributeView.class)
        .addFileAttributeView(PosixFileAttributeView.class)
        .build("perm")) {

      final Path path = fs.getPath("readable-at-first");
      Files.createFile(path,  PosixFilePermissions.asFileAttribute(PosixFilePermissions.fromString("r--r--r--")));

      assertThat(Files.isReadable(path)).isTrue(); // ok
      assertThat(Files.isWritable(path)).isFalse(); // ok
      assertThat(Files.isExecutable(path)).isFalse(); // ok

      Files.setPosixFilePermissions(path, PosixFilePermissions.fromString("rw-r--r--")); // FAIL.

      assertThat(Files.isReadable(path)).isTrue(); // ok
      assertThat(Files.isWritable(path)).isTrue(); // (should be) ok
      assertThat(Files.isExecutable(path)).isFalse(); // ok
    }
  }

I get this exception:

java.nio.file.AccessDeniedException
    at com.github.marschall.memoryfilesystem.MemoryEntry$MemoryPosixFileAttributeView.checkAccess(MemoryEntry.java:810)
    at com.github.marschall.memoryfilesystem.MemoryEntry.checkAccess(MemoryEntry.java:168)
    at com.github.marschall.memoryfilesystem.MemoryEntry$MemoryPosixFileAttributeView.setPermissions(MemoryEntry.java:788)

This fails here:

    @Override
    public void setPermissions(Set<PosixFilePermission> perms) throws IOException {
      if (perms == null) {
        throw new IllegalArgumentException("permissions must not be null");
      }
      try (AutoRelease lock = this.entry.writeLock()) {
        this.entry.checkAccess(AccessMode.WRITE); // <--- HERE.
        this.permissions = toMask(perms);
      }
    }

I think the check should not verify for the current permissions, but simply if the current owner is the file's owner, like if I were calling chmod.

On the other hand: delete the file works.

Should be passed in from FileSystem or view

Directory operations like create, move, delete should require a write permission.

environment: osx 10.9 java 8 (1.8.0), memoryfilesystem 0.5.3

    FileSystem fs = MemoryFileSystemBuilder.newEmpty().build("53");
    Path path = fs.getPath("one").toAbsolutePath();
    Files.write( path, "hallo world".getBytes("UTF-8"));
    Files.readAllBytes(path);

produces

Exception in thread "main" java.nio.channels.NonReadableChannelException
at     com.github.marschall.memoryfilesystem.BlockChannel.readCheck(BlockChannel.java:65)
at com.github.marschall.memoryfilesystem.BlockChannel.readLock(BlockChannel.java:75)
at com.github.marschall.memoryfilesystem.BlockChannel.read(BlockChannel.java:82)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
at java.nio.file.Files.read(Files.java:3102)
at java.nio.file.Files.readAllBytes(Files.java:3155)
at org.opencage.lindwurm.niotest.Marschal53Test.main(Marschal53Test.java:21)

this works fine in the latest java 7.

see: https://github.com/openCage/niotest/blob/16ee86ed6d855ea129799c3185464b011049fe26/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest7ClosedIT.java
testClosedFSisClosed

FileTime objects can be quite large compared to a long. We can get a way with storing just a long and "expanding" to FileTime when reading the attributes.

This is made possible by the refactoring for #16

should throw FileSystemNotFoundException

The contract of java.nio.file.attribute.BasicFileAttributeView.setTimes(FileTime, FileTime, FileTime) says null is OK and means don't change the value. Currently we throw an exception.

needs a refactoring of the views

From the documentation of Files.newBufferedWriter:

If no options are present then this method works as if the CREATE, TRUNCATE_EXISTING, WRITE options are present.

I believe the problem should be addressed here: https://github.com/marschall/memoryfilesystem/blob/master/src/main/java/com/github/marschall/memoryfilesystem/MemoryFileSystem.java#L268

This is very easy to test by doing:

try (BufferedWriter writer = Files.newBufferedWriter(fs.getPath("/foo/bar/baz.txt"))) {
  writer.write('c');
}

this can be done with a single byte

When you copy a path with the attributes the basic attributes are set on the source path instead of the target path.

This code fails:

    import java.nio.file.*; import java.io.*;
    @Test
    public void testMemoryFileSystemMultipleClose() {
      FileSystem fs = MemoryFileSystemBuilder.newLinux().build("ID");
      Path foo = fs.getPath("foo");

      Files.write(foo, java.util.Collections.singleton("Foo"));

      InputStream is = Files.newInputStream(foo);
      is.close();
      is.close(); //remove the second call to close() and all is OK

      InputStream is2 = Files.newInputStream(foo); //NoSuchFileException
    }

This is because the openCount of a MemoryFile being less than zero is an indication that the file does not exist. I would suggest one of two fixes.

First option: BlockInputStream keeps track of whether it has been closed; closing an already-closed stream is a no-op. This seems to be the behaviour of the standard library's streams (as I have found this "bug" in code which works in reality!)

    @Override
    public void close() throws IOException {
      if (this.checker.isOpen()) {
        this.checker.close();
        this.memoryContents.modified();
        this.memoryContents.closedStream(this.path, this.deleteOnClose);
      }
    }

Second option: closing a stream/channel on a Path should not let the openCount drop below zero. that is

    this.openCount = Math.max(this.openCount - 1, 0)

However, I have not tested either changes

memoryfilesystem.getPath("").getName() should return itself.

See also: https://github.com/openCage/niotest/blob/master/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest1NoContentIT.java test testDefaultHasANameAndItsItself()

Currently we store data in views, they should only be views on data.

http://docs.travis-ci.com/user/caching/

currently only available for private repos

This line is wrong: https://github.com/marschall/memoryfilesystem/blob/master/src/main/java/com/github/marschall/memoryfilesystem/BlockInputStream.java#L131

If you compare to java.io.ByteArrayInputStream which also reads a byte from a byte array, the byte should have a & 0xff in order to return as an int in the specified range.

that is, the method should be:

@Override
public int read() throws IOException {
  byte[] data = new byte[1];
  int read = this.read(data);
  if (read == -1) {
    return read;
  } else {
    return data[0] & 0xff;
  }
}

    Path parent = FS.getPath( "linkParent").toAbsolutePath();
    Files.createDirectories( parent );
    Path target = FS.getPath( "target").toAbsolutePath();
    Files.createDirectories( target );
    Files.createSymbolicLink( parent.resolve("link"), target );

    Files.write( parent.resolve("link").resolve("kid"), "hallo".getBytes("UTF-8"));

throws:
java.nio.file.NotDirectoryException: /linkParent/link
at com.github.marschall.memoryfilesystem.MemoryFileSystem$11.value(MemoryFileSystem.java:670)
at com.github.marschall.memoryfilesystem.MemoryFileSystem.withLockDo(MemoryFileSystem.java:724)
at com.github.marschall.memoryfilesystem.MemoryFileSystem.withWriteLockOnLastDo(MemoryFileSystem.java:665)
at com.github.marschall.memoryfilesystem.MemoryFileSystem.getFile(MemoryFileSystem.java:321)
at com.github.marschall.memoryfilesystem.MemoryFileSystem.newOutputStream(MemoryFileSystem.java:295)
at com.github.marschall.memoryfilesystem.MemoryFileSystemProvider.newOutputStream(MemoryFileSystemProvider.java:266)

A name count of 1 seems strange but is called for in the spec at fits the behaviour of the default filesystem implementation

see https://github.com/openCage/niotest/blob/master/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest1NoContentIT.java test testDefaultHasNameCount1

When you use .setUmask() on the Linux memory fs builder, it sets the mask applied to all file creations as in:

mode | umask

But this is not the meaning of an umask! u is for Undo, it is the set of permissions to be removed from each mode:

mode & ~umask

Hello!

I realize the title of the ticket is a little strange, but I feel that the "case-insensitive" setting of the memory filesystem may be too strong. That is, when enabling a case insensitive filesystem via MemoryFileSystemBuilder.setCaseSensitive(false), the case of the original filename appears to be lost. Rather than just perform comparisons of filenames in a case insensitive manner, it seems that all names are translated to uppercase before being written to "disk".

Why does this matter? Let's say you're writing a program to catalog disks. You walk a filesystem, reading filenames and metadata into a graph/tree structure (effectively, a read-only filesystem that does not store actual file data). In order to allow the graph/tree structure to be filesystem-agnostic, the names in the tree structure have to be case sensitive. If the original filesystem was case insensitive, there is no problem, because files FILE0.TXT and FiLe0.TxT cannot both be in the same directory. If the original filesystem was case sensitive, there is no problem because the graph/tree representation is also case sensitive.

The user creates a file called File0.txt on a case insensitive filesystem. The user then runs the catalog program on this filesystem. Later, the user tries to open File0.txt in the catalog: Uh oh! The filesystem quietly converted the name to FILE0.TXT when the catalog was created and the file cannot be found! This leads to subtle problems when writing test suites that are parameterized by filesystems: The tests have to always assume completely uppercase filenames to be able to check for the same results across filesystems. On a real case-insensitive filesystem such as NTFS, this does not happen.

I believe the correct behaviour should be to return filenames using the case that was used when they were created, but to do case-insensitive string comparisons when it becomes time to compare names.

Should probably be its own class

• dependencies
• schema URLs
• URLs in Readme

This can be implemented with a single byte

REPLACE_EXISTING

If the target file exists, then the target file is replaced if it is not a non-empty directory. If the target file exists and is a symbolic link, then the symbolic link itself, not the target of the link, is replaced.

NOFOLLOW_LINKS

Symbolic links are not followed. If the file is a symbolic link, then the symbolic link itself, not the target of the link, is copied. It is implementation specific if file attributes can be copied to the new link. In other words, the COPY_ATTRIBUTES option may be ignored when copying a symbolic link.

Check if the default file system has any specific implementations.

see https://github.com/openCage/niotest/blob/master/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest2DirIT.java test testKidsOfRelativeDirAreRelative

should throw FileSystemException

see: https://github.com/openCage/niotest/blob/16ee86ed6d855ea129799c3185464b011049fe26/src/main/java/org/opencage/lindwurm/niotest/tests/PathTest4CopyIT.java
testMoveRoot