GithubHelp home page GithubHelp logo

ome-common-java's People

Stargazers

avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

sbesson snoopycrimecop melissalinkert simleo dgault hflynn mtbc mohofo7 jburel joshmoore allencellsoftware manics-archive atarkowska linruicheng1997 connortruex nicokiaru bulksecuritygeneratorprojectv2 asbaharoon java-cds-club

ome-common-java's Issues

Add write buffering in RandomAccessOutputStream

In looking briefly at glencoesoftware/raw2ometiff#104 and ome/bioformats#3983, in both cases repeated calls to NIOFileHandle.setLength cause a significant slowdown. Looking more closely at how writing is implemented in NIOFileHandle and RandomAccessOutputStream, it appears that there is effectively no write buffering happening.

While we can think about shorter term ways to improve bfconvert, raw2ometiff, etc., I think ultimately we'll want to modernize some of the writing implementations here. Some specific ideas:

  • Allow setting an initial file length for writable handles/streams. In the file conversion use cases, we know the size of the uncompressed pixel data being written, and can estimate an initial length based on the compression type used.
  • Buffer writes, similar to what RandomAccessInputStream does for reads. Things like NIOFileHandle.writeInt already create a ByteBuffer, store the int in the ByteBuffer, and then write out the ByteBuffer; switching that to use a reusable ByteBuffer of configurable size should be straightforward.
  • Possibly better file length handling on close. Calls to setLength are pretty specific to what's being written, so that the length is typically not larger than what actually gets written. close() methods don't currently do anything to fix the length, e.g.:
    • Open an existing file for writing. The size on disk was 100 bytes.
    • Set the length to 250 bytes, anticipating 150 bytes of writes before close().
    • Write 110 bytes (40 anticipated bytes not written due to incorrect estimation of compression ratio).
    • Call close() - should the length remain 250 bytes as now, or be reset to 210?

Re-activate HTTP and S3 tests

See #53 (comment)

The execution of the tests was lost during the migration from Travis CI to GitHub workflows and should be re-activated either as a separate workflow or as an extra step of the maven.yml workflow

Trying to find location of commons-io 1.4 requirement in repository

I'm trying to trace the place where the 1.4 requirement for commons-io is in this repository.

[INFO] +- ome:formats-gpl:jar:6.1.1:compile
[INFO] | +- org.openmicroscopy:ome-common:jar:6.0.3:compile
...
[INFO] | | +- commons-lang:commons-lang:jar:2.4:compile
[INFO] | | - commons-io:commons-io:jar:1.4:compile

Fix findbugs and activate in Travis

Follow-up task of #1, activating mvn findbugs:check in Travis requires to fix 30 bugs in the code base

[INFO] --- findbugs-maven-plugin:3.0.4:check (default-cli) @ ome-common ---
[INFO] BugInstance size is 30
[INFO] Error size is 0
[INFO] Total bugs: 30
[INFO] loci.common.BZip2Handle.isBZip2File(String) may fail to clean up java.io.InputStream on checked exception [loci.common.BZip2Handle, loci.common.BZip2Handle, loci.common.BZip2Handle] Obligation to clean up resource created at BZip2Handle.java:[line 86] is not dischargedPath continues at BZip2Handle.java:[line 87]Path continues at BZip2Handle.java:[line 88] OBL_UNSATISFIED_OBLIGATION_EXCEPTION_EDGE
[INFO] loci.common.BZip2Handle.isBZip2File(String) ignores result of java.io.FileInputStream.read(byte[]) [loci.common.BZip2Handle] At BZip2Handle.java:[line 88] RR_NOT_CHECKED
[INFO] Nullcheck of ByteArrayHandle.buffer at line 112 of value previously dereferenced in loci.common.ByteArrayHandle.setLength(long) [loci.common.ByteArrayHandle, loci.common.ByteArrayHandle] At ByteArrayHandle.java:[line 109]Redundant null check at ByteArrayHandle.java:[line 112] RCN_REDUNDANT_NULLCHECK_WOULD_HAVE_BEEN_A_NPE
[INFO] loci.common.ByteArrayHandle.read(ByteBuffer, int, int) ignores result of loci.common.ByteArrayHandle.read(byte[]) [loci.common.ByteArrayHandle] At ByteArrayHandle.java:[line 171] RR_NOT_CHECKED
[INFO] loci.common.ByteArrayHandle.readUTF() ignores result of loci.common.ByteArrayHandle.read(byte[]) [loci.common.ByteArrayHandle] At ByteArrayHandle.java:[line 362] RR_NOT_CHECKED
[INFO] loci.common.CRC.CRC_32_TABLE should be package protected [loci.common.CRC] At CRC.java:[line 64] MS_PKGPROTECT
[INFO] Switch statement found in loci.common.DateTools.convertDate(long, int, String, boolean) where default case is missing [loci.common.DateTools] At DateTools.java:[lines 144-158] SF_SWITCH_NO_DEFAULT
[INFO] loci.common.GZipHandle.isGZipFile(String) may fail to clean up java.io.InputStream on checked exception [loci.common.GZipHandle, loci.common.GZipHandle, loci.common.GZipHandle] Obligation to clean up resource created at GZipHandle.java:[line 83] is not dischargedPath continues at GZipHandle.java:[line 84]Path continues at GZipHandle.java:[line 85] OBL_UNSATISFIED_OBLIGATION_EXCEPTION_EDGE
[INFO] loci.common.GZipHandle.isGZipFile(String) ignores result of java.io.FileInputStream.read(byte[]) [loci.common.GZipHandle] At GZipHandle.java:[line 85] RR_NOT_CHECKED
[INFO] loci.common.IniList.flattenIntoHashMap() makes inefficient use of keySet iterator instead of entrySet iterator [loci.common.IniList] At IniList.java:[line 75] WMI_WRONG_MAP_ITERATOR
[INFO] loci.common.IniWriter.saveINI(IniList, String, boolean, boolean) makes inefficient use of keySet iterator instead of entrySet iterator [loci.common.IniWriter] At IniWriter.java:[line 87] WMI_WRONG_MAP_ITERATOR
[INFO] loci.common.Location.equals(Object) does not check for null argument [loci.common.Location] At Location.java:[lines 518-528] NP_EQUALS_SHOULD_HANDLE_NULL_ARGUMENT
[INFO] Should loci.common.Location$ListingsResult be a _static_ inner class? [loci.common.Location$ListingsResult] At Location.java:[lines 86-89] SIC_INNER_SHOULD_BE_STATIC
[INFO] loci.common.NIOFileHandle.defaultBufferSize should be package protected [loci.common.NIOFileHandle] At NIOFileHandle.java:[line 66] MS_PKGPROTECT
[INFO] loci.common.NIOFileHandle.defaultRWBufferSize should be package protected [loci.common.NIOFileHandle] At NIOFileHandle.java:[line 71] MS_PKGPROTECT
[INFO] loci.common.NIOFileHandle.readFully(byte[]) ignores result of loci.common.NIOFileHandle.read(byte[]) [loci.common.NIOFileHandle] At NIOFileHandle.java:[line 361] RR_NOT_CHECKED
[INFO] loci.common.NIOFileHandle.readFully(byte[], int, int) ignores result of loci.common.NIOFileHandle.read(byte[], int, int) [loci.common.NIOFileHandle] At NIOFileHandle.java:[line 367] RR_NOT_CHECKED
[INFO] loci.common.NIOInputStream.findString(boolean, int, String[]) may fail to close stream [loci.common.NIOInputStream] At NIOInputStream.java:[line 249] OS_OPEN_STREAM
[INFO] Unread public/protected field: loci.common.NIOInputStream.filename [loci.common.NIOInputStream] At NIOInputStream.java:[line 81] URF_UNREAD_PUBLIC_OR_PROTECTED_FIELD
[INFO] Unused public or protected field: loci.common.NIOInputStream.channel [loci.common.NIOInputStream] In NIOInputStream.java UUF_UNUSED_PUBLIC_OR_PROTECTED_FIELD
[INFO] loci.common.RandomAccessInputStream.findString(boolean, int, String[]) may fail to close stream [loci.common.RandomAccessInputStream] At RandomAccessInputStream.java:[line 323] OS_OPEN_STREAM
[INFO] loci.common.RandomAccessInputStream.skipBits(long) ignores result of loci.common.RandomAccessInputStream.skipBytes(int) [loci.common.RandomAccessInputStream] At RandomAccessInputStream.java:[line 393] SR_NOT_CHECKED
[INFO] loci.common.ReflectedUniverse.setVar(String, boolean) invokes inefficient Boolean constructor; use Boolean.valueOf(...) instead [loci.common.ReflectedUniverse] At ReflectedUniverse.java:[line 333] DM_BOOLEAN_CTOR
[INFO] loci.common.ReflectedUniverse.getVar(String) invokes inefficient new Integer(String) constructor; use Integer.valueOf(String) instead [loci.common.ReflectedUniverse] At ReflectedUniverse.java:[line 395] DM_NUMBER_CTOR
[INFO] loci.common.ReflectedUniverse.getVar(String) invokes inefficient new Long(String) constructor; use Long.valueOf(String) instead [loci.common.ReflectedUniverse] At ReflectedUniverse.java:[line 399] DM_NUMBER_CTOR
[INFO] loci.common.ReflectedUniverse.setVar(String, byte) invokes inefficient new Byte(byte) constructor; use Byte.valueOf(byte) instead [loci.common.ReflectedUniverse] At ReflectedUniverse.java:[line 338] DM_NUMBER_CTOR
[INFO] new loci.common.ReflectedUniverse(URL[]) creates a java.net.URLClassLoader classloader, which should be performed within a doPrivileged block [loci.common.ReflectedUniverse] At ReflectedUniverse.java:[line 85] DP_CREATE_CLASSLOADER_INSIDE_DO_PRIVILEGED
[INFO] Null passed for non-null parameter of new java.util.zip.ZipInputStream(InputStream) in loci.common.ZipHandle.resetStream() [loci.common.ZipHandle, loci.common.ZipHandle] Method invoked at ZipHandle.java:[line 175]Known null at ZipHandle.java:[line 170] NP_NULL_PARAM_DEREF
[INFO] loci.common.ZipHandle.isZipFile(String) ignores result of loci.common.IRandomAccess.read(byte[]) [loci.common.ZipHandle] At ZipHandle.java:[line 130] RR_NOT_CHECKED
[INFO] Incorrect lazy initialization of static field loci.common.services.ServiceFactory.defaultFactory in new loci.common.services.ServiceFactory() [loci.common.services.ServiceFactory] At ServiceFactory.java:[lines 78-79] LI_LAZY_INIT_STATIC
[INFO]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.