Comments (6)
Thank you for using this library and for taking the time to register an issue. Could you provide a sample project (preferably a test) so that we can reproduce this?
from spring-batch-extensions.
I do not have any test nor sample project ready, but it is just about using StreamingXlsxItemReader
with Excel file >10MB
from spring-batch-extensions.
Could you take some time to prepare a test project? You know what you've done to do this, when I try to setup a test it might not work due to differences in file, test etc. etc. So please take some time to make this into a test. You can share the repo or the project as a zip with this ticket.
from spring-batch-extensions.
I did a quick test with a 50Mb test file and I had no issues reading it. 14 columns of data with 500000 rows. So without a sample or test to reproduce this I'm afraid I have no clue on where to look for fixing this. I tested with both the standard Apache POI used in the build version 4.1.2 and the most recent version 5.2.2, neither of them throw an exception.
from spring-batch-extensions.
from spring-batch-extensions.
A start would be to add the full stacktrace to this thread instead of only the message, that might clear some things up. I did some quick googling and do you have an XLSX with large text blocks in them? That appears to be an known issue inside Apache POI?!
As a workaround (as mentioned in the error message as well), before executing your job you can do the following
org.apache.poi.util.IOUtils.setByteArrayMaxOverride(Integer.MAX_VALUE);
It will now use a larger range. The range I found for 100_000_000 is for OLE object (wrapper around some binary object). Looking at that I'm not sure if that is something we should fix in Spring Batch Excel, or that you as a user should set a better maximum. We could however make it easier to specify the maximum for a job/step and reset it afterwards, although you still would need to provide a new MAX value.
The max for this specific record type is there to prevent malicious XLSX files to be read and prevent OOM or worse.
from spring-batch-extensions.
Related Issues (20)
- Upgrade maven-compiler-plugin to 3.10.1
- Upgrade surefire/failsafe to version 3.0.0-M7
- Upgrade checkstyle plugin to 3.2.0
- Upgrade Maven Javadoc Plugin to 3.4.1
- Downgrade Checkstyle to version 9.3
- Upgrade to Spring Batch 5 HOT 5
- Multithread with specific chunk size the reader distributes duplicates records across threads HOT 2
- startStatement() should not be required in Neo4jItemReader HOT 1
- Read columns as arrays HOT 4
- getColumnNames(Sheet sheet) → UnsupportedOperationException: Getting row by index not supported when streaming. HOT 1
- IsoFormattingDateDataFormatter does not work HOT 4
- DefaultRowset method getColumnValue is Missing HOT 1
- Date format when reading HOT 3
- Issue with spring-batch-excel using Resource which might not have getFile() implemented and does not throw a FileNotFoundException exception HOT 4
- Publish Spring Batch Excel 0.1.1 artifacts to maven HOT 3
- Upgrade to Spring Batch 4.3.5
- Need of reading one particular sheet. HOT 3
- AbstractExcelItemReader ignores a number of empty rows inbetween filled set of rows.It would help if the this functioning is made configurable since this causes incorrect meta data provided for the row number. HOT 2
- When extracting data from the RowSet (rs) year in date cell is shortened HOT 20
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spring-batch-extensions.