Comments (11)
@spuddybike , could you provide your config file named format32-stata please?
from extract2ddi.
agency=uk.closer
ddilang=en-GB
stats=max,min,valid,invalid,freq,stdev
outputfile=test-file-stata-3.2
sumstats=TRUE
from extract2ddi.
Just have sent you the recent jar file over the email to try replicating the OutOfMemory error
from extract2ddi.
Have replicated it with the recent dta file. Here is why the exception is hapenning:
![Screenshot 2024-03-14 at 22 28 36](https://private-user-images.githubusercontent.com/4246666/312987310-f922044a-028f-4edc-a399-0749a2d1970b.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTY0MDA2MTksIm5iZiI6MTcxNjQwMDMxOSwicGF0aCI6Ii80MjQ2NjY2LzMxMjk4NzMxMC1mOTIyMDQ0YS0wMjhmLTRlZGMtYTM5OS0wNzQ5YTJkMTk3MGIucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDUyMiUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA1MjJUMTc1MTU5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NDdmYmM4MGVmYmIwMzIxZjU0NDkxZmM3NmM5MWI2Yzg0NDViNTM0ZmE4ZjMzNjE5ODAzMjljNGVhZmIwYTA1YSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.2Baa5N5bz_mqGhTU6aXIOumKXbFH_JBBQRpGnAhqtlI)
In essence, the numberOfEntries variable gets to 1818389564
and when we initialise array of ints with that size we get OutOfMemory error.
The exception is hapenning in edu.cornell.ncrn.ced2ar.stata.impl.DtaReader
from extract2ddi.
Based on stepping through the code, @spuddybike and I uncovered that OutOfMemory
resulted from an incorrect value being read into numberOfEntries variable from a file.
from extract2ddi.
@alkondrashov see https://www.loc.gov/preservation/digital/formats/fdd/fdd000471.shtml This I think might help in resolving the OOM error, as it indicates where the Number ofEntries is stored.
from extract2ddi.
@alkondrashov see also https://www.stata.com/help.cgi?dta section 5
from extract2ddi.
As discussed, @spuddybike will provide another sample .dta file to test this error. We discovered that the issue is because the variables are being read incorrectly.
from extract2ddi.
test-file-data-types-118.dta.zip
@alkondrashov file saved to Format 118 missing Stata for testing
from extract2ddi.
Error created by SPSS export to Stata
from extract2ddi.
SPSS Error
from extract2ddi.
Related Issues (20)
- Variable Label not correct in Formats 32 and 33 HOT 2
- NumberFormatException error HOT 1
- Add validation test for 3.2 and 3.3 schemas
- Add support for Stata for 3.2 output HOT 2
- Add software tag for SPSS 3.2 HOT 1
- Add optional citation to 3.2 Format HOT 4
- Add software tag for SPSS 3.3 HOT 3
- DataItem attribute values missing from Stata output HOT 3
- DataItem attribute values missing from SPSS 3.3 output HOT 3
- DataItem attribute values not populated correctly from SPSS 3.2 output HOT 3
- TopLevelReference is not referenced again in 3.3 output
- Duplicate variable CodeListReference IDs in 3.3. output
- Duplicate VariableStatistics IDs in 3.3. output HOT 1
- Identical Category fragments are repeated in 3.3 output HOT 1
- String variables with Missing and Values specified cause a null pointer exception (3.3 Fragment)
- Debug output should write out column HOT 2
- RecommendedDataType should be Numeric not type HOT 1
- Validation of outputs
- Add software tagging for SPSS and Stata 3.3 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from extract2ddi.