Stata fails with java.lang.OutOfMemoryError: Java heap space about extract2ddi

[alkondrashov]

@spuddybike , could you provide your config file named format32-stata please?

[spuddybike]

agency=uk.closer
ddilang=en-GB
stats=max,min,valid,invalid,freq,stdev
outputfile=test-file-stata-3.2
sumstats=TRUE

[alkondrashov]

Just have sent you the recent jar file over the email to try replicating the OutOfMemory error

[alkondrashov]

Have replicated it with the recent dta file. Here is why the exception is hapenning:

In essence, the numberOfEntries variable gets to 1818389564 and when we initialise array of ints with that size we get OutOfMemory error.

The exception is hapenning in edu.cornell.ncrn.ced2ar.stata.impl.DtaReader

[alkondrashov]

Based on stepping through the code, @spuddybike and I uncovered that OutOfMemory resulted from an incorrect value being read into numberOfEntries variable from a file.

[spuddybike]

@alkondrashov see https://www.loc.gov/preservation/digital/formats/fdd/fdd000471.shtml This I think might help in resolving the OOM error, as it indicates where the Number ofEntries is stored.

[spuddybike]

@alkondrashov see also https://www.stata.com/help.cgi?dta section 5

[alkondrashov]

As discussed, @spuddybike will provide another sample .dta file to test this error. We discovered that the issue is because the variables are being read incorrectly.

[spuddybike]

test-file-data-types-118.dta.zip
@alkondrashov file saved to Format 118 missing Stata for testing

[spuddybike]

Error created by SPSS export to Stata

[spuddybike]

SPSS Error