madgik / exareme Goto Github PK

I am creating some tables whose data are imported from tsv files.

In my case, there are 3 files, where the two of them are sizing from several kilobytes to a few megabytes (about 800K to 4M) but the third one is more than 2GB. The table that uses this particular file, seems to take too much time to be created and I get an error such as "Operator EXAQUERY: timed out."

Because I wanted to experiment, I have cut the large file into some smaller ones in order to see the time needed for loading:

succeeded | time for loading | size of file
yes | 5 sec 222 msec. | 36,840 bytes (10 lines)
yes | 1 min. 34 sec 952 msec. | 36,501,198 bytes (100,000 lines)
no | timeout | 359,690,510 bytes (1,000,000 lines)

Is it possible to set a different timeout in order to load data from such big (or bigger) files? Is it the SoTimeout field of the socket configuration that could do the trick?

Hi,

I came across this project which uses titus package which is not supported in Python 3.
With the sunset of Python 2 by the end of this year, I am glad to inform that Titus 2 is fully supported in Python 3.4-3.8.
pip install titus2
All syntax and functionality remain the same and the code requires no modification.
For more details please head to this repo - https://github.com/animator/titus2
The migrated code is passing all unit tests and conformance tests.
Please don't forget to star the project in case you find it useful.

Cheers,
Ankit

Hello,

After building the project with 'mvn package', I get those test failures:

Running madgik.exareme.utils.combinatorics.FactorialTest
complete
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec <<< FAILURE!
testComplete(madgik.exareme.utils.combinatorics.FactorialTest) Time elapsed: 0 sec <<< ERROR!
java.lang.ArrayIndexOutOfBoundsException: 0
at madgik.exareme.utils.combinatorics.Factorial.complete(Factorial.java:19)
at madgik.exareme.utils.combinatorics.FactorialTest.testComplete(FactorialTest.java:26)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Running madgik.exareme.utils.combinatorics.ChooseTest
complete
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0 sec <<< FAILURE!
testComplete(madgik.exareme.utils.combinatorics.ChooseTest) Time elapsed: 0 sec <<< ERROR!
java.lang.ArrayIndexOutOfBoundsException: 0
at madgik.exareme.utils.combinatorics.Choose.complete(Choose.java:20)
at madgik.exareme.utils.combinatorics.ChooseTest.testComplete(ChooseTest.java:26)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

Running madgik.exareme.utils.combinatorics.PermutationsTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec <<< FAILURE!
testSomeMethod(madgik.exareme.utils.combinatorics.PermutationsTest) Time elapsed: 0.001 sec <<< FAILURE!
junit.framework.AssertionFailedError: The test case is a prototype.
at junit.framework.Assert.fail(Assert.java:50)
at madgik.exareme.utils.combinatorics.PermutationsTest.testSomeMethod(PermutationsTest.java:19)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:243)
at junit.framework.TestSuite.run(TestSuite.java:238)
at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

[...]

Results :

Failed tests: warning(junit.framework.TestSuite$1): No tests found in madgik.exareme.utils.embedded.db.SQLDatabaseTest
testSomeMethod(madgik.exareme.utils.combinatorics.PermutationsTest): The test case is a prototype.

Tests in error:
testNextLine(madgik.exareme.utils.iterator.ReversibleScannerTest)
testClose(madgik.exareme.utils.iterator.ReversibleScannerTest)
testHasNext(madgik.exareme.utils.iterator.ReversibleScannerTest)
testRollBack(madgik.exareme.utils.iterator.ReversibleScannerTest)
testProccess(madgik.exareme.utils.embedded.process.MadisProcessTest)
testEmbeddedJDBC(madgik.exareme.utils.embedded.ExaremeEmbeddedJDBCTest)
testGetNext(madgik.exareme.utils.combinatorics.ConstantSumGeneratorTest)
testComplete(madgik.exareme.utils.combinatorics.FactorialTest): 0
testComplete(madgik.exareme.utils.combinatorics.ChooseTest): 0
testEquals(madgik.exareme.utils.check.CheckTest)
testTrue_boolean_String(madgik.exareme.utils.check.CheckTest)
testNotNull_Object(madgik.exareme.utils.check.CheckTest): Object should be not null
testNotNull_Object_String(madgik.exareme.utils.check.CheckTest)
testTrue_boolean(madgik.exareme.utils.check.CheckTest): Condition should be true

Tests run: 52, Failures: 2, Errors: 14, Skipped: 0

Travis build-check, shown in "README.md" is happening for the older (removed) branch "mip".

This, along with checks from other branches are failing.
More info can be found here:
https://travis-ci.org/madgik/exareme/branches

I tried to skip the tests, but the project still doesn't compile

[INFO] exareme-master ..................................... FAILURE [ 1.923 s]
[INFO] exareme-distribution ............................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.692 s
[INFO] Finished at: 2015-10-15T16:32:46+02:00
[INFO] Final Memory: 32M/420M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project exareme-master: Could not resolve dependencies for project madgik:exareme-master:jar:0.1-SNAPSHOT: Could not find artifact madgik:exareme-worker:jar:tests:0.1-SNAPSHOT -> [Help 1]
[ERROR]

HTTP POST on /mining/query/WP_LINEAR_REGRESSION using an invalid body e.g. an empty array '[]' returns http code 200 and a stacktrace (see below) in the response body. This behaviour is wrong.

java.rmi.ServerException: ; nested exception is:
java.rmi.ServerException: Cannot execute madis; nested exception is:
java.rmi.ServerException: Cannot run query; nested exception is:
java.rmi.ServerException: Cannot execute db (code: 1): Madis SQLError:
Operator EXECNSELECT: Error in query no. 6 - list index out of range

You should return an http error code (instead of 200) !