dkpro / dkpro-lab Goto Github PK

Upgrade to Groovy 2.2.2. Groovy 1.8.4 is no longer supported in the Groovy 
Eclipse plugins.

In some cases it would be much more time and space efficient to import data 
into a task context using symlinks.

Two use-cases:

1) I need to import read-only data from an external (static) location on the 
file system
2) I need to import data read-only from another task, but I am generating new 
data that must be in the same directory. Thus the imported data can be a 
symlink in the directory and I can write my new data to a new file. That means, 
the directory structure must be copied completely to the task context, but the 
files themselves can be symlinks.

Several improvements have been discussed among heavy users

a) task.addImportLatest() is not a very intuitive name, maybe addDependency() / 
importRessource() ... not convinced I found the best name yet, but you get the 
idea

b) task.addImportLatest() should have the other task as first parameter

c) task.addImportLatest() should allow to omit the name of the key from the 
target task and use the name of the key from the imported task instead

Prepare deployment to Maven Central

The BatchTask internally calls Lab.getInstance() which gets a default instance 
if a custom context configuration is used. This is not what we want.

For complicated setup, the output of FlexTable can get quite large.
It would be nice to have the option to automatically omit constant rows in 
order to focus attention on the variable parts.
However, this should be an option one needs to explicitly activate. 

What steps will reproduce the problem?
1. try to obtain an Instance of the Lab with version 0.7.0

What is the expected output? What do you see instead?
Should work, worked before, but instead i get the stack trace

Exception in thread "main" 
org.springframework.beans.factory.BeanCreationException: Error creating bean 
with name 'TaskExecutionService' defined in class path resource 
[de/tudarmstadt/ukp/dkpro/lab//META-INF/spring/context.xml]: Initialization of 
bean failed; nested exception is java.lang.NoSuchFieldError: NULL
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:527)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:895)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:425)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:197)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:172)
    at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:158)
    at de.tudarmstadt.ukp.dkpro.lab.Lab.newInstance(Lab.java:61)
    at de.tudarmstadt.ukp.dkpro.lab.Lab.getInstance(Lab.java:53)
    at de.tudarmstadt.ukp.loewe.tp23.flawprediction.evaluation.WekaEvaluation.main(WekaEvaluation.java:262)
Caused by: java.lang.NoSuchFieldError: NULL
    at org.springframework.expression.TypedValue.<clinit>(TypedValue.java:32)
    at org.springframework.expression.spel.support.StandardEvaluationContext.setRootObject(StandardEvaluationContext.java:85)
    at org.springframework.expression.spel.support.StandardEvaluationContext.<init>(StandardEvaluationContext.java:74)
    at org.springframework.context.expression.StandardBeanExpressionResolver.evaluate(StandardBeanExpressionResolver.java:124)
    at org.springframework.beans.factory.support.AbstractBeanFactory.evaluateBeanDefinitionString(AbstractBeanFactory.java:1299)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.evaluate(BeanDefinitionValueResolver.java:210)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:182)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1360)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1118)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)




What version of the product are you using? On what operating system?

version 0.7.0

Please provide any additional information below.

Upgrade to uimaFIT 2.0.0

When trying to create a dimension bundle with a list of bundles
List<Map<String, Object>> listOfBundles = new ArrayList<Map<String,Object>>();
and calling toArray() on the list
Dimension.createBundle(NAME, listOfBundles.toArray());
an Object[] array is created causing the following exception:
Exception in thread "main" java.lang.ClassCastException: java.util.HashMap 
cannot be cast to java.lang.String
       at de.tudarmstadt.ukp.dkpro.lab.task.impl.DimensionBundle.<init>(DimensionBundle.java:54)
       at de.tudarmstadt.ukp.dkpro.lab.task.Dimension.createBundle(Dimension.java:103)

The lab could check whether the Object[] actually contains the expected 
Map<String, Object> and convert accordingly.

TaskContext supports a "message" method, but it should also support an "error" 
method to conveniently log errors on a more severe log level.

DKPro Lab depends on Batik which drags in an ancient version of Xalan (2.6.0). 
DKPro Lab should explicitly declare to use Xalan 2.7.1 in those modules where 
batik is a direct dependency.

This should avoid https://issues.apache.org/jira/browse/UIMA-2155 appearing for 
Lab users.

This enhancement is for experiments where dimensions are not independent. 
Consider the case that we want to create a bundled dimension and one parameter 
has a huge number of possible combinations. The other parameters are not 
affected by the additional parameters.
E.g.:
Dimension.createBundle("ranker",
new Object[] {
"rankerClass",  Ranker0},
new Object[] {
"rankerClass",  Ranker1,
"rankerParams",  Param4Ranker1});
There are two rankers but only Ranker1 needs parameters. If there are many 
possible parameters, they have to be added manually.

Proposed solution:
Param4Ranker1 could be an dimension inside this bundle. It could also be a list 
or array of objects that create a parameter space for Ranker1 only.

Currently the DefaultLifeCycleManager deletes failed tasks in the fail() 
method. This can be bad for debugging tasks. It should be possible to turn this 
of via a parameter.

When a task class is an inner class, the type name becomes quite long because 
it consists of the names of both, the outer and the inner class, e.g. 
"OuterClass$InnerClass". Only the name of the inner class should be used.

Upgrade examples to uimaFIT 2.0.0, DKPro Core 1.6.0 and ClearTK 1.4.1.

It is useful to aggregate tasks into a batch task and use imports, even if no 
parameter dimensions are specified. Currently, if no dimensions are specified, 
the BatchTask does not run its child-tasks at all - it should run them at  once.

There are no setters for dimensions and constraints, which would be required to 
use the "[] as X" syntax in Groovy.

Currently all task imports are on keys.
Sometimes (e.g. for a task that checks whether the configuration is valid) you 
only want to ensure that it is executed first.
Now, a dummy key is needed.
It would be good to have imports just on tasks that enforce a certain ordering 
of tasks without the need for keys.

Having the file name printed along with task progress (instead of 
"[ExtractFeaturesTask...] Progress [1456/1600] file") would be useful for 
debugging. Would this be possible?

It seems that for some data the Excel writer does not produce the full output. 
There was one case where the Excel file produced contained only two columns, 
while a full file with many more columns could be produced with the CSV writer.

It is not possible to create a Task class in groovy because get/setProperty 
have a special meaning in Groovy. Better rename them to get/setAttribute.

In case of a failed report, the DefaultLifeCycleManager catches the exception 
in the complete-method and reports a message to the TaskContext.
This message is on the Info-Level, even though it is pretty severe.

Also, one might wonder whether the exception should be eaten the way it is now. 
The lab keeps on running even though a report failed.

I suggest to either raise the log level or even throw an Exception (the latter 
is currently commented out)


Affected class: de.tudarmstadt.ukp.dkpro.lab.engine.impl.DefaultLifeCycleManager

Lines: 96-100
Version: 0.9.0-SNAPSHOT, revision 6902fd82de8a

Currently, a single-threaded UIMA engine (simple) and a multi-threaded UIMA 
engine based on the CPE. It would be nice to try if/how UIMA-AS can be used as 
well.

This works: 

Map<String, Object> dimReaders = new HashMap<String, Object>();
dimReaders.put(DIM_READER_TRAIN, TwentyNewsgroupsCorpusReader.class);
dimReaders.put(DIM_READER_TRAIN_PARAMS,
   Arrays.asList(new Object[] {...,
   TwentyNewsgroupsCorpusReader.PARAM_PATTERNS, new String[] { INCLUDE_PREFIX + "*.xml", INCLUDE_PREFIX + "*.xml.gz" } }));
Dimension.createBundle("readers", dimReaders);

This doesn't work:

Map<String, Object> dimReaders = new HashMap<String, Object>();
dimReaders.put(DIM_READER_TRAIN, TwentyNewsgroupsCorpusReader.class);
dimReaders.put(DIM_READER_TRAIN_PARAMS,
   Arrays.asList(new Object[] {...,
   TwentyNewsgroupsCorpusReader.PARAM_PATTERNS, new String[] { INCLUDE_PREFIX + "*/*.txt" } }));
Dimension.createBundle("readers", dimReaders);

To reproduce: go to 
de.tudarmstadt.ukp.dkpro.tc.examples.single.document.TwentyNewsgroupsDemo.java 
in the de.tudarmstadt.ukp.dkpro.tc.examples-gpl module of DKPro-TC, and add 
"new String[]" to the TwentyNewsgroupsCorpusReader.PARAM_PATTERNS parameter of 
the readers. Given the misuse of the PATTERNS parameter, this issue not be a 
big deal, but I'm keeping it here for the record.

The lifecycle fail does not take the reason for the failure. The lab throws the 
causing exception all the way up, but it might be missed by the casual user, 
e.g. because it ends up in the JUnit view in Eclipse and never gets logged to 
the console.

Use same Spring version as uimaFIT

Task.addImport() should also support importing folders from the local file 
system using the same mechanism that is used to import folders from other 
contexts.

Currently, only simple URLs are supported which support openStream().

A task cannot execute if not all of its parameters are provided by the 
parameter space. It would be helpful if the Lab could check before running a 
task or batch task if all parameters are always provided for every task - kind 
of a dry-run before actually running the whole stuff. That would avoid having 
users re-run everything in case a parameter was missing.

Since the cardinality of the dimensions is fixed when a BatchTask is invoked, 
it should be possible to calculate in advance the number of processing tasks 
which will be run.  It would be useful if, each time a processing task is run, 
the BatchTask could output some logging information indicating how many tasks 
have already been run and how many tasks there are in total.  This will help 
the user gauge the progress of the BatchTask.  Currently I think the only way 
of finding the total number of tasks is to manually count the cardinality of 
each dimension and multiply them together; there's no way of knowing how many 
tasks have already been executed except by manually examining the logging 
output and/or the repository directory.

Perhaps this information could be printed along with or right after the "== 
Running new configuration […] ==" logging message.

When outputting a report, some writers (e.g., XLS, TWiki) will suppress display 
of columns whose value does not change, and some of them (e.g., CSV) don't.  
This is bad for two reasons:

1) When outputting a report in multiple formats, they will be inconsistent in 
terms of which columns are displayed.

2) Data the user is expecting to appear in the table doesn't appear, and 
there's no explanation as to why.

Please include all columns in all report formats.

Attached find a minimal example reproducing the problem.  The column "bar" is 
missing from the output from some of the reports but not others.

A cross validation scenario can currently not be modeled with the 
implementations of tasks, dimensions and parameterspace. Several features are 
missing such as

- support for folds
- support for nested batch tasks
- probably more...

In DKPro TC we stumbled upon a problem with the maximum length of cells in 
Excel:

Getting the following exception while creating report:
     [java] Caused by: java.lang.IllegalArgumentException: The maximum length of cell contents (text) is 32,767 characters
     [java]     at org.apache.poi.hssf.usermodel.HSSFCell.setCellValue(HSSFCell.java:546)
     [java]     at org.apache.poi.hssf.usermodel.HSSFCell.setCellValue(HSSFCell.java:520)
     [java]     at de.tudarmstadt.ukp.dkpro.lab.reporting.FlexTable$5.write(FlexTable.java:484)
     [java]     at de.tudarmstadt.ukp.dkpro.lab.storage.filesystem.FileSystemStorageService.storeBinary(FileSystemStorageService.java:221)


(see issue http://code.google.com/p/dkpro-tc/issues/detail?id=53)

We propose to trim the size of cells to 32,767 characters.

In nested BatchTasks, where a Task in InnerBatchTask depends on imports from 
one of the the Tasks in the OuterBatchTask: if InnerBatchTask is run first, an 
ExecutionException is thrown causing the program to stop instead of trying to 
run OuterBatchTask first.

What steps will reproduce the problem?

1.Run the lab task many times to have lots of data (around 10GB, about 400,000 
items) in the dkpro repository

What is the expected output? What do you see instead?

Running the task gives an error of "java.lang.NumberFormatException: null"
in the batch task, in reading the task context metadata ("line 133 in 
TaskContextMetadata.java"). Very confusing one and hard to debug. Deleting the 
data from the dkpro repository fixes the problem. Probably the error message 
shall just suggest that the DkPro folder is full?


What version of the product are you using? On what operating system?

dkpro lab 0.8.0 on Ubuntu 12.04

Please provide any additional information below.

What steps will reproduce the problem?
1. Run a batch task where (a) one of the dimensions, D, has only a single 
value, and (b) the report uses an XLS or TWiki writer.

What is the expected output? What do you see instead?
Since setCompact(false) was not used, I expect to see a proper XLS or TWiki 
table which simply omits the column for D.  Instead the entire table is 
garbled; almost all the information is missing.  See attached for an example.

What version of the product are you using? On what operating system?
0.8.0 on GNU/Linux

Please provide any additional information below.
The problem does not occur with non-compact tables (i.e., with 
setCompact(false)), nor does it occur with the CSV writer.

What steps will reproduce the problem?
1. use a map or an empty array as parameter in a BatchTask
2.
3.

What is the expected output? What do you see instead?
I expect it to actually use that, instead it gives a PatternSyntaxException. 
This is due to the constraints being checked through Regular expressions in 
ImportUtil.matchConstraints. For objects that use [] or {} in their string 
representation, this is a problem.

What version of the product are you using? On what operating system?


Please provide any additional information below.

A workaround is to use an overriden map that manually escapes { and } in it's 
string representation. A fix would therefore be to just escape special 
characters before converting the String in a regular expression.

TaskContextMetadata writes some properties. It would be nice if the names of 
these properties were accessible in some way, e.g. as constants.

If the task dependencies are not satisfied it would be nice if some pointer to 
the reason could be given, i.e. what dependency could not be satisfied or why 
there is cycle in the graph, etc.

What steps will reproduce the problem?
1. Create a Constraint such that isValid() always returns false for the given 
configurations.  Trivially: public boolean isValid(Map<String, Object> 
aConfiguration) { return false; }
2. Attach this constraint to a ParameterSpace.
3. Attach the ParameterSpace to a BatchTask with a report.
4. Run the BatchTask.

What is the expected output? What do you see instead?

See the attached file dkprolab.log for example logging output.  The error 
message rather cryptically complains about a missing file.  Instead it would 
have been helpful for there to be some message that no configurations passed 
the constraint.


What version of the product are you using? On what operating system?
0.8.0 on GNU/Linux

Please provide any additional information below.

I will try producing and submitting a minimal example later.

It seems the USE_EXISTING policy uses the first found task execution that 
matches the current parameter configuration. It should use the newest execution 
instead.

A user reports an NPE during the cleaning up of a failed task which eventually 
hides the actual exception that caused the task to fail.

2012-03-26 10:31:46  INFO [main] (BatchTask) - Executing task 
[LabExperiment$ProcessingTask]
2012-03-26 10:31:47  INFO [main] (DefaultLoggingService) 
-[LabExperiment$ProcessingTask-1ffbdf7b-771e-11e1-a3b6-935d04d42ca0] Task 
failed [LabExperiment$ProcessingTask] 
2012-03-26 10:31:47  INFO [main] (DefaultLoggingService) 
-[LabExperiment$ProcessingTask-1ffbdf7b-771e-11e1-a3b6-935d04d42ca0] Shut down 
task
2012-03-26 10:31:47  INFO [main] (DefaultLoggingService) 
-[BatchTask-7d2f55e5-7718-11e1-a3b6-935d04d42ca0] Shut down task
Exception in thread "main" java.lang.NullPointerException
    at de.tudarmstadt.ukp.dkpro.lab.Util.delete(Util.java:203)
    at de.tudarmstadt.ukp.dkpro.lab.storage.filesystem.FileSystemStorageService.delete(FileSystemStorageService.java:76)
    at de.tudarmstadt.ukp.dkpro.lab.engine.impl.DefaultLifeCycleManager.fail(DefaultLifeCycleManager.java:109)
    at de.tudarmstadt.ukp.dkpro.lab.engine.impl.ExecutableTaskEngine.run(ExecutableTaskEngine.java:57)
    at de.tudarmstadt.ukp.dkpro.lab.engine.impl.DefaultTaskExecutionService.run(DefaultTaskExecutionService.java:48)
    at de.tudarmstadt.ukp.dkpro.lab.Lab.run(Lab.java:96)
    at LabExperiment.main(LabExperiment.java:83)

This should not work (task1 depends on task2 and vice versa).

Task task1 = new ExecutableTaskBase()
        {
            @Override
            public void execute(TaskContext aContext)
                throws Exception
            {
                // Nothing to do
            }
        };

        Task task2 = new ExecutableTaskBase()
        {
            @Override
            public void execute(TaskContext aContext)
                throws Exception
            {
                // Nothing to do
            }
        };

        task2.addImport(task1, "DUMMY");
        task1.addImport(task2, "DUMMY");

        BatchTask batchTask = new BatchTask();
        batchTask.addTask(task1);
        batchTask.addTask(task2);

        Lab.getInstance().run(batchTask);

Add a parameter to the DefaultLifeCycleManager manager that controls if report 
failures should be fatal (throw an exception) or if they should only be logged. 
See also issue 22.

I think it might be more intuitive if multiple constraints set on a parameter 
space would be treated as a disjunction:

   * If no constraints are set, run everything
   * If constraints are set, run a configuration if any of them says "ok", default to not running.

That should make it much simpler to write constraints.

Upgrade to UIMAJ SDK 2.6.0

What steps will reproduce the problem?
1. Create a StringAdapter().
2. Call StringAdapter#getString()

What is the expected output? What do you see instead?
Expected output: null
Actual output: "UTF-8"

What version of the product are you using? On what operating system?
0.11.0-SNAPSHOT

Please provide any additional information below.
Fix: change StringAdapter(String aString) to set encoding instead of object 
(intended?)

It would be nice if the Lab was capable of capturing all the output produced by 
a task. This would probably mean that everything that goes to System.out and 
System.err must be captured. 

It seems DKPro Lab 0.10.0 was released some time ago (at least, there are 
publically available artifacts for it) but there was no release announcement.  
The Google Code wiki still says the latest version is 0.9.1.

What steps will reproduce the problem?
1. Subclass UimaTaskBase, overwrite getAnalysisEngineDescription and return 
only a primitive description
2. Run the subclass task
3. Task run failed with CpeDescriptorException: The object reference 
casProcessor name=NULL is invalid.
(See the attached file)

Currently if a AnalysisEngineDescription is primitive, CpeBuilder will try to 
create a processor with key ""; afterwards when it comes to 
CasProcessorCpeObject, an exception will be thrown if the key is null or its 
length is 0.

I think CpeBuilder should create the processor with a non-empty key.


Partial Stacktrace:
org.apache.uima.collection.metadata.CpeDescriptorException: The object 
reference casProcessor name=NULL is invalid.  (Thread Name: main) 
    at org.apache.uima.collection.impl.metadata.cpe.CasProcessorCpeObject.setName(CasProcessorCpeObject.java:348)
    at org.apache.uima.collection.impl.metadata.cpe.CpeDescriptorFactory.produceCasProcessor(CpeDescriptorFactory.java:316)
    at de.tudarmstadt.ukp.dkpro.lab.uima.engine.cpe.CpeBuilder.createProcessor(CpeBuilder.java:168)
    at de.tudarmstadt.ukp.dkpro.lab.uima.engine.cpe.CpeBuilder.setAnalysisEngine(CpeBuilder.java:98)

DKPro Lab 0.9.0 Groovy module doesn't contain any compiled classes