flowmatters / groovyfews Goto Github PK

Hello Joel,

Our Fews configuration is overcrowded with default values. I think I could help our maintainers a lot by giving them more flexibility and oversight by showing them which attributes are default or not. I made some sample code to show you my ides, but sofar I don't know yet how to achieve this in groovy/groovyfews. I am very curious if you recognize my/the problem.

Best regards, Rick

testing.groovy

filters(version:'1.1') {
        timeSeriesSets(id:'Set1') {
                timeSeriesSet() {
                        moduleInstanceId(SetsDefault)
                        valueType(Default)
                        parameterId('par1')
                        locationSetId('loc1')
                        timeSeriesType(Default)
                        timeStep(unit:'nonequidistant')
                        relativeViewPeriod(SetsDefault)
                        readWriteMode(Default)
                        synchLevel(Default)
                }
                timeSeriesSet() {
                        moduleInstanceId('ImportA2')
                        valueType(Default)
                        parameterId('par2')
                        locationSetId('loc2')
                        timeSeriesType(SetsDefault)
                        timeStep(Default)
                        relativeViewPeriod(Default)
                        readWriteMode(Default)
                        synchLevel(Default)
                }
                timeSeriesSet() {
                        moduleInstanceId(SetsDefault)
                        valueType(Default)
                        parameterId('par3')
                        locationSetId('loc3')
                        timeSeriesType(SetsDefault)
                        timeStep(SetsDefault)
                        relativeViewPeriod(Default)
                        readWriteMode(Default)
                        synchLevel(Default)
                }
    (....)
        timeSeriesSets(id:'Set2') {
                timeSeriesSet() {
                        moduleInstanceId(SetsDefault)
                        valueType(Default)
                        parameterId('par4')
                        locationSetId('loc4')
                        timeSeriesType(Default)
                        timeStep(unit:'nonequidistant')
                        relativeViewPeriod(SetsDefault)
                        readWriteMode(Default)
                        synchLevel(Default)
                }
    (...)

__TestingHelpers.groovy

static def Default(...) {
//      moduleInstanceId('ImportA') // always timeSeriesSets specific
        readWriteMode('read only')
        relativeViewPeriod('end:0','start:-14','unit:day')
        synchLevel('1')
        timeSeriesType('external historical')
        timeStep('multiplier:900','unit:second')
        valueType('scalar')
//      locationSetId(locationSetId) // locationSetId and parameterId are never default
//      parameterId(parameterId)
}
static def SetsDefault_Set1(....)
{
    delegate.timeSeriesSet() {
        moduleInstanceId('ImportA')
        readWriteMode(Default) // the same as in Default
        relativeViewPeriod(Default) // the same as in Default
        syncLevel(Default) // the same as in Default
        timeSeriesType('external forecasting')
        timeStep('multiplier:300','unit:second')
        valueType(Default) // the same as in Default
//      locationSetId(locationSetId) // locationSetId and parameterId are never default
//      parameterId(parameterId)
    }
}
static def SetsDefault_Set2(....)
{
    delegate.timeSeriesSet() {
        moduleInstanceId('ImportB')
        readWriteMode(Default) // the same as in Default
        relativeViewPeriod('end:0','start:-7','unit:day')
        syncLevel(Default) // the same as in Default
        timeSeriesType(Default) // the same as in Default
        timeStep('multiplier:60','unit:second')
        valueType(Default) // the same as in Default
//      locationSetId(locationSetId) // locationSetId and parameterId are never default
//      parameterId(parameterId)
    }
}

If a file is deleted/renamed after it has been processed in a compile, the old, out of date files should be deleted. eg if ImportA.groovy is compiled to XML and then renamed to ImportB.groovy, the compiled ImportA.xml should be deleted.

This should probably be by an option.

Switching to StreamingMarkupBuilder helps somewhat, but can't handle the really big files. In reality, any FEWS XML file this big will have been generated from some other source, so its probably best to grab that source and construct a short groovy script to transform it into XML.

Dear flowmatters,

I only recently came across your project but I am very excited about it. As a data analyst I am working with hydrologists using deltares fews. They have big isues in maintaining the DF configuration and are asking my assistence. But it is very difficult for me to better organize the configuration data. To me it looks very similar to people organizing their data in a bunch of interconnected spreadsheets. It can be nice and workable for a while, but when the tree starts growing and gets branches and subtrees all over the place it becomes a mess and unworkable. So what I did was put all the xml data into a relational database. With this database I can make queries on the configuration and find out where data probably is redundant and/or incoherent. But now my problem is how to communicate with my hydrologist coworkers about it and how to avoid making them something only I myself can maintain. And it looks like groovyfews is exactly the missing link I was looking for.

I installed groovy and groovyfews and followed the manual on the website of flowmatters. The examples you show there I could reproduce well. I am now looking into the more magic stuff and looked into your 'GroovyFewsHelpers.groovy' file. I recognize quite a few things which I am fiddling around in my own scripts already. So I did some testing if and how I can use them. But sofar I did not succeed. I focussed on your functions 'loadTable' and 'csvTable' in 'GroovyFewsHelpers.groovy' and this example code on your website flowmatters:

So for the moment I am stuck. I probably have to start learning the groovy language first before carrying on. But it would be great if you could help me on a head start to get your GroovyFewsHelper working for me. As far as I red the docs, you don't discribe the way to do this.

Thanks for your great work and best regards,

Rick / Waterbug

Hello Joel / Flowmatters,

Okay another issue which I am not sure whetter it is a real issue or my lacking knowledge of groovy/groovyfews. Please bear with me.

As far as I can see the problem is focussing on "createNode(element,argselem)". I made a testing1 function for it in Try3.

Hope you can help again,
Rick / waterbug

testing2.xml

import static GroovyFewsHelpers.*
import groovy.xml.StreamingMarkupBuilder
import groovy.xml.XmlUtil
def outputFile = new File(args[0])
def writer = outputFile.newWriter()
def xmlBuilder = new StreamingMarkupBuilder()
xmlBuilder.useDoubleQuotes=true
//tryA: no 'import __Helpers'
//tryB
//import __Helpers
//tryC
//import static __Helpers.fromArgsOrDefault
//import static __Helpers.tsDefaults
//import static __Helpers.testing1
//tryD
//import static __Helpers.*
def builderResult = xmlBuilder.bind {
// Dummy content
someNode(label:"Hello") {
childNode("Hello Again")
}
//args=[valueType:'scalar',timeSeriesType:'external historical',readWriteMode:'add originals',synchLevel:1]
//try0: works with tryA. Maar alleen als je andere functies in __Helpers weglaat.
//__Helpers.timeSeriesSetX(delegate,'testing')
//try1 (with both tryA and tryB): not working, errors
//__Helpers.fromArgsOrDefault('locationSetId','timeSeriesSet',args)
//try2 (with both tryC and tryD): not working, errors
//fromArgsOrDefault('locationSetId','timeSeriesSet',args)
//try3: It looks like this is part of the problem
// Error is barking about 'tsDefaults' which is strange, because not called by function 'testing1'
// If I remove function 'fromArgsOrDefault' from __Helpers I get another error
//(with tryA and tryB)
Helpers.testing1('locationSetId','something')
//(with tryC and tryD)
//testing1('locationSetId','something')
}
writer << XmlUtil.serialize(builderResult)
writer.close()

Helpers.groovy

static def timeSeriesSetX(delegate,locationSetId)
{
delegate.timeSeriesSet() {
locationSetId(locationSetId)
}
}
static def tsDefaults=[valueType:'scalar',timeSeriesType:'external historical',readWriteMode:'add originals',synchLevel:1]
static def fromArgsOrDefault(element,parent,args)
{
if(args[element])
{
parent.nodeCompleted(parent,parent.createNode(element,args[element]))
}
else if(tsDefaults[element])
{
parent.nodeCompleted(parent,parent.createNode(element,tsDefaults[element]))
}
}
static def testing1(element,argselem)
{
createNode(element,argselem)
}

Errors Try1

groovy gfews.groovy compile ../tmp6 ../out6
*** Error compiling groovy code
ERROR: startup failed:
/full/path/name/tmp6/__Helpers.groovy: 14: Apparent variable 'tsDefaults' was found in a static scope but doesn't refer to a local variable, static field or class. Possible causes:
You attempted to reference a variable in the binding or an instance variable from a static context.
You misspelled a classname or statically imported field. Please check the spelling.
You attempted to use a method 'tsDefaults' but left out brackets in a place not allowed by the grammar.
@ line 14, column 11.
else if(tsDefaults[element])
^
/full/path/name/tmp6/__Helpers.groovy: 16: Apparent variable 'tsDefaults' was found in a static scope but doesn't refer to a local variable, static field or class. Possible causes:
You attempted to reference a variable in the binding or an instance variable from a static context.
You misspelled a classname or statically imported field. Please check the spelling.
You attempted to use a method 'tsDefaults' but left out brackets in a place not allowed by the grammar.
@ line 16, column 59.
ent,parent.createNode(element,tsDefaults

Errors Try2

groovy gfews.groovy compile ../tmp6 ../out6
*** Error building ../out6/testing.xml from ../tmp6/testing.groovy
ERROR: Exception evaluating property 'locationSetId' for java.util.Arrays$ArrayList, Reason: groovy.lang.MissingPropertyException: No such property: locationSetId for class: java.lang.String (org.codehaus.groovy.runtime.HandleMetaClass@1bc339e[groovy.lang.MetaClassImpl@1bc339e[class groovy.lang.MissingPropertyException]])
LOCATION:
Caught: groovy.lang.MissingPropertyException: No such property: stackTraceDepth for class: groovy.lang.MissingPropertyException
groovy.lang.MissingPropertyException: No such property: stackTraceDepth for class: groovy.lang.MissingPropertyException
at Actions$__clinit__closure4.doCall(Actions.groovy:86)
at gfews$_processTree_closure3.doCall(gfews.groovy:193)
at gfews.processTree(gfews.groovy:158)
at gfews$processTree.callStatic(Unknown Source)
at gfews.compile(gfews.groovy:67)
at gfews$compile.callStatic(Unknown Source)
at gfews$__clinit__closure4.doCall(gfews.groovy:23)
at gfews.main(gfews.groovy:12)