equinor / ert Goto Github PK

Workflows should be able to print to configurable stdin/stdout and have a logging system

https://github.com/Statoil/ert/blob/1f304612b3a5071aff974529340aabe07c80bdb3/python/python/ert_gui/simulation/models/base_run_model.py#L219-L224

This function raises exception when running single realization test run. When it is fetching ensemble size it gets the number realizations defined in config when it should be one.

There are many ValueErrors when creating plots.

The following procedure results in ValueError exceptions:

1. Start gui_test
2. press Start Simulation button (single realization test run)
3. select Yes
4. Wait until simulation is completed
5. press Create Plot button

As well a printing a curve, multiple ValueError messages are printed. Also selecting different plots results in the same curve( FGOR plot = TIME plot):

Traceback (most recent call last):
File "/private/stefos/ert/ert/build/lib/python2.7/site-packages/ert_gui/tools/plot/data_type_keys_list_model.py", line 46, in data
if self.keyManager().isKeyWithObservations(item):
File "/private/stefos/ert/install/lib/python2.7/site-packages/res/enkf/key_manager.py", line 157, in isKeyWithObservations
return key in self.allDataTypeKeysWithObservations()
File "/private/stefos/ert/install/lib/python2.7/site-packages/res/enkf/key_manager.py", line 151, in allDataTypeKeysWithObservations
self.__all_keys_with_observations = self.summaryKeysWithObservations() + self.genDataKeysWithObservations()
File "/private/stefos/ert/install/lib/python2.7/site-packages/res/enkf/key_manager.py", line 115, in genDataKeysWithObservations
report_step = obs_vector.activeStep()
File "/private/stefos/ert/install/lib/python2.7/site-packages/res/enkf/observations/obs_vector.py", line 113, in activeStep
raise ValueError("The activeStep() method can ONLY be called for obervations with one active step")

I see you have removed "View/Advanced Options" from the main ERT menu.

It is not clear to me how you now can select a loaded analysis module for Iterated Ensemble Smoother which previously appeared on the Simulation mode button when Advanced options were set.

Either dump the failed the realizations to a file/in terminal on the format the GUI expects to restart those exact realizations, or have the GUI itself support to relaunch all failed realizations from the previous run.

1. Document the new libecl function added here: equinor/resdata#487 and equinor/resdata#488

2. Fix: #151

3. Fix: equinor/libres#389

4. RMS version 2013.1.4 does not work with ert + komodo/testing. Think is really a komodo env problem and not a ert problem.

The REFCASE keyword should go away.

Should support that parameter values are strings (not only numbers). Example 50% probability for value “depth_conv1”, 25% probability for “depth_conv2” and 25% probability for “depth_conv3”.

We should setup some internal testing on Jenkins for the deployed versions of ERT. I suggest that we start by setting up a Jenkins job that runs all the cases from the FMU tutorial every night on stable. After this, we should do the same for testing, unstable and bleeding.

One of these applications are needed in order to start the ert server that will allow sending batch simulation requests to ert. These applications seem to be missing form the current deployment.
Also it would be great if these will not be linked to ert_gui.
I was told that this might not have too large priority but I want to create the issue just in case... so you know that there is at least somebody that thinks they are useful.

Users request improved plotting capabilities that would facilitate more advanced data analysis after run.

ECL_BASE should be removed. Use JOBNAME instead.

Preparation to use YAML syntax for observation files.

Bug in line 58 here: https://github.com/Statoil/ert/blob/master/python/python/ert_gui/tools/run_analysis/run_analysis_tool.py should create a RunContext.

gert_main should be turned into a single executable and all qt gui modifications should be moved to main_window

General case
We could consider utilizing feature toggling for breaking changes to avoid breaking changes from one release to another. Instead we could follow a schema like this through 4 releases:

• Old behavior
• Old behavior is default, new behavior can be activated. Warning if using default informing that this will change in the future.
• New behavior is default; warning if old behavior is utilized
• only new behavior

In this manner we would never introduce hard, braking changes. However, such changes (over time) are necessary to clean up the code base.

Example
There was a change in behavior with respect to timesteps in libres in equinor/libres#439. This will affect users. It would be rather easy to follow the schema above in this scenario.

It would be useful to be able to edit observations and generate new observations.

Some of the files in ert/share also live in libres/share.

Support for QC on forward models to ensure that the model is "compliant" before continuing to simulate or load results.

I would like a json dump of the entire ert config file in each realization directory, written upon realization initialization (same time as jobs.json is written).

Currently it is possible to set the iter number from the workflow RUN_SMOOTHER_WITH_ITER. It would be very good to have it accessible either as a workflow job "SET_ITER" or "ENSEMBLE_RUN_WITH_ITER".

I will use it to test and screen different observations sets and scalings without overwriting the runpath everytime (without explicitt chaning the runpath). I can also think of other applications, and think it will be useful for others.

Make it easier for the user to find the functionality for running update based on prior more easily available.

Adding back the possibility to only build the command line applications.

Building the GUI requires qt libraries for python among other ones and that makes it too heavy for instance for cases when the only thing you want is integration testing in a CI/CD workflow.
Also there are already other ert applications that one might want to use in a command line on a daily basis.
Using ert in a batch mode is useful in workflows that do not require user interaction.

Setting the variable BPR in refcase as both with indices (h,k,l,) and global index (n)causes error when trying to plot BPR:n.

Bug reported on yammer.

ERT should support latin hypercube sampling.

Trying ert with latest libres/libecl gives the following exceptions

Traceback (most recent call last):
  File "$KOMODO/ert_gui/tools/plot/data_type_keys_list_model.py", line 46, in data
    if self.keyManager().isKeyWithObservations(item):
  File "$KOMODO/res/enkf/key_manager.py", line 157, in isKeyWithObservations
    return key in self.allDataTypeKeysWithObservations()
  File "$KOMODO/res/enkf/key_manager.py", line 151, in allDataTypeKeysWithObservations
    self.__all_keys_with_observations = self.summaryKeysWithObservations() + self.genDataKeysWithObservations()
  File "$KOMODO/res/enkf/key_manager.py", line 115, in genDataKeysWithObservations
    report_step = obs_vector.activeStep()
  File "$KOMODO/res/enkf/observations/obs_vector.py", line 113, in activeStep
    raise ValueError("The activeStep() method can *ONLY* be called for obervations with one active step")
ValueError: The activeStep() method can *ONLY* be called for obervations with one active step



--------------------------------------------------------------------------------
  File "$KOMODO/ert_gui/tools/plot/plot_widget.py", line 85, in updatePlot
    self._plotFunction(plot_context)
  File "$KOMODO/ert_gui/plottery/plots/ensemble.py", line 33, in plotEnsemble
    plotObservations(plot_context, axes)
  File "$KOMODO/ert_gui/plottery/plots/observations.py", line 12, in plotObservations
    observation_data = data_gatherer.gatherObservationData(ert, case_list[0], key)
  File "$KOMODO/ert_gui/plottery/plot_data_gatherer.py", line 52, in gatherObservationData
    return self._observationGatherFunction(ert, case, key)
  File "$KOMODO/ert_gui/plottery/plot_data_gatherer.py", line 126, in gatherSummaryObservationData
    if ert.getKeyManager().isKeyWithObservations(key):
  File "$KOMODO/res/enkf/key_manager.py", line 157, in isKeyWithObservations
    return key in self.allDataTypeKeysWithObservations()
  File "$KOMODO/res/enkf/key_manager.py", line 151, in allDataTypeKeysWithObservations
    self.__all_keys_with_observations = self.summaryKeysWithObservations() + self.genDataKeysWithObservations()
  File "$KOMODO/res/enkf/key_manager.py", line 115, in genDataKeysWithObservations
    report_step = obs_vector.activeStep()
  File "$KOMODO/res/enkf/observations/obs_vector.py", line 113, in activeStep
    raise ValueError("The activeStep() method can *ONLY* be called for obervations with one active step")
Exception type: ValueError

$KOMODO/ert_gui/tools/plot/plot_widget.py:92: DeprecationWarning:
     BaseException.message has been deprecated as of Python 2.6
  sys.stderr.write("%s\n" % e.message)
The activeStep() method can *ONLY* be called for obervations with one active step
--------------------------------------------------------------------------------
An error occurred during plotting. This stack trace is helpful for diagnosing the problem.

Traceback (most recent call last):
  File "$KOMODO/ert_gui/tools/plot/data_type_keys_list_model.py", line 46, in data
    if self.keyManager().isKeyWithObservations(item):
  File "$KOMODO/res/enkf/key_manager.py", line 157, in isKeyWithObservations
    return key in self.allDataTypeKeysWithObservations()
  File "$KOMODO/res/enkf/key_manager.py", line 151, in allDataTypeKeysWithObservations
    self.__all_keys_with_observations = self.summaryKeysWithObservations() + self.genDataKeysWithObservations()
  File "$KOMODO/res/enkf/key_manager.py", line 115, in genDataKeysWithObservations
    report_step = obs_vector.activeStep()
  File "$KOMODO/res/enkf/observations/obs_vector.py", line 113, in activeStep
    raise ValueError("The activeStep() method can *ONLY* be called for obervations with one active step")
ValueError: The activeStep() method can *ONLY* be called for obervations with one active step

User requests integrated support for design matrices in libres. Possibility to run the the prior from a design matrix, then the posterior based on this prior.

Number of cpu’s pr job should not be picked up from the Eclipse data deck as done today . Instead, there should be a separate keyword in the config file defining number of cpus pr job.

I have received reports (from user tskille) that the stable version of GERT is unable to stop jobs that are running locally. This can be easily be reproduced with simple test case such as Reek.

Here: https://github.com/Statoil/ert/blob/master/bin/ert.in#L8 the environment of the running ERT process is manipulated runtime.

This seems convenient at first - but it comes back to haunt you; at least the PYTHONPATH come in conflict with RMS and should be removed. Might induce other breakage ... :-(

Display information while running ERT similar to what is currently done in Everest. Should be possible to see:

• how many of the realizations have completed and their status,
• how far each running forward model has got, with time stamps for each the start and end of each job.

Libecl

1. Here is some documentation of libecl Python code.

2. Here here is some example code of creating api documentation with sphinx

ERT

In [this] (https://github.com/Statoil/ert/tree/master/docs) directory the rst/ folder is where the new documentation has been added, the other subfolders here contain various old documentation. Some of that is certainly valuable and should be reused in the new manual in rst/manual

In particular: eclipse data files should be treated as an other template file.

There are many aspects to this.

• Completely remove pseudo time for parameters,
• Remove dynamic behavior of GEN_DATA,
• For summary vectors; in most situations ignore the fact that they are time dependent and consider them a vector.

When the above is executed, the data type enkf_node can be oblivious to time. This would remove the last traces of the original enkf algorithm.

Another action would be to interpolate simulated data for the observed time steps. This implies that it is not necessary to have a globally decided set of time steps for all of the simulated data.

The GUI should in a natural manner allow the users to:

• evaluate the prior
• compare prior and observed values and based on this:
  • identify outliers,
  • activate and deactivate realisations,
  • activate and deactivate observations,
  • adjust the errors of the observations,
• find and evaluate multiple posteriors based on the different sets of "observation filters" created above
• compare the different posteriors

Support the possibility of specifying multiple jobs that would run independently in parallel.