eawag-appliedsystemanalysis / simstrat Goto Github PK

When running Simstrat through a batch system the progressbar is useless in its current form, it just produces tons of useless data in the output and fills up disk space.

The code needs an option to turn off the progress bar completely at the very least, but an option to specify its behavior would be better.
Like off/progress-report-interval-in-secs or something on those lines.

E.g. if the sign of the depths in the bathymetry is positive instead of negative, Simstrat crashes and writes that it cannot open the initial conditions file. As this error doesn't allow to trace back to the mistake, there have to be checks and warnings.

Implement the possibility of outputting averaged values (over given period) to reduce size of output files.

Pressure gradient option 1 causes diverging horizontal water velocities and thus all state variables become NaN's pretty quickly (independent of latitude).

When compiling simstrat2beta with gfortran!=6.3 the execution will result in a segmentation fault.

Add a zenith angle dependent albedo to Simstrat. Could for example be done analogous to GLM, where the following formula is implemented:
Albedo = 0.08 - 0.02 * sin(2*Pi * kDays/365 +- Pi/2);

For the northern hemisphere, the halfPi is subtracted and for the southern hemisphere halfPi is added.

If there's a single number x in the output depths file, Simstrat understands "write output every x meters". Should there be a possiblity to output a single depth out? For instance if the value is zero or negative? Can be useful for calibration purposes / when simstrat is run from a script.

If the wind drag model is not 1, some code versions of Simstrat seem to use the C10 parameter as a calibrating factor for the wind drag relation. In my opinion, this is a bit risky. I would rather have it used only as an absolute value for the case that the wind drag model is 1.
In the new version, C10 would be required in this case, and there would be a warning in case the wind drag model is not 1, saying that the parameter will be ignored.

I suggest to convert the bash script build/build.sh into python.
Important issue is to have it platform-independent!!

With that then the workflow to compile the software will be the same.

I've been following the steps in 'How to setup Simstrat building environment with Docker' to build a docker executable which should run for macOS (in my case Mojave). Running it inside the docker environment works fine, but the created executable can't be executed for macOS. The file type is

$ file simstrat 
simstrat: ELF 64-bit LSB pie executable x86-64, version 1 (SYSV), dynamically linked, with debug_info, not stripped

ELF (extensible and linkable format), whereas macOS would require Mach-OS for its binaries. Is there a workaround for the docker container to this?

Thanks!

I observer an elevated turbulent diffusivity close to the sediment that depends on the time when I start the simulation.
It seems that complete mixing in winter triggers a constant mixing at the bottom of the lake that does not level off anymore.
Will provide a preview of what I mean in a personal message.

When trying to compile Simstrat on Ubuntu 20.04, I get the error that Forbear is referring to an unavailable Hash ID 24d1747e8da40c91037d6e72445a573c85748af9.

(base) robert@wintergarten:~/Projects/AEMONJ/LER-WQ/Simstrat-AED/Simstrat$ ./git_lib_initialize.sh

Synchronizing submodule url for 'lib/forbear'

Synchronizing submodule url for 'lib/libaed2'

fatal: remote error: upload-pack: not our ref 24d1747e8da40c91037d6e72445a573c85748af9

fatal: the remote end hung up unexpectedly

Fetched in submodule path 'lib/forbear', but it did not contain 24d1747e8da40c91037d6e72445a573c85748af9. Direct fetching of that commit failed.

Checking the submodule online, this seems to be an invalid link. Could it be that the master branch has some uncommited changes to the submodule. I was just reading this post here, but I am not sure if the error is on my side or on Simstrat's github repo.

Thanks,
Robert

Function is to adapted to handle several inflows, possibly entering at different depths

Simstrat seems not to handle files with only "carriage return" \r and seems to require "carriage return + line feed" \r\n
Issue seen in v1.6: Simstrat sees the inflow/outflow files as empty and disables the advection calculation, resulting in totally ignoring inflow and outflows. No warning whatsoever!

From Stackoverflow:
\r = CR (Carriage Return) Used as a new line character in Mac OS before X
\n = LF (Line Feed) Used as a new line character in Unix/Mac OS X
\r\n = CR + LF Used as a new line character in Windows

Simstrat doesn't handle a list of specific output times correctly. I didn't track the bug to its full extent but certain configurations (at least: if the first date is the start date or if the end date is reached exactly) seem to end in an endless loop.

If the model is configured with a number of grid points larger than mxl (now fixed to 1000), the model crashes darkly while reading the initial conditions. Needs an error message!

In forcing_update there is a double read when forcing_model == 1

      ! Forcing mode 1 (date, U10, V10, T_lake, H_sol)
      if (cfg%forcing_mode == 1) then
         if (cfg%ice_model == 1) then
            call error('Ice module not compatible with forcing mode 1, use 2, 3 or 5.')
         end if

         call self%read (state%datum, A_s, A_e, A_cur, 4 + nval_offset, state%first_timestep)
         call self%read (state%datum, A_s, A_e, A_cur, 4 + nval_offset, state%first_timestep)
         state%u10 = A_cur(1)*param%f_wind ! MS 2014: added f_wind
         state%v10 = A_cur(2)*param%f_wind ! MS 2014: added f_wind

The equation used is described in Flerchinger 2009, but does not seem to have a correction for altitude.

Simstrat seems to crash (invalid memory reference) if the file containing outflow values has only two columns (one for time and one for outflow) and the outflow is given at z=0.

Instead of using depth relative to water level, it is beneficial in some situations to directly use altitude above sea level. This is relevant mainly when using calibration routines with pest for lakes with strong lake level variation.

Comments are not standard in json files and need to be removed for some applications (mainly automation). All the comments are anyways available in the manual.

The inflow bug is now corrected in Version 1.4, meaning that water in- and outflow at the lake surface now work correctly in steady state.

However, there is still a problem when the in- or outflow at the lake surface is not in steady state. For example, if the inflow is between 0 and 1 m of depth and total inflow is smaller than total outflow, then the water table will sink. In the current version of the code, this will simply cut a part of the inflow (as the inflow coordinate system is linked to the lake coordinate system and not to the lake water table) and thus the water table will sink even faster.

for WINDOWS developers... can you build and test the dev/progressbar feature?
The building should be smooth and easy using FoBiS as usual if you have Python 2.X, while it should not work with Python 3.X

I cannot test because I do not have a WIN building env...
thanks!

In v2.4 release there is a feature to start a simulation from a binary file. I am interested in applying Simstrat in a forecasting framework and would like to be able to utilise this feature to integrate into our data assimilation protocol. Can you provide instructions on how to use this feature? What exactly is stored within the binary file and if it is possible to extract and/or update the model states?

This feature has been implemented in version 1.4 but is missing in simstrat version 2.0

Hi!

It would be good to have some info on which compiler, version, and flags (in case fobos settings change in the future) the results in TestCases_Results_expected/ are created with. And exactly which version/commit of the code(s) where used to create them.
Preferably in a README in the tests folder.

The code seems vulnerable to compiler versions/flags and produces small to medium numerical differences according to tests.py

And the same info should be provided for libaed2 since that also affects the results as far as I can see.

Adding an output for total ice thickness = snow ice + black ice

• Allows direct comparison to observations (e.g., for calibration)
• Easier to understand for users, no need for post-processing

I'm currently trying to build a UNIX executable using the recent version 2.4 for macOS. Here I'm having several questions and issues regarding the new addition of the progress bar using forbear.
The fobos script in /build for release-gnu-dynamic doesn't include ../lib/forbear/fobos:shared-gnu((direct)) in contrast to release-gnu. I guess that this was done intentionally in the last commit, but then the compilation of Simstrat.f90 won't work as use forbear fails.

When adding that line to release-gnu-dynamic, I also changed the fobos in lib/forbear to include the option for template-shared-gnu which was deleted in the last commit. Still when trying to compile using FoBiS.py build -mode release-gnu-dynamic it fails. The error I get is related to the building of forbear (see below).

Would it be possible that I can change my code that it won't use the progress bar feature, or is forbear now an essential part?

Thanks!
Robert

Compilation error:
Building dependency fobos into ../lib/forbear with mode shared-gnu Traceback (most recent call last): File "/Users/robertladwig/anaconda3/bin/FoBiS.py", line 4, in <module> __import__('pkg_resources').run_script('FoBiS.py==2.9.3', 'FoBiS.py') File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 666, in run_script self.require(requires)[0].run_script(script_name, ns) File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1453, in run_script exec(script_code, namespace, namespace) File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/FoBiS.py-2.9.3-py3.7.egg/EGG-INFO/scripts/FoBiS.py", line 21, in <module> File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/FoBiS.py-2.9.3-py3.7.egg/fobis/fobis.py", line 45, in main File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/FoBiS.py-2.9.3-py3.7.egg/fobis/fobis.py", line 58, in run_fobis File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/FoBiS.py-2.9.3-py3.7.egg/fobis/FoBiSConfig.py", line 65, in __init__ File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/FoBiS.py-2.9.3-py3.7.egg/fobis/Fobos.py", line 76, in __init__ File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/FoBiS.py-2.9.3-py3.7.egg/fobis/Fobos.py", line 238, in _set_cliargs File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/FoBiS.py-2.9.3-py3.7.egg/fobis/Fobos.py", line 176, in _check_local_variables File "/Users/robertladwig/anaconda3/lib/python3.7/site-packages/FoBiS.py-2.9.3-py3.7.egg/fobis/Fobos.py", line 144, in _get_local_variables File "/Users/robertladwig/anaconda3/lib/python3.7/configparser.py", line 857, in items return [(option, value_getter(option)) for option in d.keys()] File "/Users/robertladwig/anaconda3/lib/python3.7/configparser.py", line 857, in <listcomp> return [(option, value_getter(option)) for option in d.keys()] File "/Users/robertladwig/anaconda3/lib/python3.7/configparser.py", line 854, in <lambda> section, option, d[option], d) File "/Users/robertladwig/anaconda3/lib/python3.7/configparser.py", line 394, in before_get self._interpolate_some(parser, option, L, value, section, defaults, 1) File "/Users/robertladwig/anaconda3/lib/python3.7/configparser.py", line 444, in _interpolate_some "found: %r" % (rest,)) configparser.InterpolationSyntaxError: '%' must be followed by '%' or '(', found: '%^%forbear/%" -czf forbear.tar.gz *'

If the density driven inflow and the (normal) outflow are balanced at the beginning the water level remains constant, which is good.
However, if over 2 years of simulation the level first sinks (by about 0.5 m) and then rises again the water level does not end up exactly at the initial water level. The deviation is 2.4 cm in my case. In case of manual inflow placement, the water level ends up exactly at the initial water level. So to (approximate) method of density-driven inflow seems not to work perfectly or it has an error somewhere.

It seems that different lengths create problems for the restart functionality (snaphot). Either fix it so that different lengths are allowed or give a warning/error.

I try to run a single Simulation for about 100 years with daily forcing data and ran into an issue where Simstrat will act as if it successfully ran but actually no output is generated. I tried changing the "MaxLengthInputData" parameter but this did not change anything

I use Simstrat version 2.4 (using the SimstratR R package), an example output of such a run is:

[1]` " Simstrat version 2.4"                                                                                                                                             
 [2] " This software has been developed at eawag - Swiss Federal Institute of Aquatic Science and Technology"                                                            
 [3] " "                                                                                                                                                                 
 [4] " [OK] Configuration: simstrat.par"                                                                                                                                 
 [5] " [OK] Grid file successfully read"                                                                                                                                 
 [6] " [OK] Morphology file successfully read"                                                                                                                           
 [7] " [OK] Initial data file successfully read"                                                                                                                         
 [8] " [WARNING] The snow module is turned on. This module needs precipitation data, note that the last column in the forcing file will be interpreted as precipitation."
 [9] " Interval [days]:    1.0000000000000000     "                                                                                                                      
[10] " [OK] Output times successfully read"                                                                                                                              
[11] " [OK] Output depths successfully read"                                                                                                                             
[12] " [OK] Start day: 1.0                 "                                                                                                                             
[13] " [OK] End day: 37254.0             "                                                                                                                               
[14] ""                                                                                                                                                                  
[15] "  -------------------------- "                                                                                                                                     
[16] "    SIMULATION IN PROGRESS   "                                                                                                                                     
[17] "  -------------------------- "                                                                                                                                     
[18] ""                                                                                                                                                                  
[19] "    Time [d]   Surface level [m]       T_surf [degC]     T_bottom [degC]"                                                                                          
[20] ""                                                                                                                                                                  
[21] "  -------------------------- "                                                                                                                                     
[22] "     SIMULATION COMPLETED    "                                                                                                                                     
[23] "  -------------------------- "                                                                                                                                     
[24] ""                                    `

But the output file T_out is just the initial temperature profile and the whole run takes less than a second computation time.

system.time(SimstratR::run_simstrat("Simstrat/", "simstrat.par"))
   user  system elapsed 
  0.008   0.012   0.020

If needed I can provide the model setup to recreate the problem.

All the best,
Johannes

I'm trying to have Simstrat write only water temperature to a file. However, when I change the settings in Output (in the par file) "All" to "false" and "Variables" to ["T"], still all variables are written to file.

"All" : false, "Variables" : ["T"]

I also tried ["T", "S", "WaterH"], but with the same result. Am I doing something wrong with the syntax, or is this feature not yet working?

Thanks!

Can we add albsw to the parameters again?
At the moment it is not read from the inputfile anymore. It can be an important calibration parameter though.

In the output files a value > 9E+99 or < -1E-99 is stored without the E.
For example
1E+100 is stored as 1+100
-1E-100 is stored as -1-100

The model diverges and crashes if there is an anomaly in the bathymetry file (e.g. one area that is smaller at a shallower depth). A warning/error should be issued.

Simstrat seems to crash (or rather, start an infinite loop...) if the first output time corresponds exactly to the simulation start time, and is specified in a file. Did not check what happens also if output times are before the simulation start time and/or if they are specified directly in the par file.

With Forcing=2, IceModel=1, UseFilteredWind=False and SnowModel=0
forcing_update sets nval_offset=1 and then calls forcing_read with

call self%read (state%datum, A_s, A_e, A_cur, 5 + nval_offset, state%first_timestep)

This makes forcing_read to read date + 6 values from the forcing file.
A properly created forcing file with date + 5 values (Forcing=2) then gets read in such a way that it skips every second line.

The

read (20, *, end=9) tb_start, (A_s(i), i=1, nval)

will read the date, the 5 values and then the first value of the next line.
The next read (tb_end) will then read the 3rd line.

This is with GCC/10.3.

The forcing_read needs the cfg%ice_model, cfg%use_filtered_wind values to decide how much to read, or the code that sets nval_offset is wrong:

if (cfg%use_filtered_wind .and. cfg%ice_model == 0) then
      nval_offset = 1
else if (cfg%ice_model == 1 .and. cfg%use_filtered_wind) then
     nval_offset = 2
else if (cfg%ice_model == 1) then
     nval_offset = 1
else
     nval_offset = 0
end if

Should the

else if (cfg%ice_model == 1 .and. cfg%use_filtered_wind) then
  nval_offset = 2
else if (cfg%ice_model == 1) then
  nval_offset = 1

check cfg%snow_model instead of cfg%ice_model ??

The restart functionality is really useful but for our forecasting project we would like to assimilate in up-to-date initial conditions, initially for water temperature but potentially for other biogeochemical variables in the future.

We would like the ability to edit the restart file or have the file use the T_in.dat file when restarting. Currently it is not possible to edit the restart file.

When running Simstrat, sometimes large spikes in near-bottom temperatures appear (see attached figures). These do not always show up; if I run with low a_seiche values, for example, all seems normal. Any idea what may be causing this? I've attached the Tout.dat file (have a look at day 258). I've also attached a reproducible example, including the executable I'm using (you may need to add an empty "output" folder). Also as a note, the spikes are very short and the temperatures at the surface and the mid water column do not seem to be affected.

While I was figuring out what was the cause of this issue, another problem appeared. In the reproducible example, if you change "Start Year" to 2007, "End d" to 1460, and "a_seiche" to 0.00362, I get the error:

...
 [OK] Input file successfully read: input temperature
 [OK] Input file successfully read: input salinity
 Simulation progress |#######-------------------------|  21%
Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0x49cc73
#1  0x49553b
#2  0x46f2de
#3  0x7657a27c

I don't know if this is related to the issue with the near-bottom temperatures.

I am running Simstrat v 2.4.1 on Windows 10

Let me know if I need to send more information, or if I can help in any other way.

Repr_example_Simstrat.zip

T_out.txt

We can have unnecessary large output due to too many variables being written and not used by the user.
We could implement a switch "yes" or "no" whether all variables will be written. If "no", there could be an optional argument defining which variables should be written.

If a certain output time (specified in the file) is not matched by the simulation times, simstrat currently adjusts the timestep in the following way:

• Starting from some some output time (or from the start), it checks how many default timesteps are needed to match the next output time. This number is a float, for example 29.7.
• If the number is not an integer, it is rounded to the next integer below (29 in the example) and the timestep between the two output times is adjusted so that in 29 steps the next output time is reached
• This can increase the specified timestep by up to a few percent.

This is not really a neat solution for two reasons:

• The used timestep deviates from the one specified, and its not lower but larger which could potentially cause stability issues.
• The result of the code depends (very slightly) on the output times chosen by the user

We should implement a timestep adjustment which does not/minimally interfere with the simulation timestep. I see 2 possibilities:

• The model state at a certain output time is interpolated using the state just before and after
• The model uses the specified timestep until it cannot do a full timestep without crossing an output time. At this point, the model does a smaller timestep to reach the output time and goes on with the usual timestep.

I'm getting the error message "Negative heat flux not allowed for melting", when simulating a lake with (large amounts) of ice cover. In the lake, ice thickness can reach to about 50 cm. Some colleagues had exactly the same error in other lakes with ice cover when using Simstrat. The executable I used is the 2.4 Windows executable from this github, using forcing mode 3 (with precipitation and snow module on).

In my simulation, I start in January and the first winter ice cover forms and all looks normal. In the second winter, I get the error (ice forms before the model crashes). My colleagues also said that the error occurs somewhere in the middle of the simulation.

The forcing data that I use has worked in other models, and as far as I could see it should be working for Simstrat. Is this a bug, or am I doing something wrong? I can send my setup, if needed.

at the moment json files are invalid whe floating point numbers < 1.0 are writte as .xxx
it would be nice if the json library supports this format

maybe we can try to update to the newest version (https://github.com/jacobwilliams/json-fortran) and if it still doesen't work we could contact jacob williams who wrote the library.

It would be nice to have some kind of progress indication.
This is not too hard to implement. Either using a fancy existing library: https://github.com/szaghi/forbear
Or by a simple write with no-advance statement.

Is there a specific reason that t0 is omitted from the solution (i.e. output) in version 2.x?
For postprocessing, it would be easier if t0 is provided otherwise one has to read the solution AND interpolate the initial condition to get a full solution.

From version 3.0 on, Simstrat always includes the biochemical model AED2. The latter can be turned on and off using a switch in the configuration file. However, if a person really only wants to use the physical part of Simstrat, it could make sense to only compile Simstrat and not worry about the compilation of AED2. This could be done using something like

#ifdef __has_include(...)
#endif

i.e. preprocessing statements.
Currently, the AED2 part is a bit too much nested into Simstrat so to do this the code would need some refactoring.

If stability mode is 1 (defined in config file), the turbulent diffusivity of temperature is calculated as:
nu_t = cmue_2*k^2/eps, where cmue_2 = 0.072 according to Goudsmit et al., 2002.

However, in the code, cmue_2 = cmue/Prndtl = 0.09/0.8 = 0.112, so turbulent diffusivity of temperature might be overestimated by 50% (if stability mode = 1).