openalea / hydroshoot

Hydroshoot model for 3D hydraulic architecture simulation

OpenAlea Lab

Documentation

Official documentation is available at openalea.github.io

Install

To install OpenAlea use conda

If you want to retrieve the code from git, use git clone. Then use this command that will retrieve all the submodules::

git submodule update --init --recursive

Contribute

If you want to contribute to code, please have a look to github workflow

hydroshoot's Issues

remove unused 'ca' field from params.py

set air temperature as local leaf property

Setting air temperature as a property of a leaf allows to force non-constant leaf temperature values across the canopy

python version conflicts while installing hydroshoot

Hi Rami, Christian, and Christophe:
I am trying to install openAlea and hydroshoot on my linux system Ubuntu 18.04. however, i met lots of version problems.
Initially, the system has Python 3.7, 3.6 preinstalled. I installed python 2.7 and use environmenal variables to set it as the default version.
I have tried to install the hydroshoot python package using the conda environment, as given in this link: https://openalea.readthedocs.io/en/latest/install.html

However, there were conflicts in the python packages, as seen in the snapshot below. We have similar conflicts while installing other packages such as mtg too.
It would really help if we have a comprehensive guide on the version that is needed along with a step by step instruction on installation on openlea, hydroshoot and other dependent packages.

thanks very much in advance.
yours Junqi Zhu

use either of 'soil_total_volume' or 'rhyzo_total_volume' params

The soil water potential is curerntly initated each hour by considering that the soil water reservoir is identical to the total soil volume (i.e. inter-row * intra-row * depth OR pi * radius ^2 * depth) at the beggining of each time step. (code here)

Later in the code, the soil water potential is calculated as if only the volume of soil occupied by the default rhyzosphere cylender is available (code here)

• use either of both consistently
• set as optional argument

allow providing weather data as input

rm redundent properties setting

for instance in the hydraulic model:

• set once the max hydraulic conductivity here
• set once the soil properties here
• set once leaf area here

[bugfix] correct irradiance

The irradiance wrapper for caribu has two major errors :

1. It uses UTC instead of UT (big error obtained for sims over California)
2. it overwrites diffuse irradiance that is obtained from thr turtle method

refactor_energy_module

do some refactoring of the energy module so that :

• the doc string is updated and conform with that in irradiance for example
• all comments are remoevd
• forced parameters are put in the params.json and included in params_schema.json

conda install does not include the latest version

conda does not seem to include the modifications perfomed recently (at least after Sept 14th).
To verify, install hyrdoshoot following the procedure from README, then compare line 1139 in architecture with that in 'master'. Without the modifications indluded in hydroshoot, the model raises an error.

hydroshoot/src/hydroshoot/architecture.py

Line 1139 in c57eb5c

data = array(list(pos_data.values()))

any idea @pradal ? Thanks in advance !

Change Doc template to OpenAlea one

Change the template to meet the style of other OpenAlea packages.

remove unused atmospheric CO2 partial pressure from params.json

The atmospheric CO2 partial pressure (Ca) is used from weather inputs.
The parameter "ca" should be removed from params.json ('exchange' category). Its corresponding key in jsonscehma should also be removed.

update doc after irradiance bug_fix

Warnings and methods deprecated

There are several warnings:
DeprecationWarning: scipy.sqrt is deprecated and will be removed in SciPy 2.0.0, use numpy.lib.scimath.sqrt instead
azi = -scipy.arccos(min(max(-1,x/scipy.sqrt(x2+y2)),1))

• scipy.arccos, arcsin, radians, sqrt, ...
• scipy.array

Also
The pandas.datetime class is deprecated and will be removed from pandas in a future version. Import from datetime module instead.

allow using params as input

Using params as inputs to model.run() to avoid having to read params.json

reduce iteration time

bugfix soil water budget

Soil water content drops sharply due to miscalculated delta theta values. Some code correction and cleaning is required.

bugfix mtg with multiple plants

The model fails to contruct the digit of mtg when multiple plants is provided. This is due to python 2 to 3 compatibility issue (map object has to be converted into list):

hydroshoot/src/hydroshoot/architecture.py

Line 179 in 9731447

B_A_S_E = map(''.join, product(*((c.upper(), c.lower()) for c in 'base')))

• fix
• test
• bump patch

remove useless "soil_water_deficit" parameter

the user can simulate no water deficit by setting psi_soil (forced) input to some high water potential value

update to python3

[bugfix] fix geometry save function

add shape id to scene

This allows retrieving the same mtg ids for geometry and all other properties

set the irrigation routine in a separate module

https://github.com/MB3579/hydroshoot/blob/c2e4972130058c05252c3d60648e32d406911726/src/hydroshoot/hydraulic.py#L436

bugfix soil width and length set when inputs missing

take:
width = inter-row spacing
length = intra-row spacing

conda package uploading failed

@pradal @thomasarsouze , anyone has an idea ?

https://github.com/openalea/hydroshoot/actions/runs/4636910419/jobs/8205275534#step:4:3600

allow none hourly time steps

https://github.com/MB3579/hydroshoot/blob/c2e4972130058c05252c3d60648e32d406911726/src/hydroshoot/model.py#L93

Examples to be updated

Examples need to be updated so as to conform with the introduction of Path objects from the pathlib class in io.py.

Installing Hydroshoot on Windows - canestra has not finished properly => STOP

Installed hydroshoot (master) using "python setup.py install" in Windows 8 conda environment.
All other packages - visualea, mtg, plantgl, caribu etc - successfully installed.

Runs until PlantGL 3D View pops up, showing plant geometry (fig1a).
It seems that canestrad.exe is not happy (fig.1b) with linking to hydroshoot,
after the 'pirouette' stage.

Any suggestions with correctly installing hydroshoot on windows?
or solving the error output below?

Thanks

python sim.py

WARNING: QApplication was not created in the main() thread.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

• Project: C:\Users\office\Downloads\hydroshoot-master\example\modularity\ws\si
  m_2/
  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
  GDD since budbreak = 928 ┬░Cd
  Energy_budget: True
  Hydraulic structure: True
  Computing form factors...
  ... pirouette

canestra has not finished properly => STOP
Traceback (most recent call last):
File "sim.py", line 40, in
model.run(g, str(getcwd()) + '/', scene)
File "D:\Miniconda2\envs\oat1clone\lib\site-packages\hydroshoot-1.0.0-py2.7.eg
g\hydroshoot\model.py", line 170, in run
unit_scene_length=unit_scene_length)
File "D:\Miniconda2\envs\oat1clone\lib\site-packages\hydroshoot-1.0.0-py2.7.eg
g\hydroshoot\energy.py", line 121, in form_factors_simplified
raw, aggregated = caribu_scene.run(direct=True, infinite=infinite, split_fac
e=False, simplify=True)
File "d:\miniconda2\envs\oat1clone\lib\site-packages\alinea.caribu-1.4.4-py2.7
-win-amd64.egg\alinea\caribu\CaribuScene.py", line 537, in run
screen_size=screen_size)
File "d:\miniconda2\envs\oat1clone\lib\site-packages\alinea.caribu-1.4.4-py2.7
-win-amd64.egg\alinea\caribu\caribu.py", line 205, in raycasting
algo.run()
File "d:\miniconda2\envs\oat1clone\lib\site-packages\alinea.caribu-1.4.4-py2.7
-win-amd64.egg\alinea\caribu\caribu_shell.py", line 405, in run
self.canestra(opt)
File "d:\miniconda2\envs\oat1clone\lib\site-packages\alinea.caribu-1.4.4-py2.7
-win-amd64.egg\alinea\caribu\caribu_shell.py", line 539, in canestra
raise CaribuRunError(''.join(msg))
alinea.caribu.caribu_shell.CaribuRunError

add args to calculate sapflow per cordon

User Story

Today, simulations return only the total shoot sap flow with results. It is possible to get sapflow per shoot cordon but this requires manual manipulations in the code :

hydroshoot/src/hydroshoot/model.py

Lines 284 to 285 in 0a425a0

	# sapEast = []
	# sapWest = []

hydroshoot/src/hydroshoot/model.py

Lines 362 to 363 in 0a425a0

	# sapEast.append(g.node(arm_vid['arm1']).Flux)
	# sapWest.append(g.node(arm_vid['arm2']).Flux)

hydroshoot/src/hydroshoot/model.py

Line 394 in 0a425a0

# sapEast, sapWest = [np.array(flow) * time_conv * 1000. for i, flow in enumerate((sapEast, sapWest))]

To Do

• Add an argument somewhere (e.g. a kwargs in model.run()) to indicate that sapflow results are expected per grapevine shoot cordon.
• Ensure that all figures of the paper turn correctly, especially Figure 14.

Undate

rm deprecated functions in architecture

Continuous Integration

Add CI for running:

• testing
• conda build
• on Windows, Linux and Mac

add doc compilation to CI process

set photoinhibition parameters as optional

clean code

• remove useless parameters
• pep
• remove useless dependencies
• ...

unforce electron transport rate shape parameter

extract alpha from exchange.py

alpha = .24

remove g from hydroshoot inputs class

it is misleading to include hydroshoot object as an input to its own input...

hydroshoot/src/hydroshoot/io.py

Line 16 in 3eae391

def __init__(self, g: MTG, path_project: Path, scene: Scene, is_write_result: bool = True,

Release 1.0

Identify clearly the version of ydroshoot associated to the paper

• Create an archive on Zenodo
• Add a release / tag

HydroShoot tutorial

New tutorials are built on openalea web site.
Please give us a link to be able to build a tutorial.
Then @bbrument will be able to publish it on the website.

PlantGL crash _ Qt5 interactive GUI problem

@RamiALBASHA
I got the follwing error: when executing the example, the graphik window was crashing instantly after opening and the program was being blocked that point.

We found this relative discussion, and so we managed to solve the issue by running the command %gui qt5 before executing display.visu(....)

Is there a more general solution ?

Conda package name openalea.hydroshoot

Change the name of the conda package:
hydroshoot -> openalea.hydroshoot

And update the doc to install hydroshoot using a conda

Tests fail with Python 3.9

Error in test_model : test_potted_grapevine

Cause

json.load do not accept anymore encoding as keyword: jupyter/notebook#5165

Error Traceback

test_model.py:11: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../../miniconda3/envs/hydroroot/lib/python3.9/site-packages/hydroshoot-3.0.0-py3.9.egg/hydroshoot/model.py:40: in run
    params = Params(params_path)
../../../../miniconda3/envs/hydroroot/lib/python3.9/site-packages/hydroshoot-3.0.0-py3.9.egg/hydroshoot/params.py:22: in __init__
    user_params = self._get_user_params()
../../../../miniconda3/envs/hydroroot/lib/python3.9/site-packages/hydroshoot-3.0.0-py3.9.egg/hydroshoot/params.py:46: in _get_user_params
    json_file = load(f, encoding="utf-8")
../../../../miniconda3/envs/hydroroot/lib/python3.9/json/__init__.py:293: in load
    return loads(fp.read(),
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

s = '{\n  "simulation": {\n    "sdate": "2012-08-01 11:00:00",\n    "edate": "2012-08-01 11:00:00",\n    "latitude": 43.61...    0.5\n    ],\n    "rhyzo_coeff": 0.5,\n    "avg_root_spacing": 0.013,\n    "avg_root_radius":  0.0001\n  }\n}\n\n\n'
cls = <class 'json.decoder.JSONDecoder'>, object_hook = None, parse_float = None, parse_int = None, parse_constant = None
object_pairs_hook = None, kw = {'encoding': 'utf-8'}

    def loads(s, *, cls=None, object_hook=None, parse_float=None,
            parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):
        """Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray`` instance
        containing a JSON document) to a Python object.
    
        ``object_hook`` is an optional function that will be called with the
        result of any object literal decode (a ``dict``). The return value of
        ``object_hook`` will be used instead of the ``dict``. This feature
        can be used to implement custom decoders (e.g. JSON-RPC class hinting).
    
        ``object_pairs_hook`` is an optional function that will be called with the
        result of any object literal decoded with an ordered list of pairs.  The
        return value of ``object_pairs_hook`` will be used instead of the ``dict``.
        This feature can be used to implement custom decoders.  If ``object_hook``
        is also defined, the ``object_pairs_hook`` takes priority.
    
        ``parse_float``, if specified, will be called with the string
        of every JSON float to be decoded. By default this is equivalent to
        float(num_str). This can be used to use another datatype or parser
        for JSON floats (e.g. decimal.Decimal).
    
        ``parse_int``, if specified, will be called with the string
        of every JSON int to be decoded. By default this is equivalent to
        int(num_str). This can be used to use another datatype or parser
        for JSON integers (e.g. float).
    
        ``parse_constant``, if specified, will be called with one of the
        following strings: -Infinity, Infinity, NaN.
        This can be used to raise an exception if invalid JSON numbers
        are encountered.
    
        To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
        kwarg; otherwise ``JSONDecoder`` is used.
        """
        if isinstance(s, str):
            if s.startswith('\ufeff'):
                raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)",
                                      s, 0)
        else:
            if not isinstance(s, (bytes, bytearray)):
                raise TypeError(f'the JSON object must be str, bytes or bytearray, '
                                f'not {s.__class__.__name__}')
            s = s.decode(detect_encoding(s), 'surrogatepass')
    
        if (cls is None and object_hook is None and
                parse_int is None and parse_float is None and
                parse_constant is None and object_pairs_hook is None and not kw):
            return _default_decoder.decode(s)
        if cls is None:
            cls = JSONDecoder
        if object_hook is not None:
            kw['object_hook'] = object_hook
        if object_pairs_hook is not None:
            kw['object_pairs_hook'] = object_pairs_hook
        if parse_float is not None:
            kw['parse_float'] = parse_float
        if parse_int is not None:
            kw['parse_int'] = parse_int
        if parse_constant is not None:
            kw['parse_constant'] = parse_constant
>       return cls(**kw).decode(s)
E       TypeError: __init__() got an unexpected keyword argument 'encoding'

../../../../miniconda3/envs/hydroroot/lib/python3.9/json/__init__.py:359: TypeError

Automate result generation

Provide a well-documented master script (e.g., "run.sh" or "run.py) that can be executed to
reproduce the reported results end-to-end (including any preprocessing, simulation,
analysis, and visualization).

remove overridden 'par_photo' params

Vcm25, Jm25, TPU25 and cRd from par_photo field are overridden by those calculated depending on nitrogen content of leaves.
Those keys must be removed form params.json in order to avoid any confusion since they are not being used by the code.

[installation]

linked to openalea/openalea#254

correct typo in EB equation

add leaf tempearture to the energy bamance equation
https://hydroshoot.readthedocs.io/en/latest/energy.html#energy-budget
cf Eq. 13 in paper

Update README with citation

Update the README

• clearly describe the hydroshoot package
• howto reproduce results of the paper
• add a Citation to the in silico plants article
• add detailed info about the scripts and datafiles used to generate results of the paper

update readthedocs

The 'readthedoc' does not follow updated rst documentation (cf. installation.rst and corresponding installation html).
Is there any update to make on CI ?

fix the solar radiation unit

Fix the unit of Rg input (one of W(global)/m2, W(PAR)/m2, W(PPFD)/m2) and remove the "E_type" parameter from params

refacto_hydraulic

Docker

Add a dockerfile