kconnour / pyrt_disort Goto Github PK

Hi Kyle,

A quick question regarding the refactoring of the surface stuff in pyRT_DISORT from this August. Is there still a way to generate a Lambertian surface, or only Hapke now?

Thanks
Aurélien

Bug description

Evidently Atmosphere will replace n_wavelengths with n_layers if it's run with 1 wavelength.

Minimal working example

import pyrt

# More to come later

Expected behavior

No response

Screenshots

No response

Operating system

macOS

Python version

3.9

Priority

Medium

Further information

No response

The angles used in the spacecraft vs. rover scenario require very different constraints that I'm starting to understand.

Spacecraft

• The shape of the incidence, emission, and phase angles must be the same
• Emission must be between 0 and 90 degrees
• n_azimuth is 1 if we do RT on a pixel-by-pixel basis
• We want the aforementioned angles

Rover

• Incidence can be a single value, then emission and phase should have the same shape
• Emission can seemingly be between 0 and 180 degrees if I'm understanding things (evidently DISORT won't compute anything if it's >= 90 though)
• n_azimuth should be the same length as the emission angles
• We may only have incidence, emission, and azimuth angles and then need to compute the phase angles (is this true? We just have phi and phi0. Maybe having the phase angle would be beneficial in some way)

It seems that I need to specify a minimum of two different geometry classes, or add methods to the Angles class to compute everything depending on what you give it. As far as I can tell I won't need to modify classes outside of Angles, but be cognizant of the fact that my defaults are designed for the spacecraft case.

I'm open to suggestions...

please update documentation to show clear example of how the single column for UMU and PHI should be extracted for the call to disort.disort, i.e.

angles = sky_image(ina, azi0, ema_array, azi_array)
mu = angles.mu
mu0 = angles.mu0
phi = angles.phi
phi0 = angles.phi0
UMU = mu[0,:]
UMU0 = mu0
PHI = phi[0,:]
PHI0 = phi0

Feature request

TEMIS = 0 should be the default. This is setting the emissivity of the "upper boundary condition" (analogous to the surface), which is not something that it done in terrestrial atmosphere RT. So you want a value that is not going to create problems if someone turns on thermal emission and is not thinking to check this variable.

H_LYR variable name should be changed when you are setting the atmospheric scale height for vertical profiles. I suggest something like H or even "H_scale".

default value of H_LYR (which has dimension N+1 if there are N layers in the atmosphere) would just be the latitudes (AGL) of the layer boundaries (a.k.a. the z grid). there are only needed by DISORT when using the pseudo-spherical approximation. in the nomenclature of this example, right before the call to distort you could have a definition like so: H_LYR = altitude_grid

Priority

mainly a documentation request, but just in case some default values are set outside of the spacecraft retrieval example.

Feature request

it doesn't make sense to have the lambert albedo as a call to Surface since the actual Lambert invocation would be after that:

sfc = Surface(0.1, CP.n_streams, CP.n_polar, CP.n_azimuth, OB.user_angles,
            OB.only_fluxes)
sfc.make_lambertian()

i suggest make the Lambert Albedo a parameter in the make_lambertian method (it can be created with a default value if needed for passing to DISORT, i.e.,

sfc.make_lambertian(0.1)

I guess similar issue to #2 but even the trick provided there did not work... I tried on two different machines running under Ubuntu desktop 20.04 and Ubuntu server 20.04, with Python 3.9 and gfortran.

As noted in #2, pip install . did not compile the Fortran code.
python setup.py returns the following error:

usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: no commands supplied

So as suggested I tried to copy/paste the Fortran link command (last statement line)

INFO: /usr/bin/gfortran -Wall -g -Wall -g -shared /tmp/tmpqohzdv0p/tmp/tmpqohzdv0p/src.linux-x86_64-3.9/disortmodule.o /tmp/tmpqohzdv0p/tmp/tmpqohzdv0p/src.linux-x86_64-3.9/fortranobject.o /tmp/tmpqohzdv0p/tmp/tmp2yanqjl_.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/BDREF.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/DISOBRDF.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/ERRPACK.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/LAPACK.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/LINPAK.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/RDI1MACH.o /tmp/tmpqohzdv0p/tmp/tmpqohzdv0p/src.linux-x86_64-3.9/disort-f2pywrappers.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9 -lgfortran -o ./disort.cpython-39-x86_64-linux-gnu.so

Yet, that did not work either as I still get an error

(EXI) astcherbinine@tiu:~/python_modules/pyRT_DISORT$ /usr/bin/gfortran -Wall -g -Wall -g -shared /tmp/tmpqohzdv0p/tmp/tmpqohzdv0p/src.linux-x86_64-3.9/disortmodule.o /tmp/tmpqohzdv0p/tmp/tmpqohzdv0p/src.linux-x86_64-3.9/fortranobject.o /tmp/tmpqohzdv0p/tmp/tmp2yanqjl_.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/BDREF.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/DISOBRDF.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/ERRPACK.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/LAPACK.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/LINPAK.o /tmp/tmpqohzdv0p/home/astcherbinine/python_modules/pyRT_DISORT/disort4.0.99/RDI1MACH.o /tmp/tmpqohzdv0p/tmp/tmpqohzdv0p/src.linux-x86_64-3.9/disort-f2pywrappers.o -L/usr/lib/gcc/x86_64-linux-gnu/9 -L/usr/lib/gcc/x86_64-linux-gnu/9 -lgfortran -o ./disort.cpython-39-x86_64-linux-gnu.so
gfortran: error: /tmp/tmpqohzdv0p/tmp/tmpqohzdv0p/src.linux-x86_64-3.9/disortmodule.o: No such file or directory
gfortran: error: /tmp/tmpqohzdv0p/tmp/tmpqohzdv0p/src.linux-x86_64-3.9/fortranobject.o: No such file or directory

I've tested the installation on Ubuntu 20.10 and MacOS Catalina 10.15.7 and it works.

It failed on macOS 11.2.3 build 20D91. It fails to build the .so file but doesn't create an error.

since the asymmetry parameter (g) is included in forward scattering properties, it is natural that it is a property something like:

@property
def asymmetry_parameter(self) -> np.ndarray:
    """Get the asymmetry parameter on the new grid.

    """
    return self.__asymmetry_parameter

This would be useful for particle gradients where one was happy with the H-G phase function.

Feature request

The current surface routines (RHOU, RHOQ, etc.) are created by calling bdref.f. All the code is hidden there. It would be much easier to create the surface arrays in numpy and would remove all fortran dependencies in the pre-processing. It would also be testable (the current code isn't easily testable).

Priority

Low

I want to have a "return type" section for class properties. If I type hint functions, this works correctly but if I type hint class properties, the hint is part of the call signature. If I remove the type hint and add it to the "Returns" section of the docstring, it appears but includes a "Returns:" before "Return type:" that is always empty and unnecessary.

Hello,

I've tried to implement the build-in test 16 (pseudo-spherical correction) using the disort wrapper and get some array dimension errors.
But then I've noticed that a similar error is raised when I try to run the tests implemented in the pyRT_DISORT/tests/disort_tests folder.
Do you know where it comes from?

Output for python3 disotest1a.py

ValueError: 0-th dimension must be fixed to 16 but got 6


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/aurelien/python_modules/pyRT_DISORT/tests/disort_tests/disotest1a.py", line 63, in <module>
    rfldir, rfldn, flup, dfdt, uavg, uu, albmed, trnmed = disort.disort(usrang, usrtau, ibcnd, onlyfl, prnt, plank, lamber, deltamplus, do_pseudo_sphere, dtauc, ssalb,
ValueError: failed in converting 30th argument `rhou' of disort.disort to C/Fortran array

Output for python3 disotest9.py

ValueError: 0-th dimension must be fixed to 8 but got 4


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/aurelien/python_modules/pyRT_DISORT/tests/disort_tests/disotest9.py", line 63, in <module>
    rfldir, rfldn, flup, dfdt, uavg, uu, albmed, trnmed = disort.disort(usrang, usrtau, ibcnd, onlyfl, prnt, plank, lamber,
ValueError: failed in converting 30th argument `rhou' of disort.disort to C/Fortran array

Bug description

In short, the behavior of DISORT described in its documentation is wrong.

If TabularLegendreCoefficients is used as intended (i.e. where the 0th coefficient of the input Legendre decomposition is set to 1) then the 0th coefficients along this axis should always be unchanged and thus 1. If they are not initially set to 1, they won't be 1 after this class runs (which is expected). If they are not set to 1 when calling disort.disort(), DISORT does not set them to 1 like the documentation describes. Instead, it simply errors out.

Minimal working example

A small example cannot easily be created...

Expected behavior

DISORT says

The K = 0 coefficient should be unity (it will be reset to unity in any case)

so they should be reset. Instead I get this message:

 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****
 ****  Input variable  PMOM  in error  ****

So I guess I should include some sort of PMOM checker to ensure everything looks good before calling disort.disort(). In truth this variable is the trickiest to create, so it needs reworked anyway.

Screenshots

No response

Operating system

Ubuntu 20.10

Python version

3.9.6

Priority

Very low

Further information

No response

Hi Kyle!

I notice an issue in the Spacecraft Retrieval example, regarding the H_LYR parameter, which leads to errors when I tried to run the model with the pseudo-spherical approximation (enabling DO_PSEUDO_SPHERE = True led to bunch of NaN values in the logs).

I realized that it came from the fact that the pyrt.scale_height function returns the scale height values in meters, while the altitude profile is in kilometers.

So one just have to change H_LYR = pyrt.scale_height(temperature_profile, mass, gravity)
by H_LYR = pyrt.scale_height(temperature_profile, mass, gravity) / 1e3
in the documentation (and maybe mention it in the variable documentation page?).

Best,
Aurélien

We discussed a GPL vs MIT type license. I want to explore these more before adding a license.

Once I add a license, update README.md and add something to the documentation about it.

Feature request: the "source" button at the function / class signature links to a rtd themed version of the code. It'd be great to link to the actual code on Github instead.

I tried to implement this but failed. However it appears that numpy was able to do this in their current version (v 1.21) of their docs.

Feature request

Right now, this code will not compile on python3.12. I believe the root of the issue is that distutils is no longer available and thus packaging the code is trickier.

I've found by testing on Ubuntu 22.04 that if I remove the required numpy version in pyproject.toml, I can get the code to seemingly compile. f2py creates directories in /tmp but I no longer see the .so file anywhere within the project (it used to be in the disort4.0.99 directory). This looks suspiciously similar to the Mac installation issue.

Priority

medium, but likely quite challening

Bug description

Angles fails if part of the image crosses the terminator.

Angles was originally designed to handle angles of any shape. It would be more intuitive to have it handle a single set of angles, so I changed the incidence angles to cause errors at angles > 90 degrees. However, I still want the ability to compute the angles from a 2D array of incidence, emission, and phase angles, like from IUVS. This causes some problems if part / most of the image contain usable angles but some of the image happened to cross the terminator---95% of the image might be perfectly fine but the remaining 5% of pixels causes the computation to fail.

This is more that I need to redesign some things to get the functionality that I want with Mike's requests.

Minimal working example

import pyrt
import numpy as np
a = np.linspace(0, 10, num=50)
angles = np.outer(a, a)
pyrt.observation.phase_to_angles(angles, angles, angles)

Expected behavior

It should warn that some of the pixels have angles that are too large, but still compute the arrays.

Screenshots

No response

Operating system

Ubuntu 20.10

Python version

Python 3.9.6

Further information

No response

Bug description

pyrt.rayleigh_co2 wants column density in units m**-2 but pyrt.column_density outputs cm**-2.

perhaps ensure that column_density returns m**-2

Minimal working example

This is what I have to do to get the appropriate Rayleigh optical depth

column_density = pyrt.column_density(p_grid, T_grid, z_grid)
rayleigh_co2 = pyrt.rayleigh_co2(column_density*1.e4, wave_obs)

Expected behavior

No response

Screenshots

No response

Operating system

Mac OS Sonoma 14.3.1

Python version

Python 3.11.8

Priority

not, but a good idea

Further information

this is not truly a bug, but it will create that for people that do not to bench mark calculations in column density and Rayleigh optical depths.