Area tracker about py-eddy-tracker

[AntSimi]

Hi,

I add today some short documentation about AreaTracker, which could be found here.

You have only one specific criterion : cmin which must be between [0,1], it's minimal value for cost function
Cost function is Intersection(contour1, contour2)/Union(contour1, contour2) => 1 -> exactly same contour, 0 -> No intersection

To manage missing time step you must used VIRTUAL_LEGNTH_MAX keyword like in this example.

If you want modify cost function, you need to inherit from EddiesObservations class and overwrite tracking method.

Antoine

[JeremyCollin]

Hi Antoine thanks for this quick reply.

I am currently working with Louise and we are modifying the code in order to track 12h observations from ROMS.

We were curious about some eddies with great lifetime tracked at 12h and not 24h. They appear to be small fast-advecting eddies that doesn't intersect at 24h.

Jérémy

[AntSimi]

Did you use a classical detection or a detection based on okubo weiss?

[JeremyCollin]

Okubo-weiss using Jonathan Gula version

[AntSimi]

I didn't do detection with Okubo weiss, just i look at this field in this example.
Did you compare the footprint of eddies with classical detection and your detection with Okubo Weiss one? It's something that i am interested to know and maybe build an example about that.
Most of the time i use AVISO output, so eddy size and movement are compliant with overlap method.

I you want to use AreaTracker with this time sampling (24h which didn't overlap in some case), i think you need to build a new cost function to take benefit of overlap and manage quick eddies.
You could also use an old classical tracking method which are based only on ellips research area and similar amplitude and radius.

But is it a problem for you to work with 12h time step?

Antoine

[JeremyCollin]

Comparing Okubo weiss with ssh will be done in a near future. We can let you know how this goes for sure.

and no, this not a blocking issue. We will work with 12h time stepping as new detected eddies are coherent structures. We just wanted to understand how area tracker works before going forward.

[AntSimi]

Could you give me a quick summary of quantity in percent of observation which are stored in untracked/short/long after tracking?

[CoriPegliasco]

Hi, @JeremyCollin you are right your eddies are too small to have an overlap after 24h.
The overlap method (area_tracker) is designed for large eddies that move slowly. You can see here a large eddy moving very slow compared to its size:

An idea could be to to advect the eddy contour from one day to another in case of rapid displacement (but I've never tried that).

[JeremyCollin]

@AntSimi
out of topic answer: (difference between 12h and 24h obeservations)
We used Okubo-Weiss interpolated at the different vertical levels (-2000, -1500, -1000, -500, -50) meters. Tracking was made with 10 days lifetime and 3 days of virtual observations. We found around 10% more eddies at the bottom and up to 50% near the surface. Most of these eddies have a short lifetime with more than 90% between 10 to 20 days. Less than 1% lived for more than 40 days.

the appropriate answer : (I hope)
1 year global observations at -50m using okubo
tracked : obs = 466802, untracked 317175, too short 743905

[AntSimi]

When we work in global with AVISO1/4, classic statistics with AreaTracker look like that (for 30 years)
with more than 10 days for long with virtual=4

$ ncdump -h Anticyclonic.nc |grep "obs ="
	obs = 31971496 ;
$ ncdump -h Anticyclonic_untracked.nc |grep "obs ="
	obs = 433181 ;
$ ncdump -h Anticyclonic_track_too_short.nc |grep "obs ="
	obs = 2379818 ;

So statistics are very different, but we don't work with high resolution, we have around 1.3% of untracked and 7.5% of short

[AntSimi]

Another question did you look at radius distribution for each category (long/short/untracked) ?

[JeremyCollin]

Hi Antoine,
sorry for the late reply but here is some plots of eddy effective radius distribution from AreaTracker with 10 days lifetime at least and 3 days of virtual observations. Detection was performed on ROMS 2km grid Okubo-Weiss at 50 m depth with 12h outputs in the north Atlantic for over a year. We can see the number of observations in the y-axis related to the radius (km) x-axis.

I am actually surprised to see that radius are rather small, what do you think ?

[AntSimi]

I am also suprised by distribution, if you want i could give you distribution for aviso/cmems on similar area(if you give me area specification).
You could also with GUIEddy (in py eddy tracker) look at eddy paths interactively, to look if area-tracker miss somethings.

I am still afraid that OW minimize area/radius (Still to check) or maybe model resolution prevents big eddies.

[JeremyCollin]

When looking at animations detection seems correct. We are looking at 70W - 10 E x 35 N - 70 N area. We should soon look closer at ADT detection for comparison.

[AntSimi]

With following code, i extract from a global atlas (Compute with AVISO/CMEMS 1/4 allsat on ADT) your area and get those distributions:

Speed radius distribution

Effective radius distribution

[JeremyCollin]

Hi Antoine,
indeed detection using adt reveals larger structures. We are still investigating whether OW detection could be helpful in our vertical approach.

Jonathan started to develop our branch from e767238 revision and then merged master version of py-eddy-tracker into it. We are currently running some tests to ensure our devs are correct. We are comparing detection (OW) and tracking (area_tracker) and we find small differences in contour, shape error, radius and location of eddies at detection, and also a little less tracked eddies (0.1%). Was there any change on detection contour, shape, center, radius between master and e767238 or is it our code that brought an issue ?

Thanks

[AntSimi]

Right now i haven't obvious reason to explain differences.
Is it for one day or larger period?
In order to found reasons, maybe work on only one day.
To isolate eddy difference you could use match method with high cmin.

[JeremyCollin]

Well this is for a larger period. I hoped there was a change in contour definition but unfortunately we'll have to look closer at 1 day detection then.

[AntSimi]

How did you ran identification? With a python script or with eddyid?

[AntSimi]

Another question number of eddy are the same for one day?

[JeremyCollin]

How did you ran identification? With a python script or with eddyid?

EddyTracking area_tracker.yaml

Another question number of eddy are the same for one day?

yes they are exactly the same number for detection

[AntSimi]

My question was about identification and not tracking

[JeremyCollin]

python modified script
a, c = h.eddy_identification('ow', 'u', 'v', date, step=1e-11, z_min=-0.3 * 1e-10, z_max=-0.2 * 1e-10, pixel_limit=(20, 10000), shape_error=55, force_height_unit='µm',force_speed_unit='m/s', vorticity_name='vrt')

[AntSimi]

Maybe instead of radius could you compare speed_area and effective_area to avoid possible differences

[louisebeley]

Hi antoine, when we compare speed_area and effective_area, we have an aproximate error of 0.1%.

[AntSimi]

Same value with radius? or it's better with area?

The most direct thing to understand the difference is to load 2 identification file use match function with cmin around .99 (will return all index with high match) and after extract with index and reverse option to keep only different eddy.
And look at those eddies (grographic, amplitude distribution, radius distribution, ...).

Something like that:

i_0, i_1, c = d0.match(d1, cmin=0.99)
d0_ = d0.index(i_0, reverse=True)
d1_ = d1.index(i_1, reverse=True)

[AntSimi]

I wrote a small bash script in order to reproduce your problem here, i could see some difference with this example.
I will try to find changes and check if there are all wanted.

[AntSimi]

After some tests i found some informations:
new_version => last version
old_version => commit of your fork

Detection changes begin after commit a1515b, like it could see in comparison (at the end of the message).

Since this commit the default value of nb_step_to_be_mle is set to 0, which stop any eddy detection with two local maxima. But you use eddy_identification method directly, so you are not concern by that ...

In reference the commit, where you start your fork,:

[ref] old_version/Anticyclonic_20160515.nc -> 65 obs
[0] commit_5cbc1165b3e2ac469763e72d1e4018982bce7bfd/Anticyclonic_20160515.nc -> 65 obs
[1] commit_a1515bdb1dfd7ea9466b2692a45b50948b751705/Anticyclonic_20160515.nc -> 64 obs
[2] new_version/Anticyclonic_20160515.nc -> 64 obs
          nomatch    50 <= low < 80  intermediate    100 <= high    multi_match      parent          twin          complex    
[ 0]     0.0% (0)       0.0% (0)       0.0% (0)      100.0% (65)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 1]     1.5% (1)       3.1% (2)       3.1% (2)      92.3% (60)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 2]     1.5% (1)       3.1% (2)       3.1% (2)      92.3% (60)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch    50 <= low < 80  intermediate    100 <= high    multi_match      parent          twin          complex    
[ 0]     0.0% (0)       0.0% (0)       0.0% (0)      100.0% (65)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 1]     0.0% (0)       3.1% (2)       3.1% (2)      93.8% (60)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 2]     0.0% (0)       3.1% (2)       3.1% (2)      93.8% (60)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)

In reference, last version available at this time:

[ref] new_version/Anticyclonic_20160515.nc -> 64 obs
[0] commit_a1515bdb1dfd7ea9466b2692a45b50948b751705/Anticyclonic_20160515.nc -> 64 obs
[1] commit_5cbc1165b3e2ac469763e72d1e4018982bce7bfd/Anticyclonic_20160515.nc -> 65 obs
[2] old_version/Anticyclonic_20160515.nc -> 65 obs
          nomatch    50 <= low < 80  intermediate    100 <= high    multi_match      parent          twin          complex    
[ 0]     0.0% (0)       0.0% (0)       0.0% (0)      100.0% (64)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 1]     0.0% (0)       3.1% (2)       3.1% (2)      93.8% (60)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 2]     0.0% (0)       3.1% (2)       3.1% (2)      93.8% (60)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
     Point of view of study dataset
          nomatch    50 <= low < 80  intermediate    100 <= high    multi_match      parent          twin          complex    
[ 0]     0.0% (0)       0.0% (0)       0.0% (0)      100.0% (64)     0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 1]     1.5% (1)       3.1% (2)       3.1% (2)      92.3% (60)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)    
[ 2]     1.5% (1)       3.1% (2)       3.1% (2)      92.3% (60)      0.0% (0)       0.0% (0)       0.0% (0)       0.0% (0)