Add documentation for Group Items in Processing Template about raster-functions

[jdrisdelle]

@thomas-maschler

We are in the process of writing this document at the moment. Let me try to explain it here in the meantime.

When applying a function to a stack of data in a mosaic dataset aggregation is controlled at the raster function template level. The properties of an rft can be accessed by right clicking on the top of the function chain and selecting Properties (in version 10.x).

The two settings in the rft properties that are used to aggregate records are the tag field name and group field name. you are also going to want to set the type parameter to item group The two fields that you reference in these parameters also need to exist in your mosaic dataset attribute table. The records in the group field will contain a unique identifier that will represent a group of items in the mosaic dataset while the tag field name will reference items in those individual groups. For example:

When referencing these items in the Python raster function you are going to set the data type of that parameter to *rasters *. You can see how this works in the aggregate function. Items in the mosaic dataset will be passed into the python raster function as a group. You can then use this group of items do perform operations like aggregation. I hope this helps. We will try to get the help topic up for this as soon as possible. Let me know if you have any questions.

[ThomasEmge]

Jamie,

I think you got the wrong Thomas ;)

From: Jamie Drisdelle [mailto:[email protected]]
Sent: Monday, February 29, 2016 3:08 PM
To: Esri/raster-functions [email protected]
Subject: Re: [raster-functions] Add documentation for Group Items in Processing Template (#29)

Hey Thomas,

We are in the process of writing this document at the moment. Let me try to explain it here in the meantime.

When applying a function to a stack of data in a mosaic dataset aggregation is controlled at the raster function template level. The properties of an rft can be accessed by right clicking on the top of the function chain and selecting Properties (in version 10.x).

[image]https://urldefense.proofpoint.com/v2/url?u=https-3A__cloud.githubusercontent.com_assets_8249074_13406962_fc64fcb0-2Ddeda-2D11e5-2D84e5-2Dd7541209fd13.png&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=1m0wo-6vj-vRGTP5YsZeQQ&m=aSYreFdAoR1esOoznaH8DuoaVxzfjQKAsg66Dr97sXk&s=4jRnrHsRXgLfBXHI2kYhv_ym3hSS-zYUM5LCeQ5oF94&e=

The two settings in the rft properties that are used to aggregate records are the tag field name and group field name. you are also going to want to set the type parameter to item group The two fields that you reference in these parameters also need to exist in your mosaic dataset attribute table. The records in the group field will contain a unique identifier that will represent a group of items in the mosaic dataset while the tag field name will reference items in those individual groups. For example:

[image]https://urldefense.proofpoint.com/v2/url?u=https-3A__cloud.githubusercontent.com_assets_8249074_13410415_816c3d5e-2Ddeec-2D11e5-2D9df7-2D4272d7c348d4.png&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=1m0wo-6vj-vRGTP5YsZeQQ&m=aSYreFdAoR1esOoznaH8DuoaVxzfjQKAsg66Dr97sXk&s=57JAy3VZtfclSJgzPQlotQMkEgbuzQ7qtNGvZXh3wbQ&e=

When referencing these items in the Python raster function you are going to set the data type of that parameter to *rasters *. You can see how this works in the aggregate functionhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_raster-2Dfunctions_blob_master_functions_Aggregate.py&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=1m0wo-6vj-vRGTP5YsZeQQ&m=aSYreFdAoR1esOoznaH8DuoaVxzfjQKAsg66Dr97sXk&s=Nkt6_9ppkirRjgFjmL9MAB9pogQ2lOjy6RaCdjnT67Q&e=. Items in the mosaic dataset will be passed into the python raster function as a group. You can then use this group of items do perform operations like aggregationhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_raster-2Dfunctions_blob_master_functions_Aggregate.py&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=1m0wo-6vj-vRGTP5YsZeQQ&m=aSYreFdAoR1esOoznaH8DuoaVxzfjQKAsg66Dr97sXk&s=Nkt6_9ppkirRjgFjmL9MAB9pogQ2lOjy6RaCdjnT67Q&e=. I hope this helps. We will try to get the help topic up for this as soon as possible. Let me know if you have any questions.

—
Reply to this email directly or view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_raster-2Dfunctions_issues_29-23issuecomment-2D190440426&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=1m0wo-6vj-vRGTP5YsZeQQ&m=aSYreFdAoR1esOoznaH8DuoaVxzfjQKAsg66Dr97sXk&s=qITHFoQPgTAIZ2nEMUEHsnrf5HdmcHkOjkaGbSZdG2I&e=.

[jdrisdelle]

Sorry @ThomasEmge . Not sure how that happened.

[thomas-maschler]

Thanks Jamie,

but how can I reference datasets in a specific group or with a specific tag inside the processing template?

By now I already figured out that if I write the tag in the Aliases field in the Variable Manager my raster function will only use rasters with that tag.

In the windchill raster function template I saw you were using something like this

t@SFC

What does that me? Can I reference a group like this?

Thanks,

Thomas

[akferoz]

@thomas-maschler, I'm sorry you had to figure that out without any/sufficient documentation.

TL;DR: The string t@SFC is simply a tag associated with each temperature "slice" constructed by the GRIB raster type when used against an NDFD forecast data from the NWS.

Here's one way to think about it…

Windchill is a good example of a binary raster function--needing two input rasters, temperature and windspeed--housed in a template with two variables of type raster as input.

Unary functions like NDVI or KeyMetadata are even simpler needing only one raster along with one or more scalars as input.

A third category of function (based on cardinality) is one that expects an (unordered) stack of rasters, like the Aggregate function.

In addition to the notes from @jdrisdelle, these are the points I want to emphasize when we document the interaction between functions, templates, map, mosaic dataset, and the user.

• On one end, raster functions only define what their input expectations are in terms of data type and default value, using the getParameterInfo method. Raster functions do not (and cannot) determine how datasets or layers are eventually "connected" to those inputs.
• On the other end, we have higher-level constructs that are containers of datasets like a map (containing image layers) or a mosaic dataset (containing items). As users, we are looking for ways to intuitively connect our raster function(s) with raster data.
• Raster function templates help make that connection. Exactly how ArcGIS uses your template is specific to the application and the context. Input raster datasets "bind" to input parameters of raster functions through variables defined in the template. More precisely, a raster dataset identified as, say, R, will bind with a particular template variable if it's named R or if it has an alias R.

In ArcMap, raster-based layers (like image server layer, mosaic layer and raster layer) are named based on their order in the Table of Contents. So, the first layer is Raster1, the second one is Raster2, and so on. This enables us to use any template (driven by any binary function) as long as the variable name (or alias) reads Raster1 and Raster2. Additionally, the first layer also goes by the name Raster to enable unary templates to work correctly. To enable aggregation, the virtual stack of all selected rasters is called Raster[]. We use order-based naming here because other methods of tagging the layers (based on, say, the layer name) would severely diminish your function templates' re-usability.

Mosaic datasets add another level of complexity because we now have multiple groups of rasters against which the template needs to be repeatedly applied in order to construct the mosaicked image. We can use a field in the mosaic dataset to uniquely identify different groups of items—think of them as neighborhoods or subsets in your collection. As @jdrisdelle described, this field is indicated to the function template's Group Field Name parameter. We use another field—specified in the Tag Field Name parameter—to identify each item within a particular group. The item's tag is its name and is used for binding with template variables (as long as it matches the variable's name or one of the aliases). We can use this construct to group together multiple items (like temperature and windspeed) by, say, time. Analogous to the map-based scenario, all items in a group are made available as a virtual stack and tagged Raster[]. This enables aggregation-type templates to work as well. Because a mosaic dataset is carefully authored and its items managed as records in a database table, we avoid order-based tagging of items and resort to using author-define or raster-type-specified names. Lastly, these rules for a mosaic dataset are in addition to prevailing constraints on each item w.r.t. visibility- and query-based filtering.

Hope that makes sense.