esri / raster-functions

A curated set of lightweight but powerful tools for on-the-fly image processing and raster analysis in ArcGIS.

ArcGIS Raster Functions

This repository houses modern image processing and analytic tools called raster functions. Raster functions are lightweight and process only the pixels visible on your screen, in memory, without creating intermediate files. They are powerful because you can chain them together and apply them on huge rasters and mosaics on the fly.

In this repository, you will find useful function chains (*.rft.xml) created by the Esri community. You can also create custom raster functions in Python that work seamlessly with the several dozen functions that ship with ArcGIS.

Getting Started

1. Install ArcGIS for Desktop 10.4 or higher or pro 1.2 or higher, or ArcGIS for Server 10.4 or higher.
2. Install the latest release of prerequisite Python extension packages if you are setting up for the first time:
   • Download Python extensions binaries.
   • Unzip the contents to a temporary local folder.
   • Run <local-folder>/setup.py with administrator privileges.
3. Install the latest release of custom raster functions:
   • Download all custom raster functions. If you are using pro use the master branch. If you are using arcmap use the arcmap107 branch
   • Unzip the contents locally to a home folder.
   • You'll find ready-to-use templates and functions in their own subfolders.
4. Learn more about raster functions, function chains, and templates using the Resources below.
5. Learn how you can create new raster functions using the Python API.

Resources

• Fundamentals

  • ArcGIS Pro Help
  • The raster functions Wiki
  • Python in ArcGIS Pro

• Raster Functions

  • What are raster functions?
  • List of raster functions in ArcGIS Pro
  • Editing functions on a raster dataset
  • Creating custom raster functions
  • Using raster functions on Portal for ArcGIS

• Raster Function Templates

  • Applying raster function template for image analysis
  • Understanding raster function template properties
  • Editing raster function templates in a mosaic dataset
  • Applying raster function templates
  • Deploying custom python raster functions and templates

• Scientific Computing in Python

  • Scientific Computing Tools for Python
  • Python Scientific Lecture Notes

Issues

Find a bug or want to request a new feature? Please let us know by submitting an issue.

Contributing

Esri welcomes contributions from anyone and everyone. Please see our guidelines for contributing.

Featured Raster Functions and Templates

• Key Metadata

  KeyMetadata.py demonstrates how key properties can be introduced or overridden by a raster function. These are the inputs to the function:

  • Property Name, Property Value—The name and value of the dataset-level key property to update or introduce.
  • Band Names—Band names of the outgoing raster specified as a CSV.
  • Metadata JSON—Key metadata to be injected into the outgoing raster described as a JSON string representing a collection of key-value pairs. Learn more here.

  This function serves as an example of one that doesn't need to implement the .updatePixels() method.

• Mask Raster

  MaskRaster.py enables you to apply the input mask raster as the NoData mask on the primary input raster.

  MaskRaster.rft.xml is a grouping raster function template where the inputs are the primary raster and the mask raster (in that order).

• Select By Pixel Size

  SelectByPixelSize.py accepts two overlapping rasters and a threshold value indicating the resolution at which the function switches from returning the first raster to returning the second raster as output. If unspecified, the threshold parameter defaults to the average cell size of the two input rasters.

  SelectByPixelSize.rft.xml is a grouping raster function template that accepts two rasters as input and leaves threshold unspecified.

• Fish Habitat Suitability

  FishHabitatSuitability.py returns a raster representing suitability of fish habitat at a user-specified ocean depth given two rasters representing water temperature and salinity. This function demonstrates how raster functions can be exploited in analytic workflows.

  FishHabitatSuitability.rft.xml is a grouping raster function template that accepts the temperature and salinity rasters (in that order). This template—when used in the Add Rasters to Mosaic Dataset tool with the Table raster type or as a processing template on a mosaic dataset—is capable of obtaining the value of the depth parameter from a specific field (StdZ, if available) in the table.

• Vineyard Analysis

  VineyardAnalysis.py serves to demonstrate how you can compute a suitability raster given the elevation, slope, and aspect of the region.

  VineyardAnalysis.rft.xml accepts the elevation input raster and uses built-in raster functions to compute slope and elevation before feeding the output to the Vineyard Analysis raster function.

• Topographic C-Correction

  Topographic c-correction is used to remove the effects of hillshade on multispectral images. It reduces the effects of reflectance variability in areas of high or rugged terrain, thus improving the consistency of the multispectral image pixel values and the quality of images as additional processing is applied. There are many different topographic correction algorithms. These algorithms have been compared by Ion Sola et. al (2016) and the c-correction proposed in Teillet, Guindon, and Goodenough (1982) was ranked as one of the best topographic correction methods.

Licensing

Copyright 2014 Esri

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

A copy of the license is available in the repository's License.txt file.

raster-functions's Issues

Focal Statistics output pixel size

The output cell size is define in the updaterasterInfo method. However, it is for some reason changed inside the updatePixels method. Also, the change is different in the x and y direction.

Add documentation for how a raster processing model (RFT) references Python modules

which path for extension packages?

When I run 'python setup.py install' to install the extensions, do I want
C:\Python27\ArcGIS10.3\python.exe or
C:\Python27\ArcGISx6410.3\python.exe
in my path variable?

Enable support for Cython

Implement FMask based cloud filtering

We should develop a cloud filtering algorithm that uses FMask. There are some sensor specific capabilities out there that use FMask, such as sentinel2-cloud-detector

Add python function for computing fish habitat suitability.

Multiple calls of function with different parameters replace all results

If a function, with different user specified parameters, is called more than one time inside the same map document, all previous results are replaced by the last one.

can not import scikit-image in ArcGIS Pro 1.4

I'm using ArcGIS Pro 1.4 and want to create my own raster functions with scikit-image. ArcGIS Pro 1.4 is using python 3.5. I downloaded every library from this page in version 3.5 and have it installed, but it is not working.
http://www.lfd.uci.edu/~gohlke/pythonlibs/#cython
Has somebody any idea what my fault is or maybe it's not working with python 3.5?

How to reference a field value in mosaic table to use in Python RFT

Hi,
I made a simple Python RFT chained with another RFT template. I use a Rasterize Feature to access a value inside a feature table. Then I am able to apply my python RFT to a mosaic. But It is not optimal for me.
I would like to directly use this numeric value stored in a field of my mosaic (this value can change from one raster to another inside the mosaic).
I made lot of test but unsuccessful.
Does someone has an idea on how to do that ?

thanks for your help.

Add documentation for Group Items in Processing Template

Please add documentation for Group Items in Processing Template to wiki.

When looking at RTF and python scripts alone (without the actual data), it is not clear how I can reference different raster datasets inside the raster mosaic as variables.

How do I need to prepare the data/ raster mosaic table? How do I reference the different rasters in the template afterwards?

Support padding of pixel block edges to enable neighborhood processing.

Hillshade: severe performance degradation when crossing antimeridian

I've been trying out the Hillshade function on mosaic datasets containing bathymetric data. I really like how the Python version looks vs. the built-in mosaic function -- it's much smoother-looking. However, I'm running into a problem when the viewing window crosses the antimeridian. The drawing performance gets severely degraded, and it may take several minutes for the image to display. This becomes an issue when the mosaic is published as an image service to be used in a map viewer, and the user zooms into an area surrounding 180 longitude in the Pacific.

I tried tracing what was going on by using print statements in the Hillshade.py code. I can see that, instead of the updatePixels() function getting called once, it gets called dozens or hundreds of times in a row, with what seems to be tiny windows of pixels.

Here are some steps to replicate the problem:

1. Add 2 rasters to a mosaic dataset, on either side of the antimeridian. Here are some example files (MRF format, LERC compression): https://drive.google.com/file/d/0Bz0z_q8d9UpDYzhBeUFRd1dKdG8/view?usp=sharing
2. Add the Python Hillshade function into the raster function chain.
3. Change the map's projection to something Pacific-centered, such as PDC Mercator.
4. Zoom in near the antimeridian, and it will take a very long time to display.

Not sure if this is a problem with the mosaic dataset code in ArcGIS, or with the Python function. I'm using ArcMap 10.4.1.

Thanks,
Jesse

Enable support for spatial reference in the Select By Pixel Size function.

As a user, I want to specify Cell Size Threshold in mercator meters and not in the units of the coordinates of the image request. The raster function must internally reproject to request SRS to compare request and threshold cell sizes.

Hyperlinks Missing for select Raster Functions and Templates

The hyperlinks for MultidirectionalHillshade.pyd and the accompanying MultidirectionalHillshade.rft.xml raster function template do not seem to be working:

https://github.com/Esri/raster-functions/blob/master/README.md#multidirectional-hillshade

error running Clip raster function using Foreach loop

I'm writing a script that needs to clip a multidimensional raster (Gridmet temperature data) to a smaller area. Poking around the documentation this is the workflow that I came up with
temp_max_file = r"D:\r3_analyst_work\Climate_Change_Modeling\climate_envelope\GridMetData_2020\tmmx_2020.nc" max_temp_rast = arcpy.Raster(temp_max_file,True) clip_frame= r"D:\r3_analyst_work\GIS_work\test_working_netcdf_climate_data\Default.gdb\small_area" max_temp_clip_raster = arcpy.sa.Foreach(max_temp_rast,"Clip",{"ClippingGeometry":clip_frame, "ClippingType":1})
When I run this I get
**RuntimeError Traceback (most recent call last)
In [39]:
Line 1: max_temp_clip_raster = arcpy.sa.Foreach(max_temp_rast,"Clip",{"ClippingGeometry":clip_frame, "ClippingType":1})

File C:\Program Files\ArcGIS\Pro\Resources\ArcPy\arcpy\ia_ia.py, in Foreach:
Line 114: return arcgisscripting._ia.Foreach(in_raster=in_raster,

RuntimeError: Failed to apply Raster Function: 'Clip' (The parameter is incorrect. )**

There is no indication what parameter is incorrect.
I had to extract the parameter name by exporting the Clip raster function and then reading the XML document.

Using the ArcPro interface the raster function works correctly on those data sets.

From what I can tell, my code is exactly the same as the NDVI example provided in the Foreach documentation.

Deploy Raster function on server

Hello,

I successfully managed to develop a small python raster function. I created a new custom category in C:\Program Files\ArcGIS\Pro\Resources\Raster\Functions\Custom\ and in C:\Program Files\ArcGIS\Server\framework\runtime\ArcGIS\Resources\Raster\Functions\Custom\

I copied the python script and the xml raster function in these two folders. it should be possible to use my python raster function on the server and arcgis Online. It works on Arcgis pro on the server but, impossible to to apply the function on mosaic service on Arcgis Online.

Thanks for your help.

Updated key metadata missing on an image service

Updated properties are not transmitted to the client by an image service when key metadata raster function is applied on the back-end mosaic dataset (either on top of the mosaic function or as part of a processing template).

cc: @amitjain27, @jdrisdelle, @wju, @rsjiezhang

Holding variables within one central table

We want users to be able to view weather data in the context of how it will affect their activities. Using the REMAP function, and a set of thresholds, we can remap weather values into “Favourable”, “Marginal” and “Unfavourable”, relative to the activity. The thresholds at which the values are remapped would be something like;

• Favourable is FROM wind speeds of 0 m/s TO 3 m/s
• Marginal is FROM wind speeds of 3 m/s TO 7 m/s
• Unfavourable is FROM wind speeds of 7 m/s TO 2000 m/s

Storing these ranges of values within each of the raster functions means that they are somewhat inaccessible for editing / checking / updating / re-use. However:

• At present the values are stored within the raster functions themselves because the REMAP function will not allow the use of an external table with remap ranges.
o But even if we could use a table for the remap ranges, nevertheless every time a user wanted to look at the impact of a weather variable on a new activity (e.g. the effect of temperature on running performance) they would need to create a new table – even though they are really just creating new data under a pre-defined schema.
• What we would prefer is for all the activities, weather variables and threshold values to be held in a single table so that users can add / edit / delete thresholds all in one place. For this to occur any one raster function would need to be able to query against that table using both activity and weather variable in the WHERE clause to pull out the correct REMAP values. The table columns would be something like: [ACTIVITY, WEATHERVAR, FROMVAL, TOVAL, UNITS, REMAPVAL], where REMAPVAL would represent ‘Suitability’, and a typical set of values might be: [“Running”, “Temperature”, -30, -20, “Centigrade”, -2], where -2 might represent ‘Unfavourable’.

How to write YOLO or U-Net Deep Learning code which is integrated with RasterFrames?

Hi,
I have read the RasterFrames supervised machine learning on the page of "https://rasterframes.io/supervised-learning.html", now I would like to implement Deep Learning similar to “supervised machine learning”, such as "YOLO object detection" and "U-Net semantic segmentation".
However, I am so confused about how to write YOLO or U-Net Deep Learning code which is integrated with RasterFrames? So, would you pls help to give me some suggestions? Many Thanks!

Add Section to Wiki on Writing RFTs to Process Collections of Imagery

To do: Add a section to the Wiki on writing image processing algorithms to process collections of imagery using the Group Items By capability in the RFT properties.

How to use code

Hello

I'm not used to use python scripts. How would I implement the code into my ArcMap project?
If I try to generate a new toolbox the script gives no error but also does not produce any output...

Thanks!

Python extension packages missing

The following link is dead:

https://github.com/Esri/raster-functions/releases/download/v1.0-beta.1/python-extensions-1.0-beta.1.zip

Write Wiki to have instructions for ArcGIS Pro instead or in addition to ArcMap

@jdrisdelle I think the Wiki should reference how to do this in ArcGIS Pro. That's what I am using and some other people that I work with are using. Also, we submit RA jobs from ArcGIS Pro, so it would be beneficial to have the instructions written from ArcGIS Pro.

Difficulties with publishing a rft with a python raster function from Pro to ArcGIS server 10.9.1

Hey!

We're unable to add our rft.xml file with a python raster function. We are referencing this path in the rft file:

[Install]\Server\framework\runtime\ArcGIS\Resources\Raster\Functions\Custom\subfolder\OurRasterFunction.py

Everything seems to work fine in pro where we can update the parameters we want to update and add the rft in our mosaic processing template.
But when we try and publish an image service with the rft file it throws an error telling us the template is invalid.
From the server logs:

Method failed.HRESULT = 0x80043010 : This is a FACILITY_ITF error that is specific to the interface that returned this error. See the documentation of the interface that returned this error for information about this HRESULT.

We were able to publish from arcmap 10.8.1, but then we dont get the capability of changing the parameter in the image service. Its stuck to the default value and we need to be able to deliver this to a customer that uses Pro.

It seems this is a compatibility issue between arcgis server 10.9.1 and the latest pro version when it comes to the template xml structure.
Do we have to upgrade to arcgis server 11 to make this work? Or is it any workaround that dont require to use Arcmap?

I can't share too much information as this is some sensitive data im working with. But i can answer any questions about the setup.

MultiDirectional Hill shade function does not return the correct out put

MultiDirectional Hill shade function add support for nodata handling

Padding configuration for input pixel blocks works incorrectly.

The padding configuration provided for the input rasters pads only the left and top side of the pixel blocks, causing issues after manipulation of pixel values.

How reference many Rasters to other rasters (Warb)

I am try to reference many Rasters to other rasters (Warb) using python code, But I feild, I need help.

import arcpy

i = 0

while i <= 1:

inRasterworkspace = r'C:\Users\AgmnDesk\Desktop\GooleImage\GI_R'
#inRasterworkspace = arcpy.GetParameterAsText(0)
arcpy.env.workspace = inRasterworkspace
inRasters = arcpy.ListRasters()
inRaster = inRasters[i]

SoTop=arcpy.GetRasterProperties_management(inRaster , "TOP")
SoLeft=arcpy.GetRasterProperties_management(inRaster , "LEFT")
SoRight=arcpy.GetRasterProperties_management(inRaster , "RIGHT")
SoBottom=arcpy.GetRasterProperties_management(inRaster , "BOTTOM")
Source_pnt = '''+ str(SoLeft) + ' ' + str(SoTop) + '' ;' + str(SoRight) + ' ' + str(SoTop) + '' ;' + str(SoLeft) + ' ' + str(SoBottom)+'''

========================

taRasterworkspace = r'C:\Users\AgmnDesk\Desktop\GooleImage\GI2018'
#taRasterworkspace = arcpy.GetParameterAsText(1)
arcpy.env.workspace = taRasterworkspace
taRasters = arcpy.ListRasters('*', 'jpg')
taRaster = taRasters[i]

TaTop=arcpy.GetRasterProperties_management(taRaster , "TOP")
TaLeft=arcpy.GetRasterProperties_management(taRaster , "LEFT")
TaRight=arcpy.GetRasterProperties_management(taRaster , "RIGHT")
TaBottom=arcpy.GetRasterProperties_management(taRaster , "BOTTOM")
Target_pnt = '''+ str(TaLeft) + ' ' + str(TaTop) + '' ;' + str(TaRight) + ' ' + str(TaTop) + '' ;' + str(TaLeft) + ' ' + str(TaBottom)+'''

========================

arcpy.env.workspace = r'C:\Users\AgmnDesk\Desktop\GooleImage\GI2018'
arcpy.env.overwriteOutput = True
arcpy.Warp_management(str(taRaster), Source_pnt, Target_pnt, str(taRaster[:-4])+'tif', "POLYORDER2", "BILINEAR")

========================

i = i + 1

Add error catching to pixel percentile calculations

For thoroughness, we should add try-except to the pixel percentile calculations

Hillshade.py in ArcGIS Pro

I haven't been able to get the Hillshade function to work in ArcGIS Pro (2.2.0). I just get a blank image as a result, with no error message. Other functions do work as expected, such as AspectSlope.py. Has anyone else had success with Hillshade.py in Pro?

Note: I didn't install the separate Python extension packages, since the required scipy and numpy packages are already available in the ArcGIS Pro environment.

Issue with XGBoost loading a pretrained model in a custom raster function

Hello,

I'm looking for help trouble shooting a custom raster function. The heart of the rf is an XGBoost model, but something goes wrong when I try to load the model. I've used Pickel to help narrow down the issue, and verified that the xgboost library IS loading and that the filepath for the model is correct. For whatever reason, the model is not loading. Using a notebook in ArcPro, I could not replicate the issue (everything runs as it should). It seems to be an issue specifically with the raster function. I'm really at a loss as to how to fix this and any insight would be much appreciated!

Other notes: Everything up to the loading of the model is working (verified this using Pickel). The rf on a whole runs, but provides no output (because the model is not loading).

The code around the issue:

    def updatePixels(self, tlc, shape, props, **pixelBlocks):
        # Preprocessing: Create mask(s)(vector and raster) as needed

        # Get bands
        image_array = np.array(pixelBlocks['NAIP_ras_pixels'], 'float32')
        red = image_array[0,:,:]
        green = image_array[1,:,:]
        blue = image_array[2,:,:]
        nir = image_array[3,:,:]
        
        def calc_rdvi(nir_values, red_values):
            nom = nir_values - red_values
            denom = np.sqrt(nir_values + red_values)
            return (nom/denom) 

        def calc_osavi(nir_values, red_values):
            nom = nir_values - red_values
            denom = nir_values + red_values + 0.16
            return (nom/denom)
        
        def calc_evi(nir_values, red_values, blue_values, G=2.5, C1=6, C2=7.5, L=1):
            nom = nir_values + red_values
            denom = nir_values + (C1 * red_values) - (C2 * (blue_values + L))
            evi = G*(nom/denom)
            evi = np.nan_to_num(evi, posinf=0, neginf=0)
            return evi

        def calc_vari(green_values, red_values, blue_values):
            return (green_values - red_values)/(green_values + red_values - blue_values) 

        def calc_msavi(nir_values, red_values):
            term1a = np.square((2*nir_values) + 1)
            term1b = 8*(nir_values-red_values)
            term1 = np.sqrt(term1a-term1b)
            term2a = (2*nir_values)+1
            term2 = term2a - term1
            return term2/2

        def calc_gli(green_values, red_values, blue_values):
            nom = (green_values - red_values) + (green_values - blue_values)
            denom = (2*green_values) + red_values + blue_values
            return (nom/denom)

        rdvi = calc_rdvi(nir, red)
        osavi = calc_osavi(nir, red)
        evi = calc_evi(nir, red, blue)
        vari = calc_vari(green, red, blue)
        msavi = calc_msavi(nir, red)
        gli = calc_gli(green, red, blue)

        array_shape = rdvi.shape

        vi_array = np.array([rdvi.flatten(), osavi.flatten(), evi.flatten(), vari.flatten(), msavi.flatten(), gli.flatten()])

        del (red, green, blue, nir, rdvi, osavi, evi, vari, msavi, gli)

        # Fix any nan or inf that result from VI calculation (problems in nodata areas)
        vi_array = np.nan_to_num(vi_array, posinf=0, neginf=0)

        # Step 2: Create the dataframe
        def generate_vi_df(two_dim_vi_stack):
            # create a DF to match XGBoost model
            df = pd.DataFrame(columns=['ID','RDVI','OSAVI',
                                      'EVI','VARI','MSAVI','GLI',
                                      ], dtype='float32')

            df['ID'] = range(0, two_dim_vi_stack.shape[1])
            df['RDVI'] = two_dim_vi_stack[0,:]
            df['OSAVI'] = two_dim_vi_stack[1,:]
            df['EVI'] = two_dim_vi_stack[2,:]
            df['VARI'] = two_dim_vi_stack[3,:]
            df['MSAVI'] = two_dim_vi_stack[4,:]
            df['GLI'] = two_dim_vi_stack[5,:]
            return df

        df = generate_vi_df(vi_array)

        # Step 3: Load the XGBoost model

        # Able to output in a Pickel a dict with configs (xgb.get_config()). So XGBoost *is* being imported. 22 Nov 2023.
        # test = xgb.get_config()
            
        #(The JSON will load independently, so the path is correct. 22 Nov 2023.)
        model_fn = 'TreeCAP_Full_US_NoOrgSpecBandsOREco_RDVI_OSAVI_EVI_VARI_MSAVI_GLI.json'
        model_filename = os.path.join(xgb_model_directory, model_fn)

        ## Unclear if model is actually loading. 22 Nov 2023
        bst = xgb.Booster(model_file=model_filename)  

        ## This should provide a dict output (this works in and Arcpro notebook) to test if model is loading,
        ## but Pickle fails to save output. No file is created at all. 22 Nov 2023
        output = bst.get_fscore()

        ########## save a pickle ##########
        fname = 'pixelBlocks.p'
        pickle_filename = os.path.join(debug_logs_directory, fname)
        pickle.dump(output, open(pickle_filename,"wb"))
        ########## pickle saved ##########

Add python function sample that demonstrates creation of output raster's NoData mask.

Zonal stats function

Hi, I'm seeing if you think this is possible. I want to create a raster function where I pass in a polygon and it returns the zonal stats average value of a value raster. I want the output raster but just the min or max of the output zone raster. I want this to function in ImageServer and I use the JS API to pass in a polygon feature set and it returns the resulting summary value. I currently have a raster function that successfully chains Rasterize Features and then Zonal Statistics functions to create the correct zone raster but not sure about the rest. Do I need a python raster function to get the output value?

Enable output and input rasters to be of different resolutions.

We want to enable the python raster function to specify a scaleFactor in .getConfiguration() indicating the difference in input raster resolution w.r.t. the current request resolution.

Elevation Fill Void - Fill raster function

I am looking at using the Elevation Fill Void function (I believe it is called Fill Raster under the function folder) as an alternative to the Spatial Analyst Fill function. When I run the function in ArcGIS Pro, the result I get are not what I expect and analysis run on the result of the raster function are incorrect.
I looked at the code under the function folder here. There is no inline documentation in the code. I am having trouble following what it does and how it relates the raster tool in ArcGIS Pro.

1. Is the fill raster function code the code for the Elevation Fill Void? If not can you please point me to the code for that function?
2. If it is the code can you please provide additional information so I can understand exactly what the code is supposed to be doing.
3. Is there a way to directly access the function code from ArcGIS Pro?
   Thank you.

Slope-Aspect Customized Slope Values

Hello,

I was wondering how I would be able to set customized slope values in for my results? S instead of the last one being 40% I would like Greater than 30 DEGREES slope.

Thanks,
Ryan

Compute Histogram

Hi Guys & Girls,

I wrote a python raster function (slightly changed a given function)

and after using it in ArcMap/Image Analysis, I got a monochrome result raster. After calculating a histogram in order to classify the Image, the result is fine. However, I want to include that histogram computation in my script, so that there is a classified result raster.
Is that possible?

Thats the script:

import numpy as np

class Arithmetic():
def init(self):
self.name = "Arithmetic Function"
self.description = "Performs simple arithmetic operations on two rasters."
self.op = None
self.applyScaling = True
self.applyColormap = False

def getParameterInfo(self):
    return [
        {
            'name': 'r1',
            'dataType': 'raster',
            'value': None,
            'required': True,
            'displayName': "Early NDVI",
            'description': ""
        },
        {
            'name': 'r2',
            'dataType': 'raster',
            'value': None,
            'required': True,
            'displayName': "Late NDVI",
            'description': ""
        },
        {
            'name': 'op',
            'dataType': 'string',
            'value': 'Add',
            'required': False,
            'domain': ('Add', 'Subtract'),
            'displayName': "Operation",
            'description': ""
        },
        {
            'name': 'method',
            'dataType': 'string',
            'value': 'Colormap',
            'required': False,
            'domain': ('Raw', 'Grayscale', 'Colormap'),
            'displayName': "Output Image Type",
            'description': ("The type of output expected from this function. Specify Raw for scientific analysis. "
                            "Pick Grayscale or Colomap for visualization.")
        },
    ]

def getConfiguration(self, **scalars):
    return {
        'inheritProperties': 2 | 4 | 8,
        'invalidateProperties': 2 | 4 | 8,
    }

def updateRasterInfo(self, **kwargs):
    method = kwargs.get('method', 'Colormap').lower()
    self.applyColormap = method == 'colormap'
    self.applyScaling = self.applyColormap or method == 'grayscale'

    maximumValue = 2.0
    minimumValue = -2.0
    if self.applyScaling:                                   # maximum output value depends on whether we are scaling
        maximumValue = 200.0

    colormap = ()
    pixelType = 'f4'
    if self.applyColormap:
        pixelType = 'u1'
        colormap = (np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255], dtype='int32'),
                    np.array([36, 36, 36, 36, 245, 245, 245, 245, 247, 247, 247, 247, 247, 247, 247, 247, 247, 247, 247, 247, 247, 247, 247, 247, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 250, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 252, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 253, 250, 244, 238, 234, 231, 223, 217, 211, 205, 200, 195, 189, 184, 180, 174, 169, 163, 160, 154, 148, 143, 138, 134, 130, 126, 117, 115, 112, 106, 100, 94, 92, 90, 81, 75, 71, 66, 62, 56, 51, 51, 51, 50, 50, 50, 50, 49, 49, 49, 48, 48, 48, 48, 48, 48, 48, 48, 47, 47, 47, 47, 46, 46, 46, 46, 45, 45, 45, 45, 44, 44, 44, 43, 43, 43, 43, 43, 43, 42, 42, 42, 42, 42, 42, 42, 41, 41, 41, 41, 40, 40, 40, 40, 40, 39, 39, 39, 39, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38, 38], dtype='uint8'),
                    np.array([0, 0, 0, 0, 20, 24, 29, 31, 33, 33, 37, 41, 41, 41, 45, 45, 47, 49, 49, 54, 54, 56, 58, 58, 62, 62, 62, 67, 67, 67, 69, 71, 71, 75, 75, 78, 79, 79, 79, 81, 83, 83, 87, 87, 90, 92, 93, 93, 97, 97, 97, 97, 101, 101, 101, 101, 105, 105, 107, 109, 109, 113, 118, 119, 121, 126, 132, 133, 135, 141, 144, 150, 152, 153, 159, 163, 165, 168, 174, 176, 181, 183, 186, 191, 197, 201, 203, 205, 209, 214, 216, 218, 224, 228, 234, 236, 238, 243, 248, 252, 252, 252, 250, 247, 246, 245, 240, 237, 235, 233, 230, 227, 224, 222, 220, 217, 214, 212, 210, 207, 204, 201, 199, 197, 194, 191, 189, 186, 184, 181, 179, 176, 174, 173, 168, 166, 163, 160, 158, 156, 153, 153, 153, 150, 150, 150, 150, 148, 148, 148, 145, 145, 145, 145, 143, 143, 143, 143, 140, 140, 140, 140, 138, 138, 138, 138, 135, 135, 135, 135, 133, 133, 133, 130, 130, 130, 130, 130, 130, 128, 128, 128, 125, 125, 125, 125, 122, 122, 122, 122, 120, 120, 120, 120, 120, 117, 117, 117, 117, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115, 115], dtype='uint8'),
                    np.array([255, 255, 255, 255, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 13, 20, 23, 25, 33, 38, 40, 43, 48, 54, 59, 61, 64, 69, 77, 79, 82, 87, 92, 97, 99, 102, 107, 115, 120, 123, 125, 130, 138, 141, 143, 150, 156, 163, 165, 168, 173, 181, 186, 186, 187, 180, 176, 173, 169, 163, 157, 150, 146, 142, 136, 132, 126, 123, 119, 114, 108, 105, 101, 96, 93, 88, 84, 81, 77, 70, 68, 64, 60, 55, 49, 47, 45, 37, 33, 28, 24, 21, 14, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], dtype='uint8'))



    m = kwargs.get('op', 'Add').lower()

    if m == 'add':          self.op = np.add
    elif m == 'subtract':   self.op = np.subtract

    kwargs['output_info']['statistics'] = ({'minimum': minimumValue, 'maximum': maximumValue}, )  # we know something about the stats of the outgoing NDVI raster. 
    kwargs['output_info']['histogram'] = ()           # we know nothing about the histogram of the outgoing raster.

    kwargs['output_info']['colormap'] = colormap      # optional colormap if requesting for an color image
    return kwargs

def updatePixels(self, tlc, shape, props, **pixelBlocks):
    r1 = np.array(pixelBlocks['r1_pixels'], dtype='f4', copy=False)
    r2 = np.array(pixelBlocks['r2_pixels'], dtype='f4', copy=False)

    np.seterr(divide='ignore')
    pixelBlocks['output_pixels'] = self.op(r1, r2).astype(props['pixelType'], copy=False)
    return pixelBlocks

def updateKeyMetadata(self, names, bandIndex, **keyMetadata):
    if bandIndex == -1:
        keyMetadata['datatype'] = 'Processed'               # outgoing raster is now 'Processed'
    elif bandIndex == 0:
        keyMetadata['wavelengthmin'] = None                 # reset inapplicable band-specific key metadata
        keyMetadata['wavelengthmax'] = None
    return keyMetadata

Enable modifications to the geometry of an image.

Provide an example of python raster function that's supplies or modifies the Xform object to the raster's geometry.

Add support for non c-contiguous arrays in the python raster function

Currently the python raster function assumes that all arrays are c-contiguous. Need to add support for non-c contiguous arrays.

aspect-slope raster function problems

From Cristelle:
Bug 1: White Patches/Areas appear at zoomed out levels, on application of Aspect-Slope Raster Function on a “Mosaic Dataset” in ArcMap
Version tested on: 10.5

Data Location : \spider\Swap\Cristelle\Zikang\Aspect_Slope\ArcMap\WhiteAreas_Mosaic

• From ArcCatalog, access “Mosaic” ->Right-click “Properties” -> Processing Templates -> “+” -> Right- Click on “identity function” -> Insert Python Raster function -> Add AspectSlope.py to the Python module parameter -> OK-> Apply
• Now Add the Mosaic to Table of contents -> Layer Properties -> Processing Templates -> Select the newly added Python raster function -> Apply
• Notice the White spots/areas present at zoomed out levels only
• Issue not reproducible on Raster Dataset or Image service. Reproducible only on Mosaic Datasets.

Bug 2: Only Pixel borders appear colorized, below scale levels of 20000, on application of Aspect-Slope Raster Function on an Image Service in ArcMap
Version tested on: 10.5

Data Location : https://elevation.arcgis.com/arcgis/rest/services/WorldElevation/Terrain/ImageServer
\spider\Swap\Cristelle\Zikang\Aspect_Slope\ArcMap\PixelBorders_Service

• From ArcCatalog, add ArcGIS Server connection to https://elevation.arcgis.com/arcgis/rest/services/WorldElevation/Terrain/ImageServer (You may need to be logged into AGOL within ArcMap in order to access this elevation service)
• Add the Terrain service to ArcMap
• From ArcMap “Windows” option -> Image Analysis -> Select layer -> Add function -> Right- Click on “identity function” -> Insert Python Raster function -> Add AspectSlope.py to the Python module parameter -> OK
• Now zoom into the Function Layer. Notice at zoomed out levels the colomap appears fine, however below scales of about 20,000 only the pixel borders appear to be colorized
• Issue not reproducible on Raster Dataset or Mosaic Datasets. Reproducible only on Image service.

Bug 3: Legend/Colormap doesn’t get applied , on application of Aspect-Slope Raster Function on a Raster Dataset, Mosaic or Image Service in ArcGIS Pro
Version tested on: ArcGIS Pro 1.4

Data Location : \spider\Swap\Cristelle\Zikang\Aspect_Slope\Pro\NoColorMap
https://elevation.arcgis.com/arcgis/rest/services/WorldElevation/Terrain/ImageServer

• Copy over AspectSlope.py to C:\Program Files\ArcGIS\Pro\Resources\Raster\Functions\Custom\AspectSlope
• From ArcGIS Pro, view either “SETSM_WV01_20130312_1020010021857100_102001002012D800_seg1_8m_dem.tif”, “Mosaic” or Image service “ https://elevation.arcgis.com/arcgis/rest/services/WorldElevation/Terrain/ImageServer”
• From Ribbon -> Imagery -> Function Editor -> Add the Raster Dataset and AspectSlope.py into the editor and link them & save it as a AspectSlope.rft
• From Raster Functions Tab -> Custom -> AspectSlope.rft -> Select input -> Create new layer
• Notice the Legend is white and data does not display. If you use the identify tool the Pixel Values show up.
• Issue not reproducible in ArcMap. Reproducible only in ArcGIS Pro.