esri / collector-tools

A set of python scripts and geoprocessing tools to automate common tasks and workflows in conjunction with Collector for ArcGIS

A set of utilities to help configure web maps and feature layers for use with Collector for ArcGIS. Browse the sections below, or jump to a specific section.

Note: Many of the scripts, notebooks, and toolboxes can also be applied to ArcGIS Field Maps.

Notebooks

Scripts

Toolboxes

Notebooks

Jupyter notebooks are provided to demonstrate various methods to streamline and automate configuration, administration, and deployment of data collection projects.

Functionality
Exclude web maps from Collector
Configure search for a web map
Generate app links for Collector

Scripts

Supports ArcGIS API for Python v1.4.1 (https://developers.arcgis.com/python/)

Functionality	Format
Reset Required Fields (ArcGIS API for Python)	Python script
Add GNSS Metadata Fields (Pro) - FeatureClass	Python Script
Add GNSS Metadata Fields (Pro) - Hosted Feature Service (ArcGIS API for Python)	Python Script
Configure GNSS Popup (ArcGIS API for Python)	Python Script
Project Z	Model Builder

Toolboxes

The scripts and ModelBuilder are also available as a GeoProcessing Toolbox for use in ArcGIS Pro.

If you are still using ArcMap, we offer a GeoProcessing Toolbox and associated python scripts that works with ArcMap 10.4 or higher.

Dependencies

• arcpy 3.4+
• ArcGIS API for Python

Instructions

Script instructions

1. Install ArcGIS API for Python package as described here.
2. Clone or download this repository.
3. Run the scripts from command line.

Toolbox instructions

1. Clone or download this repository.
2. If you prefer to use in ArcGIS Pro, right-click on Toolboxes, and select Add Toolbox. Navigate and select CollectorUtils_Pro toolbox.
3. Run the tools in the toolbox (Pro requires 2.0+) or run the scripts from command line.

Resources

• Collector for ArcGIS
• ArcGIS API for Python
• ArcGIS Arcade Expressions

Issues

Although we do our best to ensure these scripts, notebooks, and toolboxes work as expected, they are provided as is and there is no official support.

If you find a bug, please let us know by submitting an issue.

Contributing

Esri welcomes contributions from anyone and everyone. Please see our guidelines for contributing.

Licensing

Copyright 2020 Esri

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

A copy of the license is available in the repository's LICENSE file.

collector-tools's Issues

Add GNSS Metadata Field (Pro) - Hosted Feature Service Failed

I;m getting the error below when trying to run the Add GNSS Metadata Field (Pro) - Hosted Feature Service tool on a feature layer in our ArcGIS Online site:

Start Time: Thursday, August 10, 2017 11:33:36 AM
Running script AddGNSSFieldsPythonAPI...
Failed script AddGNSSFieldsPythonAPI...
Traceback (most recent call last):
File "C:\ArcGIS\ArcGIS_Desktop_Tools\collector-tools-master\CollectorUtils\add_update_gnss_fields_python_api.py", line 17, in
from arcgis.gis import GIS
ImportError: No module named 'arcgis'
Failed to execute (AddGNSSFieldsPythonAPI).
Failed at Thursday, August 10, 2017 11:33:37 AM (Elapsed Time: 0.88 seconds

OfflineUtils tool does not show on ArcGIS 10.3

The script tools for OfflineUtils aren't available when opened in ArcGIS 10.3.

VS.

Error when running 'ProjectZ' tool

I'm trying to use the 'ProjectZ' tool to display my Z data collected by a Trimble R2. I downloaded the 'CollectorUtils_Pro' toolbox, added the toolbox to ArcGIS Pro, and navigated to the 'ProjectZ' tool. I filled in the tool's required information, then clicked run. Every time I try to run this tool, as well as other tools in the toolbox, I receive the following error: "ERROR 000576: Scrpit associated with this tool does not exist". Is there a way to resolve this issue?

Reset required fields not working.

Hello! Is this tool still functional? I have tried running it but it doesn't do anything to my data. It still shows that it's completed without any warnings. The integers default are 0, and strings are ' ' , the fields are non-nullable. Why do you think it might not be working? Thank you!

OfflineUtils

After the download of the CollectorUtils toolbox, the OfflineUtils toolset with the projectZ model is not included. Only the OnlineUtils toolset is added.

ProjectZ tool not working in ArcPro 2.2

Hi there,
I am working with a Leica Zeno GG04 GPS unit and have paired it with with my IPad using Leica Zeno Connect App and Beta version of Collector for ArcGIS. Through the Zeno Connect App I am connecting to an Internet RTK network for real time corrections.

I have used the Add GNSS Metadata Fields tool to begin collecting the GNSS Metadata and that is working very well. However, I kept getting error when running the ProjectZ tool, please help me fix this! Thanks,

Amy
-------------------------------------------------------------------------------------------------------------------**
Parameters
Input Features C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\Export_Output_2_RecreateGeom
Input Coordinate System (of the x, y, z values) GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],VERTCS['NAD_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PARAMETER['Vertical_Shift',0.0],PARAMETER['Direction',1.0],UNIT['Meter',1.0]]
X-Value Coordinates ESRIGNSS_LONGITUDE
Y-Value Coordinates ESRIGNSS_LATITUDE
Z-Value Coordinates ESRIGNSS_ALTITUDE
Output Dataset or Feature Class C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\monu
Output Coordinate System GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],VERTCS['NAVD_1988',VDATUM['North_American_Vertical_Datum_1988'],PARAMETER['Vertical_Shift',0.0],PARAMETER['Direction',1.0],UNIT['Meter',1.0]]
Geographic Transformation NAD_1983_To_NAVD88_CONUS_GEOID12B_Height

Messages
Start Time: Monday, July 2, 2018 11:59:14 AM
Executing (Recreate Geometry): RecreateGeometry C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\Export_Output_2_RecreateGeom GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],VERTCS['NAD_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PARAMETER['Vertical_Shift',0.0],PARAMETER['Direction',1.0],UNIT['Meter',1.0]] ESRIGNSS_LONGITUDE ESRIGNSS_LATITUDE ESRIGNSS_ALTITUDE C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\Export_Output_2_RecreateGeom1
Start Time: Monday, July 2, 2018 11:59:15 AM
Running script Recreate Geometry...
Creating new FC...
Copied data to new FC...
0 record(s) skipped due to empty metdata fields
All Rows Updated..
Completed script Recreate Geometry...
Succeeded at Monday, July 2, 2018 11:59:17 AM (Elapsed Time: 1.80 seconds)
Executing (Project): Project C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\Export_Output_2_RecreateGeom1 C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\monu GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],VERTCS['NAVD_1988',VDATUM['North_American_Vertical_Datum_1988'],PARAMETER['Vertical_Shift',0.0],PARAMETER['Direction',1.0],UNIT['Meter',1.0]] NAD_1983_To_NAVD88_CONUS_GEOID12B_Height GEOGCS['GCS_North_American_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],VERTCS['NAD_1983',DATUM['D_North_American_1983',SPHEROID['GRS_1980',6378137.0,298.257222101]],PARAMETER['Vertical_Shift',0.0],PARAMETER['Direction',1.0],UNIT['Meter',1.0]] NO_PRESERVE_SHAPE # VERTICAL
Start Time: Monday, July 2, 2018 11:59:17 AM
Succeeded at Monday, July 2, 2018 11:59:18 AM (Elapsed Time: 1.17 seconds)
Executing (Maintain Attachments): MaintainAttachments C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\Export_Output_2_RecreateGeom C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\monu
Start Time: Monday, July 2, 2018 11:59:19 AM
Running script Maintain Attachments...
Enabling Attachments
Enabled Attachments
Copying Attachments..
Failed script Maintain Attachments...
Traceback (most recent call last):
File "C:\Python27\collector-tools-master\CollectorUtils\maintain_attachments.py", line 41, in enable_copy_attachments
arcpy.Append_management(inputRow, outputTable)
File "c:\users\chena9\appdata\local\programs\arcgis\pro\Resources\arcpy\arcpy\management.py", line 4318, in Append
raise e
File "c:\users\chena9\appdata\local\programs\arcgis\pro\Resources\arcpy\arcpy\management.py", line 4315, in Append
retval = convertArcObjectToPythonObject(gp.Append_management(*gp_fixargs((inputs, target, schema_type, field_mapping, subtype, expression), True)))
File "c:\users\chena9\appdata\local\programs\arcgis\pro\Resources\arcpy\arcpy\geoprocessing_base.py", line 496, in
return lambda *args: val(*gp_fixargs(args, True))
arcgisscripting.ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000732: Input Datasets: Dataset C:\Users\chena9\Documents\ArcGIS\Projects\MyProject1\MyProject1.gdb\Export_Output_2_RecreateGeom__ATTACH does not exist or is not supported
Failed to execute (Append).

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Python27\collector-tools-master\CollectorUtils\maintain_attachments.py", line 58, in
enable_copy_attachments(args.input_fc, args.output_fc)
File "C:\Python27\collector-tools-master\CollectorUtils\maintain_attachments.py", line 44, in enable_copy_attachments
arcpy.Error(e)
AttributeError: module 'arcpy' has no attribute 'Error'
Failed to execute (Maintain Attachments).
Failed at Monday, July 2, 2018 11:59:21 AM (Elapsed Time: 1.85 seconds)
Failed to execute (ProjectZ).
Failed at Monday, July 2, 2018 11:59:21 AM (Elapsed Time: 6.69 seconds

Attachments fail to get copied over to output feature class when running the ProjectZ tool, when the input feature class resides in a feature dataset.

• The ProjectZ tool fails to maintain attachments in the output feature class if the input feature class resides inside a feature dataset inside the geodatabase.

The workaround for now, would be to make sure the input feature class is not stored inside a feature dataset.

ERROR 000732 While Running Project Z

I'm attempting to run the Project Z tool in Pro on a set of survey points and I'm running into the following error when the tool reaches the Maintain Attachments script:

Running script MaintainAttachments...
Enabling Attachments
Enabled Attachments
Copying Attachments..
Failed script Maintain Attachments...
Traceback (most recent call last):
File "C:\Users\tfrauenfelder\Documents\ArcGIS\Projects\GPS\maintain_attachments.py", line 41, in enable_copy_attachments
arcpy.Append_management(inputRow, outputTable)
File "c:\program files\arcgis\pro\Resources\arcpy\arcpy\management.py", line 3984, in Append
raise e
File "c:\program files\arcgis\pro\Resources\arcpy\arcpy\management.py", line 3981, in Append
retval = convertArcObjectToPythonObject(gp.Append_management(*gp_fixargs((inputs, target, schema_type, field_mapping, subtype), True)))
File "c:\program files\arcgis\pro\Resources\arcpy\arcpy\geoprocessing_base.py", line 506, in
return lambda *args: val(*gp_fixargs(args, True))
arcgisscripting.ExecuteError: Failed to execute. Parameters are not valid.
ERROR 000732: Input Datasets: Dataset C:\Users\tfrauenfelder\Documents\ArcGIS\Projects\GPS\GPS.gdb\SURVEY__ATTACH does not exist or is not supported
Failed to execute (Append).

These are the setting I am using for conversion in the Project Z tool-

Any insight into what is going wrong here would be much appreciated, thanks!

"Failed to Execute" error when using Project Z tool

Greetings,

I am trying to use the Project Z tool to convert point data collected using an Arrow 200 GNSS (data collected is in WGS84(G1674)/ITRF08 w Z values in meters) to NAD 1983 HARN StatePlane Oregon North FIPS 3601 Feet Intl. I am using ArcGIS 10.4.1. The conversion seems to work fine up to the point of actually saving the Feature Class in the location specified- see below

These are the setting I am using for conversion in the Project Z tool-

I have set the environment option to point to where I want to save the output, tried various drives Local & Network. Any suggestions would be greatly appreciated!
MC

Add GNSS Metadata fields (Pro) error messages "Tool has failed to open..."

I am working in 2.0.1 ArcPro. I downloaded and extracted the collector-tools. When I double click to open the "Add GNSS Metadata Fields (Pro) - FeatureClass" tool, I get the following error message.

Tool has failed to open: S:\GIS_Data\Mapping\7429Production\MAPS\Production\Laura\Mobile Field\GPS Pilot Program\JOC Contract\collector-tools-master(4)\collector-tools-master\CollectorUtils\pro\CollectorUtils_Pro.tbx\AddGNSSFieldsPro

If you have any idea of what the issue is, please let me know.

Thank you,

Laura

ProjectZ for ArcMap

Where did it go?

Geographic Transformation Help

Thanks for putting together this tool. It looks like just what I need to accomplish a project I am working on.

I am starting to work with a Trimble R2 centimeter accuracy GPS unit and have paired it with with my android phone using Trimble's GNSS Status App and Collector for ArcGIS. Through the GNSS Status App I am connecting to an Internet RTK network for real time corrections.

I have used the Add GNSS Metadata Fields tool to begin collecting the GNSS Metadata and that is working very well.

What I am having trouble with is an understanding of how to use the ProjectZ tool. Particularly, I am having trouble applying the geographic transformation. I am not sure how to do it correctly. Can you provide and help with this?

Thanks,
Dennis

Having issue finding project Z model

Currently, I am having an issue finding the project Z model in the tool box. I have esri 10.5 and downloaded the collect tools and extracted to a specified location. When I open up the folder in windows explorer it shows everything in the folder but when opening the folder location in catalog or arc map it doesn't show anything in the CollectorUtils

Getting Warning Error 001839

I am getting this error telling me

a datum transformation is expected, but the parameter is empty

Can someone decifer why this is happening.

I ran the tool earlier with no problem

Issue with Recreate Geometry

"Script associated with this tool does not exist."

I get Error 000576: "Script associated with this tool does not exist." whenever I try to run the script tool from the toolbox in ArcMap 10.5.

Converting GNSS Heights from HAE to MSL

I'm am curious if these tools can help me out. I have an esri personal geodatabase (.mdb) of 80,000+ gps records exported from trimble pathfinder office. These records all have heights exported using HAE (Height Above Ellipsoid). I need all of the heights in these records converted to MSL (Mean Sea Level). Can this tool or any other tool out there accomplish this? The only way i've been able to solve this is to re-export each .cor file that is exported into the .mdb and re-create the .mdb using the new exported tables.

Any help would be greatly appreciated!

Sam Hildebrand

Unable to generate token

Have used this previously but now get an error message indicating "Unable to generate token. Invalid username or password. (error code: 400)". Using the same credentials I use for AGO. Not sure what the issue is. Only major difference is I have recently upgraded to pro 2.7.2.

Error 000816 The tool is not valid

Would anyone have any idea as to what might be causing an invalid Project Z tool? As soon as I add any feature class to the Input Features parameter, I get the error warning: Error 000816 The tool is not valid.

My feature class has the GNSS fields added to it and I am using ArcGIS Pro 2.5. I also have ArcGIS for Desktop installed with python 2.7, so perhaps that is causing the issue?

Any help would be appreciated!

Unable to select transformations

I've tried the ProjectZ tool in both ArcMap and ArcPro. Neither allow me to select a transformation. Also, in testing a dataset, the Z values provided only seem to convert units (i.e. meters to feet). Is an ellipsoid to geoid transformation supported (e.g. ITRF 2008 to NAVD88)? The Z values output are only the GNSS "Altitude" field multiplied by 3.28. They're not close to the reported NAVD88 height of a nearby benchmark.

Serious Security Issue

Two of the tools call for an AGOL password, however, the password is displayed as plain text and stored in the GP history. This can be changed by setting the parameter to "String (hidden)", but should be default. In addition, this security issue isn't disclosed in the documentation for this tool, and instructions on how to obscure the password should be clearly communicated to tool users.

arcpy has no attribute Fail

I ran the script "as is" and am receiving an error that says that arcpy has no attribute fail, I change the line in file:add_update_gnss_fields_python_api.py that says arcpy.Fail(e) to print(e) and the script runs but does not update the fields in my arcgis online data.
Any ideas why the fields are not added?
The feature service is set to editable.
Thanks for this script by the way.
ArcPro 2.1.2
ArcGIS Desktop 10.5.1
Python - Installed with ArcPro

Reset Required Fields Failed

I am so glad you are developing this tool. I really hope I can get it to work. I get this error after running it in arcgis pro:

Messages
Start Time: Thursday, July 19, 2018 3:25:35 PM
Running script Reset Required Fields (Python api)...
Parsing Arguments..
Done parsing arguments..
'NoneType' object has no attribute 'type'
Completed script Reset Required Fields (Python api)...
Failed to execute (ResetRequiredFields).
Failed at Thursday, July 19, 2018 3:25:43 PM (Elapsed Time: 7.84 seconds)

Add GNSS Metadata Fields (Pro) - FeatureClass

I keep getting a syntax error for both FeatureClass and Hosted Service "Add GNSS Metadata Fields" ... the solution says make sure the script file is in the right location but when I refer to the folder the toolbox is in, there is no script "Add GNSS Metadata Fields (Pro).py" How do I extract that script out so its referencing the proper script or am I missing a step? I am using ArcGIS Pro 2.0.1

Issue with ProjectZ

When trying to run ProjectZ, as soon as I pick the input feature I get an ERROR 000816 message

Any ideas would be much appreciated

Thanks

Recreate Geometry Z Value not hold

Great script, but when I have a feature class that has a value for a feature height that in the schema is called height, the script wants to use this that field. In ArcPro 2.2, when I select the GNSS recorded "Altitude" field, the window will default back to the first instance of anything that appears to be height or elevation. My thought is the script is grabbing the first instance it see is a Z value, not the one I want it grab.

Maybe there is another step I was not aware when using this script.

Thank you.

SyntaxError line 7

When I attempt to run the tool I get this error:

How do I fix this? Thank You!

Z-Value parameter keeps defaulting to picking a random numeric field

I know that the model is going to attempt to autofill, but even when I go into the parameter and select "Altitude", it will revert the parameter back to a different numeric field. The numeric field it keeps autoselecting, is the first numeric field contained by the feature. It autoselects latitude and longitude fine. Just altitude appears to be autoselecting the first numeric field in the list, and will continue to revert back to it, regardless of what field I tell it to use.

GNSS Metadata

Hello,

Is there anyway to set up a feature class so the LLQ message from the NMEA stream can be stored? I have set local coordinates and a geoid in the receiver and Im trying to get this GNSS metadata stored so I do not have to do a coordinate transformation post processing.

Thanks

Hugh

Not getting a correct elevation value

Yesterday morning I took the receiver out to where one of our elevation monuments is located and collected a point. Came back into the office and ran the Project Z tool on the feature class containing the point I went collect. Plus the Project tool and Add XY Coordinates tool which the tutorial I followed said. It gave me a value very close to what the last calibrated reading for that monument was back in 2013. The reading on this monument from 2013 was 19.39 ft and the reading I got yesterday morning was 19.92 ft

Yesterday evening I went out again to the same monument to collect another point which I ran through the same process (I think) and I got a totally different elevation value than what I got in the morning. 25 ft

So this morning I went to another area to try again and am still getting a different reading than what I initially got.

My steps are as follows.

First I open up my web map in ArcGIS Pro and export the hosted feature class out to a file geodatabase feature class
Then I bring that feature class into the Project Z tool

Then once get that value I use the Project tool to project into a coordinate system we use here and then transform it into a NAVD88 (height)(USft)

Once I have an elevation value, I then use the Add XY Coordinates tool to convert it to ft

Am I missing a step?

I had it working what seemed perfect before

Thanks,

Scott Louque
St James Parish GIS Analyst

ProjectZ runs, but doesn't actually convert elevations

I have been trying to figure out if there is some kind of issue with my inputs for this tool. I am able to run it successfully, but the output Z value has not been changed. The output values are exactly the same as the input values. I have attached an image of the inputs. Maybe there is something that I'm not doing correctly that you'll be able to see. I am using ArcGIS Pro 2.3.2. Any insight as to what is happening?

Thanks!