esri / arcrest

python package for REST API (AGS, AGOL, webmap JSON, etc..)

This repository is no longer being maintained. Please use the ArcGIS Python API.

Site: https://github.com/Esri/arcgis-python-api

ArcREST Version 3.5.x

A latest release(v3.5.9) of ArcREST can be downloaded here: https://github.com/Esri/ArcREST/releases/tag/3.5.9

• If you are using an older version(v3.0.1) of ArcRest, you can find it here: https://github.com/Esri/ArcREST/releases/tag/v3.0.1
• If you are using an older version(v2) of ArcRest, you can find it here: https://github.com/Esri/ArcREST/tree/FinalV2
• If you are using an older version(v1) of ArcRest, you can find it here: https://github.com/Esri/ArcREST/tree/October2014v1.0Final

A set of python tools to assist working with ArcGIS REST API for ArcGIS Server (AGS), ArcGIS Online (AGOL), and ArcGIS WebMap JSON.

This is not a full implementation of the Esri REST API, but we would like to make it, so help out! Please feel free to contribute.

Features

• Add, Delete, Update and Query Feature Services
• Upload attachments to feature services
• Assists in managing and publishing content
• Allows users to control, migrate and update online content
• Manage users on Portal, ArcGIS Server, and ArcGIS Online sites
• Plus additional information not even listed here!

Documentation

The API reference is hosted here. The Esri portal API reference is hosted here

General Help

New to Github? Get started here.

Requirements

• Python 2.7.x/Python 3.4 (https://www.python.org/)
• numpy >= 1.7.1 (numpy is included with ArcGIS default installation)
• Six (https://pypi.python.org/pypi/six) (Six is included in current version)
• pip (https://pip.pypa.io/en/stable/installing/)

Recommended Installation

• ArcPy (optional)
• ArcGIS Desktop 10.2, 10.3, 10.4, 10.5 (optional)
  • If ArcPy is not installed, there will be limited functionality.

Installation

Download from PyPi using pip

pip install arcrest_package

Download a geoprocessing package to install ArcREST!

Download the GeoProcessing Package here: https://github.com/Esri/ArcREST/blob/master/ArcGIS%20Desktop%20Installer/installing_arcrest.gpk
* Note: ArcMap/Catalog is required to use the gpk format.
* Note: this tools can be used to update ArcREST as well
1. Open the GPK in the ArcGIS Desktop Product of your choosing.
2. Run the tool by double clicking on the tool icon.
3. Test the import

• Note: If you have not done so, you may need to add your python install path and scripts folder to your environment variables. In your system PATH environment variable, add both the path to Python and the Python Scripts folder. ex: C:\Python27\ArcGIS10.3;C:\Python27\ArcGIS10.3\Scripts

1. Install requirements
2. run the setup.py. This should copy it to your python's site-package folder.

pip install -r requirements.txt
python setup.py install

Getting Started

Fetch your folders:

import arcrest
from arcresthelper import securityhandlerhelper

config = {'username': 'myusername', 'password': 'myp4ssword'}
token = securityhandlerhelper.securityhandlerhelper(config)
admin = arcrest.manageorg.Administration(securityHandler=token.securityhandler)
content = admin.content
userInfo = content.users.user()
userInfo.folders

Get item metadata:

item = admin.content.getItem(itemId=itemId)
item.title
 u'Streets'

Issues

Find a bug or want to request a new feature? Please let us know by submitting an issue.

Contributing

Esri welcomes contributions from anyone and everyone. Please see our guidelines for contributing.

PyPi

Please see our PyPi page.

To build the wheel - python setup_wheel.py bdist_wheel

Licensing

Copyright 2016 Esri

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

A copy of the license is available in the repository's LICENSE file.

[](Esri Tags: AGS AGOL ArcGIS Server ArcGIS Online Utilities Telecommunications ArcGISSolutions) [](Esri Language: Python)

arcrest's Issues

Samples for calcExpression

I'm trying to update some fields in a featureLayer based on a local feature class (a copy of the hosted layer).
I'm not sure how to create the input of calcExpression for many fields besides:

fl.calculate(where="OBJECTID={} AND SymbolID={}".format(updateRow[3], updateRow[0]), calcExpression={"field" : "field 1", "value" : "{}".format(updateRow[1]),
.....
 "field" : "field 15", "value" : "{}".format(updateRow[15])}

Is the anything within ArcREST to make calling each field name and value more efficient?

I also looked into updateFeatures but I'm not sure how that would work within my main cursor:

with arcpy.da.UpdateCursor(T2, fields_2) as updateRows:
    for updateRow in updateRows:
        # For features with Null TAGs....
        if updateRow[1] == None:
            # Set templist to show available integer TAGs for 1 SiteID at a time.
            templist = SiteIDDict[updateRow[0]]
            # Set Null tag to first available
            updateRow[1] = templist[0]
            # Add Integer to NewTags, insert to local sql server table, update online from sql results, then Remove used tag.
            # print "SiteID: {} UPDATING TAG {}".format(updateRow[0],updateRow[1:])
            try:
                cnxn_SQL = pyodbc.connect(Trusted_Connection='yes', Driver = '{SQL Server Native Client 10.0}', Server = '####', database = '#####')
                cursor_SQL = cnxn_SQL.cursor()
                # get ID required by insert trigger
                propitemid = cursor_SQL.execute("SELECT [PropertyItemID] from [PropertyItems] Where [GIS_name] = ?",updateRow[4]).fetchone()[0]
                # Insert New rows into SQL Server T1
                cursor_SQL.execute("IF NOT EXISTS (SELECT 1 FROM Property WHERE SiteID = ? and Location = ?) INSERT INTO Property(SiteID, Location, Description, CBHL, PropertyQuantity, PropertyItemID, DateEntered, LastChangeDate, Special_Note) values (?,?,?,?,?,?,?,?,?)", updateRow[0], updateRow[0], updateRow[0],updateRow[1],updateRow[4],updateRow[5],updateRow[6],propitemid,updateRow[8],updateRow[9],updateRow[10])
                cnxn_SQL.commit()
                # Get New Primary Key
                newPropID = cursor_SQL.execute("Select PropertyID From Property where SiteID=? AND Location=?", updateRow[0], updateRow[1]).fetchone()[0]
                cnxn_SQL.close()
            except:
                print "failed sql server update"
            # Update Primary Key and TAG for hosted feature
            try:
                fl.calculate(where="OBJECTID={} AND SiteID={}".format(updateRow[3], updateRow[0]), calcExpression={"field" : "TAG", "value" : "{}".format(updateRow[1]), "field" : "PropID", "value" : "{}".format(newPropID)}
            except:
                print "failed hosted feature update"
            # Update local feature class just in case everything else fails.
            try:
                updateRow[2] = newPropID
            except:
                print "Already had a PropertyID"

            templist.remove(updateRow[1])
            updateRows.updateRow(updateRow)

I am only updating 2 fields in the example but want to know if there is a better way to update multiple fields for 1000's of features.

Problem with Date Field containing Null Values

Hello,

This library is fantastic, and it should make working with Server and AGOL via python a lot easier. I am running into a problem that I am hoping you can help me out with. I am trying to add features to a feature service using the "_add_feature_from_feature_class.py" script as a sample. This script works perfectly with most data sources, however my client is working with a dataset that contains null values in the Date field. When I attempt to run this fc_to_features function I receive the following error:

Runtime error
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\agol\common.py", line 948, in fc_to_features
row[fields.index(df)] = int(json.dumps(_date_handler(row[fields.index(df)])))
ValueError: invalid literal for int() with base 10: 'null'

So, I figured I'd populate the field with a date and try again. After populating the field with the value of 07\22\2014, I receive this error:

Runtime error
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\ArcGIS10.2\lib\site-packages\arcrest\agol\common.py", line 948, in fc_to_features
try:
ValueError: invalid literal for int() with base 10: '1406016000000.0'

Is there any way to correct this issue? I am happy to provide a sample dataset.

Kind regards,
Maura

Installation error

I get an error "package directory 'arcrest/manageagol' does not exist. Please help.

zip export issue and credential issues

This is my first attempt at using ArcREST and python to interact with ArcGIS Online content. I am trying to use Andrew's script (http://anothergisblog.blogspot.com/2014/10/using-arcrest-to-export-hosted-feature.html) to download a feature service with related data but am running into issues depending on the parameters I use.

Issue #1 - If I set itemData(f="zip"..., the script successfully downloads a zip file to my C: drive but it is empty. By removing the piece of script that deletes the feature after downloading, I can see the feature is successfully created in my ArcGIS Online content and when downloaded manually, contains all features, attachments, and related data.

Issue #2 - If I set itemData(f="json"..., nothing downloads to my C: drive and I receive the following error:
SSLError: [Errno 1] _ssl.c:499: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

The error is generated at line 44, filePath = exportItem.itemData(f="json", savePath=filePath), which seems weird because once again, the script successfully generates a file geodatabase and has already logged in using the security handler.

Thanks for the help!
See below for code

import arcrest
import uuid
import os
import time

Inputs

itemID = "7754fff2e8144f6b83b96fae3690e7d4"
username = "calebsusername"
pw = "****************"
url = "http://salem.maps.arcgis.com/sharing"
filePath = r"c:\temp"

Logic

Create security handler

shAGOL = arcrest.AGOLTokenSecurityHandler(username, pw)

Connect to AGOL

org = arcrest.manageorg.Administration(url=url, securityHandler=shAGOL)

Grab the user's content (items)

content = org.content
usercontent = content.usercontent(username=username)

Create a export item with a random name using UUID

and export it to a zipfile

item = content.item(itemID)
uc = content.usercontent(username=item.owner)
result = uc.exportItem(title="%s" % uuid.uuid4().get_hex(),
itemId=itemID,
exportFormat= "File Geodatabase",
exportParameters=None)
exportedItemId = result['exportItemId']
jobId = result['jobId']
exportItem = content.item(itemId=exportedItemId)

Ensure the item is finished exporting before downloading

status = usercontent.status(itemId=exportedItemId, jobId=jobId, jobType="export")

while status['status'].lower() == "processing":
time.sleep(3)
status = usercontent.status(itemId=exportedItemId,
jobId=jobId,
jobType="export")
filePath = exportItem.itemData(f="zip", savePath=filePath)

Erase the exported item to clean up

AGOL

usercontent.deleteItems(items=exportItem.id)
del exportItem
print 'finished!'

arcrest.agol.layer.FeatureLayer.query fails when returnGeometry=False

Executing
fl.query(where="1=1",out_fields='OBJECTID_1,ZIPNUM,OUTAGE,SERVED,UP,PercentOut,Updated',returnGeometry=True) succeeds.
But
fl.query(where="1=1",out_fields='OBJECTID_1,ZIPNUM,OUTAGE,SERVED,UP,PercentOut,Updated',returnGeometry=False) fails:

Traceback (most recent call last):
  File "<pyshell#37>", line 1, in <module>
    fl.query(where="1=1",out_fields='OBJECTID_1,ZIPNUM,OUTAGE,SERVED,UP,PercentOut,Updated',returnGeometry=False)
  File "C:\Python27\ArcGIS10.3\lib\site-packages\arcrest\agol\layer.py", line 756, in query
    return FeatureSet.fromJSON(json.dumps(results))
  File "C:\Python27\ArcGIS10.3\lib\site-packages\arcrest\common\general.py", line 584, in fromJSON
    features.append(Feature(json_string=feat, wkid=wkid))
  File "C:\Python27\ArcGIS10.3\lib\site-packages\arcrest\common\general.py", line 78, in __init__
    json_string['geometry']['spatialReference']  = {"wkid" : wkid}
KeyError: 'geometry'

This is the service involved and the query is against layer 0:
http://services.arcgis.com/FK2lj6e2iAhV2fHW/arcgis/rest/services/Ameren/FeatureServer

Feature set is empty when querying AGOL hosted feature service with table

Consider a hosted feature service with Feature Layer and Table. Trying to run the sample https://github.com/Esri/ArcREST/blob/master/samples/query_agol_service.py (with a public example feature service URL: http://services.arcgis.com/eSnyVpqwqWBADfzp/ArcGIS/rest/services/Household_size_for_water_quantity/FeatureServer) will return a feature set with fields (schema), but with no features (data).

ldf.addFilter(8, where="1=1",outFields="*")
recordSet = fs.query(layerDefsFilter=ldf,returnCountOnly=False,returnGeometry=True)

>>> recordSet[0].features 
>>> [] 

ArcREST addItem error in ArcGIS Online

When accessing the added item in AGOL a general error message appears i.e
It is a CSV; the error that is returned

Error retrieving file from http://myorg.maps.arcgis.com/sharing/content/items/a935aa00f237477ba9912ca2bd32b79d/data?token=AYJwMLSoVIlx7LzbY_vcFNMb9yZ6wyutOARMXSKQd21g17xNE96bqzMBns97ySZlYr2nXVhAHBWxldT57NKS5HCL1i8xFyDYVLPBoVzZqer1VzcbeOVSBSdo8xzoCyzWRY8b5iNRKhow4oeD6Mlg5w..

The item adds to AGOL but does not allow users to publish or download.
Perhaps this has something to do with issue # 52.

import arcrest

if name == "main":
username = ""
password = ""
portalId = ""
url = ""
thumbnail_url = ""
kml_path = "C:\Users\GeoffreyWest\Desktop\aCSV.CSV"
securityHandler = arcrest.AGOLTokenSecurityHandler(username,
password)
# Create the administration connection
#
admin = arcrest.manageorg.Administration(url, securityHandler)
# Connect to the portal
#
portal = admin.portals(portalId)
# Access the content properties to add the item
#
content = admin.content
# Provide the item parameters
#

itemParams = arcrest.manageorg.ItemParameter()
itemParams.thumbnailurl = thumbnail_url
itemParams.title = "aCSV"
itemParams.type = "CSV"
itemParams.tags = "aCSVTest,CSV,test"
#   Enter in the username you wish to load the item to
#
usercontent = content.usercontent(username=username)
print usercontent.addItem(filePath=kml_path,
                          itemParameters=itemParams)

Geometry Filter missing inSR

When a Geometry filter is applied to a layer, it is expecting a parameter with name inSR from the gemoetryFilter

if not geometryFilter is None and
isinstance(geometryFilter, filters.GeometryFilter):
gf = geometryFilter.filter
params['geometry'] = json.dumps(gf['geometry'])
params['geometryType'] = gf['geometryType']
params['spatialRelationship'] = json.dumps(gf['spatialRel'])
params['inSR'] = json.dumps(gf['inSR'])

Make ArcREST available to machines without ArcGIS Desktop

For my specific case, I will like to use the API to read all the items within an organization and their attributes.

using updateFeatures to update attributes from dictionary

I can update features from the updateFeatures REST page with the following:

/* This is a valid updateFeatures parameter */
[{"attributes" : {"OBJECTID": 3325, "Name" : null}},
{"attributes" : {"OBJECTID": 152504, "Name" : "Joe Smith"}}]

ArcREST's layer function to updateFeatures only appears to update from fully built feature objects. No error is thrown when the features parameter only contains attributes. updateFeatures succeeds but only returns {'updateResults': []} instead of {'updateResults': [{'objectId': 3325, 'globalId': None, 'success': True}, {'objectId': 152504, 'globalId': None, 'success': True}]}

Here is my workaround, but it would be nice to allow attribute updates from the function:

"""
dict1 - is a dictionary of hosted features. Values have been changed to match a different data source.
list_CustomerIDs -  is a list. It lists the dict1 keys which are used to update specific features
attributes - is a list of dicts of individual feature attribute updates
"""
attributes = [{"attributes":{"OBJECTID":dict1[key][4],"Name":dict1[key][0],"CustomerID":dict1[key][1],"Address":dict1[key][3]}} for key in list_CustomerIDs]
# setup initial params dict
params = {"f" : "json", "rollbackOnFailure" : True}
result = None
 # There should be a check here to make sure the attributes has at least the feature layer OID or GlobalID
if len(attributes) > 0:
        #add features key:values
        # using json dumps here change syntax from python to json (None to null, escape special characters, etc...)
        params['features'] = json.dumps(attributes)
        params['token'] = fl._token
        uurl= fl.url + "/updateFeatures"
        result = fl._do_post(url=uurl, param_dict=params, proxy_port=fl._proxy_port, proxy_url=fl._proxy_url)
        print result
        # {'updateResults': [{'objectId': 152504, 'globalId': None, 'success': True}, {'objectId': 3325, 'globalId': None, 'success': True}]}

It would be nice to just call:

fl.updateFeatures(attributes)

My code above is a little too workflow specific, but updateFeatures could take a list of dicts of field:attribute like:

attributeUpdates = [{"OBJECTID":3325,"Name":None},{"OBJECTID":152504,"Name":"Jeff's Grocery Store"}]
#build features param for post
params['features'] =[{"attributes":{fields:values for fields,values in row.iteritems()} for fields,values in row.iteritems()} for row in attributeUpdates]
#Results in [{'attributes': {'Name': None, 'OBJECTID': 3325}}, {'attributes': {'Name': "Jeff's Grocery Store", 'OBJECTID': 152504}}]

Hope this is useful

An exception thrown when trying to query table within AGOL feature service

Consider a hosted feature service with two objects - layer id 0 is of type Feature Layer, and layer id 1 is of type Table. Trying to run the sample https://github.com/Esri/ArcREST/blob/master/samples/query_agol_service.py will throw an exception:

features.append(Feature(feat, jd['spatialReference']['latestWkid']))
KeyError: ('spatialReference',)

It seems as it cannot find spatial reference for the table (when trying to return a featureset from a JSON string - fromJSON method of FeatureSet). Is it a bug? When working with a feature service with feature layers, accessing them is no problem.
It breaks on the last line:

ldf.addFilter(1, where="1=1")
print fs.query(layerDefsFilter=ldf,returnCountOnly=False)

ArcREST PublishCSVParameters error

I am consistently receiving this error when trying to publish a CSV as a hosted service.

{u'error': {u'message': u'Unable to publish item.', u'code': 400, u'details': [u"'publishParameters' must be specified."]}}

The script that I am passing is below;

import arcrest

if name == "main":
username = ""
password = ""
portalId = ""
url = "http://www.arcgis.com/sharing/rest" #should this be my organization or /sharing/rest??
itemId = ""
securityHandler = arcrest.AGOLTokenSecurityHandler(username,
password)
admin = arcrest.manageorg.Administration(url, securityHandler)

portal = admin.portals(portalId)


content = admin.content

layerInfo = {
"name": "CSV File",
"type": "CSV",
"fields": [
{
"name": "X_COR",
"type": "esriFieldTypeDouble",
"sqlType": "sqlTypeInteger",
"nullable": "false",
"editable": "false",
"domain": "null"
},
{
"name": "Y_COR",
"type": "esriFieldTypeDouble",
"nullable": "false",
"editable": "false",
"domain": "null"
}
],
"types": [],
"templates": [],
"capabilities": "Query,"
}

publishParameters = arcrest.manageorg.PublishCSVParameters(name="csvTest", locationType="none", layerInfo=layerInfo)

usercontent = admin.content.usercontent(username=username)
statusinfo = usercontent.publishItem(fileType="csv", itemId=itemId, publishParameters=publishParameters)

print statusinfo

GetItems GetFolder fails when user does not have any items

In org, created a new user
User has nothing in the “My Content” component of AGO.
Errors out on the “get content” component
It appears that if a user doesn’t have any folders
If I create a folder, any folder with any name, the script runs find.

CreateService Example and Support

Add support to copy one FeatureService from one AGOL to another AGOL. I believe CreateService is what is required.

http://resources.arcgis.com/en/help/arcgis-rest-api/index.html#/Create_Service/02r30000027r000000/

Overwrite CSV file Issue

When referencing closed issue #53, I receive a 409 error which indicates the actual CSV that I am attempting to add exists. While ip.overwrite is set to true.

{'error': {'message': "Item 'csp_encampment_4_20_15.csv' already exists.", 'code': 409, 'details': []}}

Using AGSTokenSecurityHandler with featurelayer

This code snippet fails

ts = AGSTokenSecurityHandler(_apiUserName,_apiPassword,URL_TOKEN)
imageFS = ags.featureservice.FeatureService(URL_PHOTO_ADMIN, ts)

the error is related to the referer_url property of securityHandler in featureservice.py:

def __init__(self, url, securityHandler=None,
                 initialize=False, proxy_url=None, proxy_port=None):
        """Constructor"""
        self._proxy_url = proxy_url
        self._proxy_port = proxy_port
        self._url = url
        if securityHandler is not None and \
           isinstance(securityHandler,
                      security.AGSTokenSecurityHandler):
            self._securityHandler = securityHandler
        if not securityHandler is None:
            self._referer_url = securityHandler.referer_url
            self._token = securityHandler.token
        elif securityHandler is None:
            pass

From what I can tell, AGOLTokenSecurityHandler has the referer_url property and works ok. Also, If I wrap the offending line in featureservice.py in a try statement, I can move ahead in my script and use the feature service.

I'm not sure if featureservice.py should be changed to not assume referer_url is a property or if AGSTokenSecurityHandler should have the referer_url property added.

Replace Tags and Multiple webmaps

If I have multiple web maps that all share the same services, replace tags are only used to filter the web map, it would be nice to only have to have the replace tags for services once and replaced throughout all webmaps. For example: I am creating 5 maps filtered by surveys assigned to crew. In the json I currently have to a prerequired service, which is not published in the StageApp script. Because of the a user will manually having to update a prerequisite service 5 times.

Create Tile Service Access Classes for AGOL

Create Tile Service Access Classes for AGOL

See: http://resources.arcgis.com/en/help/arcgis-rest-api/index.html#/Administering_ArcGIS_Online_hosted_services/02r30000022t000000/

Enhancement request for publish_to_agol: Allow publishing of tiled services.

Enhancement request for publish_to_agol: Allow publishing of tiled services.

Sample code:

import arcrest
username, password, mxd, serviceName = "username", "password", "pathtoMXD", "serviceName"
serviceType = "MapServer" #To publish tiles
agoInstance = arcrest.agol.admin.AGOL(username, password)
agoInstance.publish_to_agol(mxd, service_name = serviceName, type = serviceType)

special character retrieval in item data

hi,

in my workflow I create new operations views based on a template AGOL item. In run into an issue where special characters are encoded while reading the json.

When I create the new operations view object based on the template, I would like to store the euro-sign in the json data like the snippet below. Not in encoded form.
The encoded character storage will lead to problems in windows dashboard application.

This is how the euro-sign is stored in my template operations view definition:
{
"caption": "Kosten (€)",
"clickFeatureAction": {
"type": "highlightAction",
"updateExtent": "pan"
}, ( ... )

Any way to get this approach working in ArcRest?

I have been able to resolve this by retrieving the template item data (using requests library) and uploading the item data with json.dumps(data, ensure_ascii=False).
Would be nice to get this integrated in ArcRest as well.

Best Regards,
Laurens

OAuth2 for Layer in ArcMap

Is there an example of how to pass signin credentials for arcmap?
My script makes pdf export from feature services in an mxd. Currently, I must manually sign in on the arcmap credentials popup and keep an instance of ArcMap open. Otherwise, my script fails. I'm currently using AGOLSecurityHandler for other purposes but I'm not sure where to put OAuthSecurityHandler when opening an mxd. I've got a client_id and client_secret made already.

getting users list is very slow

users function in Portals class (arcrest/manageorg/_portals.py) takes about 30 secs.

It seems that urllib2.urlopen(request).read() function waits some response from AGOL...

Enhancement: Rename class and missing class in setup

Rename ManageAGOL to ManageOrg
Setup is missing HostedServices

Error Updating Itme

Andrew or Mike,

I'm running into an issue on UpdateItem where it returns the following error:

Message File Name Line Position
Traceback
82
TypeError: updateItem() got an unexpected keyword argument 'itemParameters'

Here is my code...

import datetime, os, sys, time, traceback, uuid
from arcrest import manageorg
import arcrest

Start Logging

logMessage("Exporting " + FStitle + "feature service to a feature collection")

Organization URL *Fianl Script will be configurable

baseURL = "http://esridot.maps.arcgis.com/sharing/rest"

portal credentials *Fianl Script will be configurable

username = ""
pw = ""

fsID

fsID = '7d3314cd5a4842abbe6c5b088bdcf6e8'

Item Title *Fianl Script will be configurable

FStitle = "DriveTexas_FS"
FCtitle = "DriveTexas_FC"
FCtemp = "DriveTexas_tempFC"

Item Description Info *Final script will be configurable

tags = "Driving Conditions,Road Conditions,Weather Conditions,Construction,TxDOT,Texas"
snippet = "Current driving conditions"
description = "Current driving conditions for roads and highways managed by Texas Department of Transportation"
licenseInfo = "Public Information licensed under the Texas Public Information Act."
thumnail = r'D:\Libraries\Box Sync\Projects\Development\Python\511Sync\DriveTexas.jpg'

fcType = "Feature Collection"

Publish FS Logic

Create security handler

sh = arcrest.AGOLTokenSecurityHandler(username, pw)

Connect to AGOL

org = manageorg.Administration(url=baseURL, securityHandler=sh)

content = org.content
usercontent = content.usercontent(username)
if isinstance(usercontent, manageorg.administration._content.UserContent):
pass
result = usercontent.exportItem(title=FCtitle,
itemId=fsID,
exportFormat=fcType,
exportParameters={"layers" : [{"id" : 1},{"id" : 0}]})

exportedItemId = result['exportItemId']
jobId = result['jobId']
exportItem = content.item(itemId=exportedItemId)

Ensure the item is finished exporting before downloading

status = usercontent.status(itemId=exportedItemId, jobId=jobId, jobType="publish")

while status['status'].lower() == "processing":
time.sleep(3)
status = usercontent.status(itemId=publishedItemId,
jobId=jobId,
jobType="publish")

Set the itemParameters

itemParams = manageorg.ItemParameter()
itemParams.title = FStitle
itemParams.tags = tags
itemParams.thumbnail = thumnail
itemParams.description = description
itemParams.tags = tags
itemParams.snippet = snippet

productionFC = usercontent.updateItem(itemParameters=itemParams)

FeatureLayer Editing on AGS Layer

Hello,

Do you have any plans to include update, delete and add operations to the AGS featurelayer class?

Thanks,
Maura

Additional Proxy Support

Add in support for proxies with username/password:

Somthing like:
pxy_srvr = config.get('PROXY', 'SERVER')
pxy_port = config.get('PROXY', 'PORT')
pxy_user = config.get('PROXY', 'USER')
pxy_pass = config.get('PROXY', 'PASS')
http_proxy = "http://" + pxy_user + ":" + pxy_pass + "@" + pxy_srvr + ":" + pxy_port
https_proxy = "http://" + pxy_user + ":" + pxy_pass + "@" + pxy_srvr + ":" + pxy_port
ftp_proxy = "http://" + pxy_user + ":" + pxy_pass + "@" + pxy_srvr + ":" + pxy_port
proxyDict = {"http" : http_proxy, "https":https_proxy,"ftp": ftp_proxy}

ArcREST/src/arcrest/agol/__init__.py No Module named helperservices

Throwing an error, no module named helperservices

Update existing feature service with local data

Can you create an example how to automate updating a Feature Service in AGOL with a local GDB.
I suppose it should be in this form:

• Drop all features online
• insert al local features into the feature service

if have a lot of features this would create a lot of requests, so is their a bulk way to upload a lot of features ?

List field names of featureLayer

Is there something similar to arcpy.ListFields? I could see this being helpful when creating feature objects for updateFeatures. Perhaps returning field.name from featureLayer.fields?

Bug: GeometryFilter Runtime error - argument mismatch

I've encountered a bug when attempting to build a GeometryFilter to use as part of a query against a Feature layer. Below is a section of the code that identified the behaviour. The section that is commented out at the bottom of the script will through the error below it if executed.

I've shared the full snippet with Mike, including credentials to services it can be tested against.

Thanks in advance for dealing with this one quickly!
Brian

url2 = "XXXX" #Url to AGOL-hosted polygon feature service
Regionfl = layer.FeatureLayer(
url=url2,
securityHandler=agolSH,
proxy_port=proxy_port,
proxy_url=proxy_url,
initialize=True)
result = Regionfl.query(where="1=1",
out_fields='*',
returnGeometry=True)

for res in result:
    """gfilter = GeometryFilter(res.geometry,spatialFilter='esriSpatialRelIntersects')
    subresult = Incidentfl.query(where="1=1",
                                 out_fields='OBJECTID,CUSTOMERS,DEVICETYPE',
                                 geometryFilter=gfilter)"""

Error Response:
Runtime error
Traceback (most recent call last):
File "", line 51, in
File "C:\Python27\ArcGIS10.2\Lib\site-packages\arcrest\common\filters.py", line 77, in init
if isinstance(geomObject, ) and
TypeError: isinstance expected 2 arguments, got 1

Sample

A sample to query a Feature Layer, determine what features have attachments, update a field with a value and push the updates back to the layer

Documentation for editorTrackingInfo

Hello,

Do you have any information on the editorTrackingInfo function and the information it returns?

Kind regards,
Maura

itemData

I'm seeing an issue where itemData is exporting a completely different set of JSON than I am getting when I go through the Sharing API.

The token on this will probably expire but I get one set of JSON delivered through my browser going directly to this url.

http://esridot.maps.arcgis.com/sharing/rest/content/items/a10c31fb361644ab921a4df095a88da8/data?token=kQizaJIR4VvTvIzSYBbgwCGjA4ASVeodPdiIDcPzEVrrc14r4ciBqW2ljNAwXuGELyn1f1PuVKg6jh1JgJoJ_WeW2q-SBtjt1XT3k09C5uhEm1S2LFKQyy6qXP0DZMlJDJtZVWOwQkMWmfAM6HTnzw..&f=json

I get an entirely different set of JSON when I use the code below. Both make the same exact get request according to Wireshark

Here is the code I wrote to get a json dump from a Feature Collection
from arcrest import manageorg
import arcrest

Application Program Title

appProgram = "Drive Texas"

Organization URL *Fianl Script will be configurable

baseURL = "http://esridot.maps.arcgis.com/sharing/rest"

portal credentials *Fianl Script will be configurable

username = "username"
pw = "password"

tempFC_itemID = 'a10c31fb361644ab921a4df095a88da8'
proFC_itemID = 'bb32cf8c13f44610860dc685ec5dd977'

tags = "Driving Conditions,Road Conditions,Weather Conditions,Construction,TxDOT,Texas"
snippet = "Current driving conditions"
description = "Current driving conditions for roads and highways managed by Texas Department of Transportation"
licenseInfo = "Public Information licensed under the Texas Public Information Act."
thumbnail = r'D:\Libraries\Box Sync\Projects\Development\Python\511Sync\DriveTexas.jpg'

def main():

sh = arcrest.AGOLTokenSecurityHandler(username=username, password=pw)
admin = manageorg.Administration(url=baseURL, securityHandler=sh)
content = admin.content
item = content.item(tempFC_itemID)
usercontent = content.usercontent(username)

exportItem = content.item(itemId=tempFC_itemID)

#JSON Export Directory
print exportItem
print exportItem.itemData(f="json")

calculate field to null?

I can't figure out the syntax for setting values to null.
I've tried:

fl.calculate(where="CustomerID=168453", calcExpression={"field" : "CompanyName_1", "value" : null})
fl.calculate(where="CustomerID=168453", calcExpression={"field" : "CompanyName_1", "value" : NULL})
fl.calculate(where="CustomerID=168453", calcExpression={"field" : "CompanyName_1", "value" : <Null>})
#all errors as expected

fl.calculate(where="CustomerID=168453", calcExpression={"field" : "CompanyName_1", "value" : 'null'})
fl.calculate(where="CustomerID=168453", calcExpression={"field" : "CompanyName_1", "value" : '<Null>'})
#return string values

fl.calculate(where="CustomerID=168453", calcExpression={"field" : "CompanyName_1", "value" : None})
# {'error': {'message': 'Unable to calculate fields.', 'code': 400, 'details': ['Object reference not set to an instance of an object.']}}

Thanks

AttributeError: 'module' object has no attribute 'AnalyzeParameters'

See issue # 53, when using the provided code sample the error returns,
analyzeParams = arcrest.manageorg.AnalyzeParameters()
AttributeError: 'module' object has no attribute 'AnalyzeParameters'

code that I am using;

import arcrest

if name == "main":
username = ""
password = ""
csv_file = r"C:\Users\GeoffreyWest\Desktop\bing_output.csv"

sh = arcrest.AGOLTokenSecurityHandler(username, password)
admin = arcrest.manageorg.Administration(securityHandler=sh)
content = admin.content
usercontent = content.usercontent(username=username)
ip = arcrest.manageorg.ItemParameter()
ip.title = "BingParameterTest"
ip.type = "CSV"
ip.tags = ""
ip.overwrite = True
ip.thumbnailurl = ""
res = usercontent.addItem(itemParameters=ip,
                          filePath=csv_file, overwrite= True)


print res


itemId = res['id']
print itemId
#  Now you need to analyze the item to publish it
#
featureContent = content.featureContent
analyzeParams = arcrest.manageorg.AnalyzeParameters()
analyzeResult = featureContent.analyze(itemId=itemId, analyzeParameters=analyzeParams)
#  Published based off the analyze function
#
if 'publishParameters' in analyzeResult:
    pp = analyzeResult['publishParameters']
    publishParams = arcrest.manageorg.PublishCSVParameters(name=pp['name'],
                                           locationType=pp['locationType'],
                                           layerInfo=pp['layerInfo'],
                                           latitudeFieldName=pp["Y_COR"],
                                           longitudeFieldName=pp['X_COR'])
    print usercontent.publishItem(fileType="csv",
                                  publishParameters=publishParams,
                                  itemId=itemId, overwrite = True)
print 'finished'

returnExtentOnly=True parameter

Could this be added to layer query please? It was just added at 10.3.
It would be perfect for an exportToPDF tool based setting extent to the where clause.
Thanks

arcrest.manageags.administration.Log.query() returns all log entries regardless of startTime and endTime parameters

I am having an issue using the arcrest.manageags.administration.Log class. I think there may be an issue with line 1268 where the params['startTime'] = endTime.strftime("%Y-%m-%dT%H:%M:%S"). Seems like it should be params['endTime'] = endTime.strftime("%Y-%m-%dT%H:%M:%S"). In any event the function seems to be returning all log entries..

Group Layers in Map Service causes failure - BUG

This code:

import arcpy, arcrest
from arcrest.ags import MapService
if name == "main":
try:
url = "http://sampleserver6.arcgisonline.com/arcgis/rest/services/Water_Network/MapServer"
fs = MapService(url=url)
print fs.currentVersion
except ValueError, e:
print e

Fails with this error:
Traceback (most recent call last):
File "C:/Users/scot5141/Documents/GitHub/OpenData-MapServiceLayerRegistrar/simpleTest.py", line 8, in
print fs.currentVersion
File "C:\Python27\ArcGIS10.3\lib\site-packages\arcrest\ags\mapservice.py", line 168, in currentVersion
self.init()
File "C:\Python27\ArcGIS10.3\lib\site-packages\arcrest\ags\mapservice.py", line 116, in __init
proxy_url=self._proxy_url)
File "C:\Python27\ArcGIS10.3\lib\site-packages\arcrest\ags\layer.py", line 785, in __init
self.init()
File "C:\Python27\ArcGIS10.3\lib\site-packages\arcrest\ags\layer.py", line 795, in __init
proxy_port=self._proxy_port))
File "C:\Python27\ArcGIS10.3\lib\json__init.py", line 338, in loads
return _default_decoder.decode(s)
File "C:\Python27\ArcGIS10.3\lib\json\decoder.py", line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
TypeError: expected string or buffer

Publish Service

Hey guys, I'm running into an issue with Publish Item... doc found here:
http://resources.arcgis.com/en/help/arcgis-rest-api/index.html#/Publish_Item/02r300000080000000/

I get the following error:
PYTHON ERRORS:
Traceback info:
File "D:\Libraries\Box Sync\Projects\Development\Python\511Sync\511Sync.py", line 130, in publishFeatureService
result = usercontent.publishItem(fileType="fileGeodata", publishParameters=publishParams, itemId=gdbID)

Error Info:
dictionary update sequence element #0 has length 1; 2 is required

Soem of my publishParameters are variables
publishParams = arcrest.manageorg.PublishFGDBParameter(name=FStitle, description=description, maxRecordCount=-1,
copyrightText=licenseInfo, layerInfo=None, targetSR=102100)

And my itemID references an uploaded fileGDB.

Am I missing a required parameter? I think I've got my bases covered.

Samples with Toolbox for v2.0.100

Provide better samples with some common workflows for publishes, managing and modifying hosted online content on either portal or AGOL.

Compare+Update Oldest Of Two Or Local Server Replica Sample

Thank you for creating this great tool! At the moment I am using this to delete all features from a hosted service and add all features from a local master gdb. The script seems to fail around feature number 1600 when I attempt to add thousands of features at once. However, I think it is a REST issue due to timing out.

I'm having a tough time combining some of the samples to match local and hosted features that are both being edited. Perhaps this isn't the best workflow or you are already making a "Create_And_Sync_Local_Replica" Sample.
There is a master feature class with an incremental uniqueidentifer field stored in a SQL server. Editor tracking was enabled through arccatalog. A nearly identical hosted feature service with editor tracking exists.

Here is a layout of changes to the feature class or feature layer are based mainly on the uniqueidentifier and edit date fields:

1. query feature layer and join to local feature class by uniqueidentifier. Include all hosted features even if uniqueidentifier is null.
2. if (or foreach/while) hosted feature editor date > local edit date: update local feature attributes and shape.
3. if a feature from hosted layer has a null uniqueidentifier: add feature to local feature class. Then delete the hosted feature.
4. Query local feature class for all uniqueidentifiers not in hosted feature layer and add features to online.
5. Make a new join with arcpy including all local features and any matching hosted features.
6. update all hosted feature attributes and shapes where local edit date > hosted editor date.

The last step may cause some unnecessary updates to hosted features from step 2, but I'm unsure what other options would do when offline users sync.

Sorry for the long read. ArcREST seems like the best solution for this until the ArcGIS platform or REST API supports feature service management similar to distributed geodatabase tools found in desktop.

Thank, Davin

typo in "attribute not implemented"

Instances of implemented in /src/arcrest/ are spelled implmented

Portal Token Security Handler

So, for the url for this Class, are we suppose to provide the Rest path, or the Portal path. The sample url shows www.arcgis.com (which is AGOL), but I have tried both the Rest path and Portal path for an internal portal. and both are triggering errors. Hopefully I am doing this right. I am using your sample from ArcREST/samples/generate_portal_server_token.py

error on line: line 30
error in file name: C:\py\PortalManager.py
with error message: SSLError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

Enable logging v2.0.100 and beyond

Enable logging within ArcREST to provide meaningful feedback on errors and warning.

Erase all items Failed to execute

Hi,

I use ArcRest Toolbox. I have a problem with Erase all items tool since several month. i have this message
scriptDeleteUserContent...
error on line: line 73
error in file name: C:\Users\jpdeboos\Documents\ArcGIS\ArcREST-July15Release\tools\src\deleteUserContent.py
with error message: KeyError: 'items'
Completed script scriptDeleteUserContent...
Failed to execute (scriptDeleteUserContent).

I try with the new version (july15) but it's the same.

Someone has a solution ?

Test Scripts

In order to ensure the integrity of the package, better test scripts need to be developed to ensure that functionality does not break.

Sample To Copy/Move a Feature Service with Features

The idea to would be to make an exact copy of an existing feature service with features and attachments intact. The status of publishing would have to be checked so the new service layers are ready to receive the copied features/attachments.
The current feature service, admin connection (arcrest.manageorg.Administration), and new feature service name would be main inputs. This would allow services to be moved to an entirely different portal (such as one for a collaborating organization, a client, or a technical support environment)
The service to service copy is the main request. However, some optional parameters would make it a powerful sample for more than myself:

#what the main sample function may look like
service_to_service(current_fs, admin_connection, new_fs=None,
overwrite=True,
delete_original_service=False,
include_features=True,
include_attachments=True)

I'm stuck on what is needed from the existing service in order to publish a copy. Aside from this, it's mostly a combination of samples that already exist.
Could you please add a sample?
Thank you!

Limits on Posts to Feature Service in AGOL

Is there a known limit on how many features can be posted to a feature service?

Also, when running the tool, I seem to recall if I post 500, it prints that 200 are posted, but if I verify using Pro to Select the data in AGOL, 500 have in fact been posted. For my next update I will take a screen shot of this behavior.

UpdateItem & PublishItem

Andrew, while building this script to hopefully resurrect the ArcGIS for State Government 511 app, I found an two issues that I was able to fix myself but I wanted to bring to your attention if you would prefer to upload the fix.

In the rest API UpdateItem supports Async=True to asynchronously update a file. Currently that is missing from the current version of ArcREST. I was able to add it to my version however and am getting the results I need.

I also noticed that PublishItem is missing Overwrite = True which allows me to take the File GDB I updated in the previous step with new data and push that to the PublishItem Operation. I also updated the ArcRest site package I have to reflect this change in file geodatabase.

So for now I have what I need but wanted to make you aware in case others need to perform similar workflows to what I am doing.

Port to Python 3

There is a need to port ArcREST to Python 3 to support ArcGIS Pro users.

I'm looking for porters to help us out! Anyone interested respond of this thread.

Andrew