Issue

Generated Java language bindings fail to install using mvn install with:

[ERROR] /Volumes/Git/swagger-codegen/isi_sdk_java_7_2/src/main/java/io/swagger/client/model/SmbSettingsGlobalSettingsAuditGlobalSaclItem.java:[73,4] error: variable DELETE_CHILD is already defined in enum PermissionEnum

Analysis

The root of the problem appears to be the PAPI GET_output_schema for /1/protocols/smb/settings/global where delete_child is listed twice in the audit_global_sacl/permission enum.

        "audit_global_sacl": {
          "items": {
            "type": "object", 
            "description": "Specifies properties for a SACL Access Control Entry.", 
            "properties": {
              "flags": {
                "enum": [
                  "successful", 
                  "failed"
                ], 
                "type": "string", 
                "description": "Determines if audit is performed on success or failure.", 
                "required": true
              }, 
              "permission": {
                "items": {
                  "enum": [
              <snip>
                    "delete_child",  <== appears here
              <snip>
                    "delete_child", <== and again here
              <snip>
                  ], 
                  "type": "string", 
                  "description": "Specifies filesystem access permissions."

This results in generated Java code that fails to compile:

  public enum PermissionEnum {
    <snip>
    DELETE_CHILD("delete_child"),
    <snip>
    DELETE_CHILD("delete_child"),
    <snip>

Steps to reproduce:

Generate swagger spec file:
components/create_swagger_config.py -i <IP for 7.2.1 cluster> -u root -p a -e excluded_end_points_7_2.json -o output_7_2.json

Generate Java language bindings:
java -jar modules/swagger-codegen-cli/target/swagger-codegen-cli.jar generate -i output_7_2.json -l java -o isi_sdk_java_7_2

Attempt to install the generated bindings:
cd isi_sdk_java_7_2
mvn install

Among some other errors you will see:
[ERROR] /Volumes/Git/swagger-codegen/isi_sdk_java_7_2/src/main/java/io/swagger/client/model/SmbSettingsGlobalSettingsAuditGlobalSaclItem.java:[73,4] error: variable DELETE_CHILD is already defined in enum PermissionEnum

While trying to import the library with python3.7, the following error occurs:

    import isi_sdk_8_1_0
  File "C:\Users\ricej4\AppData\Roaming\Python\Python37\site-packages\isi_sdk_8_1_0\__init__.py", line 19, in <module>
    from isi_sdk_8_1_0.api.antivirus_api import AntivirusApi
  File "C:\Users\ricej4\AppData\Roaming\Python\Python37\site-packages\isi_sdk_8_1_0\api\__init__.py", line 6, in <module>
    from isi_sdk_8_1_0.api.antivirus_api import AntivirusApi
  File "C:\Users\ricej4\AppData\Roaming\Python\Python37\site-packages\isi_sdk_8_1_0\api\antivirus_api.py", line 129
    async=params.get('async'),
        ^
SyntaxError: invalid syntax

The HDFS protocol settings and Swift protocol settings and some other settings do not accept a zone argument through the SDK. Also some other protocols or auth settings are missing arguments.

The Python bindings packages distributed on PyPI are vulnerable to a form of dependency confusion. Because each bindings version is released under a different project in PyPI, anyone could masquerade as the next unreleased version by creating their own project on PyPI. For example, isi-sdk-9-0-0 is the latest released version today, but someone could just create a isi-sdk-9-1-0 project.

The most common setup is for one project that has many release versions under it. There are more options as well.

See also an article from a vendor in the build artifact industry.

When using the API to create or update the duration field of a snapshot schedule the store value is not the as the supplied value.

Setting 608400 will actually result in 694800 being returned when querying the schedule at a later time.

The actually value is always bigger than the set value. When using the WebUI it is possible to set 608400, when querying the value with the API I do get the correct result.

As far as I can tell the duration that is send via the WebUI is the same as the one I send to the SDK/API.

Tested against 8.0.1.2

Ran basic NFS export create test, just created a new export with /ifs/data like in the test example, and I get:

Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/site-packages/isi_sdk/apis/protocols_api.py", line 445, in create_nfs_export
callback=params.get('callback'))
File "/usr/local/lib/python2.7/site-packages/isi_sdk/api_client.py", line 322, in call_api
response_type, auth_settings, callback)
File "/usr/local/lib/python2.7/site-packages/isi_sdk/api_client.py", line 149, in __call_api
post_params=post_params, body=body)
File "/usr/local/lib/python2.7/site-packages/isi_sdk/api_client.py", line 358, in request
body=body)
File "/usr/local/lib/python2.7/site-packages/isi_sdk/rest.py", line 209, in POST
body=body)
File "/usr/local/lib/python2.7/site-packages/isi_sdk/rest.py", line 177, in request
raise ApiException(http_resp=r)
isi_sdk.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Date': 'Wed, 28 Feb 2018 17:56:24 GMT', 'Content-Length': '230', 'Content-Type': 'text/html; charset=iso-8859-1', 'Server': 'Apache/2.2.34 (FreeBSD) DAV/2 mod_ssl/2.2.34 OpenSSL/1.0.2k-fips'})
HTTP response body:

Not Found

The requested URL /platform/2/protocols/nfs/exports was not found on this server.

There is the minimal code reproducing this error:

>>> from isi_sdk_8_0 import ApiClient
>>> ApiClient(ISILON_API_URL)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/lucas_cimon/.local/share/virtualenvs/infralib-py2/lib/python2.7/site-packages/isi_sdk_8_0/api_client.py", line 71, in __init__
    self.rest_client = rest.RESTClientObject(configuration)
  File "/home/lucas_cimon/.local/share/virtualenvs/infralib-py2/lib/python2.7/site-packages/isi_sdk_8_0/rest.py", line 68, in __init__
    if configuration.verify_ssl:
AttributeError: 'unicode' object has no attribute 'verify_ssl'

https://www.python.org/dev/peps/pep-0008/#package-and-module-names

"Python packages should also have short, all-lowercase names, although the use of underscores is discouraged."

This is causing problems when you add isi_sdk_7_2 to setup.py install_requires, as it converts the underscore to dashes making the package not being found.

Hi,
The NamesSpaceObject definitions in the papi_schema (https://github.com/Isilon/isilon_sdk/blob/master/papi_schemas/definitions.json) misses the container/container_path attributes. When searching recursively this is really required and currently this gets omitted by conversion into the swagger model class.
Please fix that :)

Cheers,
Peter

"name": "SyntaxError",
"message": "Swagger schema validation failed.
Data does not match any schemas from 'oneOf' at #/paths//platform/3/event/eventgroup-occurrences/{EventEventgroupOccurrenceId}/get/responses/200
Data does not match any schemas from 'oneOf' at #/paths//platform/3/event/eventgroup-occurrences/{EventEventgroupOccurrenceId}/get/responses/200/schema
Data does not match any schemas from 'anyOf' at #/schema/properties/eventgroup-occurrences/items
Data does not match any schemas from 'anyOf' at #/properties/eventgroup-occurrences/items/properties/causes/items
Data does not match any schemas from 'anyOf' at #/properties/causes/items/type
No enum match for: {"description":"Event Group cause as name and description","items":{"type":"string"},"type":"array"} at #/type
Expected type array but found type object at #/type
Expected type array but found type object at #/properties/causes/items
Expected type array but found type object at #/properties/eventgroup-occurrences/items
Additional properties not allowed: properties at #/schema
Missing required property: $ref at #/paths//platform/3/event/eventgroup-occurrences/{EventEventgroupOccurrenceId}/get/responses/200

create_swagger_config.py is passing a bad PAPI schema through to the spec file. The bad schema originates as the GET_output_schema for /platform/3/event/eventgroup-occurrences.

This should be fixed in a future release of OneFS (bug 176274), however to generate a compliant spec file for existing releases create_swagger_config.py will need to correct the schema for the spec file.

Found in v0.1.2 isi_sdk_8_0/output.json

Hello, Not an issue more of a how would you do this...I'm looking to convert the sdk to a service by placing the following file content for isilon_sdk.service under /lib/systemd/system/

Below doesn't seem to work can I ask for an example of a working file so that I can make the sdk a running service?

[Unit]
Description=Isilon SDK for Grafana
After=influxdb.service
Requires=influxdb.service
StartLimitIntervalSec=350

[Service]
Type=simple
Restart=always
Restart=on-failure
RestartSec=30
User=root
ExecStart=/root/start-sdk.sh
ExecStop=/root/stop-sdk.sh
[Install]
WantedBy=multi-user.target

"name": "SyntaxError",
"message": "Swagger schema validation failed.
Data does not match any schemas from 'oneOf' at #/paths//platform/3/statistics/operations/get/responses/200
Data does not match any schemas from 'oneOf' at #/paths//platform/3/statistics/operations/get/responses/200/schema
Data does not match any schemas from 'anyOf' at #/schema/properties/operations/items
Expected type object but found type array at #/properties/operations/items/properties/operations
Expected type array but found type object at #/properties/operations/items
Additional properties not allowed: properties at #/schema
Missing required property: $ref at #/paths//platform/3/statistics/operations/get/responses/200

The root of this problem is an invalid PAPI GET_output_schema for /platform/3/statistics/operations. This should be fixed in a future OneFS release (bug 176250), but for existing releases create_swagger_config.py would need to handle this to generate a compliant spec file.

Here's the offending bit in the spec file. The "operations" array is where the trouble starts:

        "StatisticsOperation": {
            "properties": {
                "operations": [ 
                    {
                        "operation": {
                            "description": "The name of the operation.",
                            "required": true,
                            "type": "string"
                        }
                    }
                ]
            },
            "type": "object"
        },

Found in v0.1.2 isi_sdk_8_0/output.json

There should be information in the README.md on where to communicate security issues to enable for responsible disclosure.

There is currently no SDK release for OneFS 8.2.0, the most recent one for 8.1.1 has some issues.

For example when calling create_smb_share it complains about unable to set the id field.

"name": "SyntaxError",
"message": "Swagger schema validation failed.
Data does not match any schemas from 'oneOf' at #/paths//platform/1/protocols/smb/settings/global/put/parameters/0
Data does not match any schemas from 'oneOf' at #/paths//platform/1/protocols/smb/settings/global/put/parameters/0
Data does not match any schemas from 'anyOf' at #/schema/properties/audit_global_sacl/items
Data does not match any schemas from 'anyOf' at #/schema/properties/audit_global_sacl/items/properties/permission/items
Array items are not unique (indexes 13 and 27) at #/properties/permission/items/enum
Expected type array but found type object at #/properties/permission/items
Expected type array but found type object at #/schema/properties/audit_global_sacl/items
Missing required property: type at #/
Missing required property: $ref at #/paths//platform/1/protocols/smb/settings/global/put/parameters/0

In the permission enum, delete_child appears twice in the PAPI schema. This is OneFS bug 174819. It could be handled in create_swagger_config.py by deduplicating enums when generating the spec file.

Found in v0.1.2 isi_sdk_7_2/output.json.

Data does not match any schemas from 'oneOf' at #/paths//platform/1/storagepool/status/get/responses/200
Data does not match any schemas from 'oneOf' at #/paths//platform/1/storagepool/status/get/responses/200/schema
Data does not match any schemas from 'anyOf' at #/schema/properties/unhealthy/items
Data does not match any schemas from 'anyOf' at #/properties/unhealthy/items/properties/health_flags/items
Expected type array but found type boolean at #/properties/health_flags/items/required
Expected type array but found type object at #/properties/health_flags/items
Expected type array but found type object at #/properties/unhealthy/items
Additional properties not allowed: properties at #/schema
Missing required property: $ref at #/paths//platform/1/storagepool/status/get/responses/200

A draft 3 style "required": <boolean> attribute is the source of the failure. Swagger spec requires draft 4 style required attributes "required": ["<attr1>", "<attr2>", ... ]. This is compounded by the fact that the required attribute appears to be misplaced in the PAPI GET_output_schema. It is applied to the item definition, but it should be applied to the array. The PAPI schema is fixed in a later OneFS build.

As it appears in spec file (v0.1.2 isi_sdk_7_2/output.json):

                "health_flags": {
                    "description": "The disk pool health status.",
                    "items": {
                        "enum": [
                            "underprovisioned",
                            "missing_drives",
                            "devices_down",
                            "devices_smartfailed",
                            "waiting_repair"
                        ],
                        "required": true,
                        "type": "string"
                    },
                    "type": "array"
                }

id = '1-223-L2lmcy9hdXJpc2kwMi9hdXJhejUvdDEvc21iL0FVMjAwNE5QMDA2Mi9BVTIwMDROUDAwNjJfdm9sMDAxL2dyb3VwZGF0YS9Db3Jwb3JhdGUvcHJ...WZpY2F0aW9uL1JFQ1lDTEVSL1MtMy04LTMwLTQ2MDM0MTAwMzAtODk2NDk2ODgxMS02MjUxMjExMDE3LTE0ODgvQXNlLnVvdQAA10-MTY2NDI2NTE1NAAA'

    @id.setter
    def id(self, id):
        """Sets the id of this ReportsThreatsReport.
    
        A unique identifier for the report.  # noqa: E501
    
        :param id: The id of this ReportsThreatsReport.  # noqa: E501
        :type: str
        """
        if id is not None and len(id) > 255:
>           raise ValueError("Invalid value for `id`, length must be less than or equal to `255`")  # noqa: E501
E           ValueError: Invalid value for `id`, length must be less than or equal to `255`

.venv/lib/python3.11/site-packages/isilon_sdk/v9_5_0/models/reports_threats_report.py:128: ValueError

The virus report ID returned by the API is > 255 chars

I guess this might be a bug with the product not the SDK?

thanks

Hello,
When I run isi_sdk.NetworkApi function get_network_interfaces() I receive this output. There is no information for interfaces. Variable type is none, although total interface count is valid.
{'interface': None, 'resume': None, 'total': 6}

I am able to retrieve correct output when I query /platform/3/network/interfaces link directly

It would be helpful if the tests/test_quota.py example included a threshold, so that it is easier for users to understand how the formatting of that section works.

Thanks,
Ken

I didn't see mention of the Go bindings in the top-level README. I may be misunderstanding that project's purpose. Were those bindings generated from the OpenAPI spec in your repo?

$ pip install --no-binary :all: isi-sdk-8-0
Collecting isi-sdk-8-0
  Could not find a version that satisfies the requirement isi-sdk-8-0 (from versions: )
No matching distribution found for isi-sdk-8-0

$ pip install --only-binary :all: isi-sdk-8-0
Collecting isi-sdk-8-0
  Using cached isi_sdk_8_0-0.1.5-py2.py3-none-any.whl
Collecting certifi (from isi-sdk-8-0)
  Using cached certifi-2017.4.17-py2.py3-none-any.whl
Collecting urllib3>=1.15 (from isi-sdk-8-0)
  Downloading urllib3-1.21.1-py2.py3-none-any.whl (131kB)
    100% |████████████████████████████████| 133kB 747kB/s 
Requirement already satisfied: six>=1.10 in ./vtmp/lib/python2.7/site-packages (from isi-sdk-8-0)
Collecting python-dateutil (from isi-sdk-8-0)
  Using cached python_dateutil-2.6.0-py2.py3-none-any.whl
Installing collected packages: certifi, urllib3, python-dateutil, isi-sdk-8-0
Successfully installed certifi-2017.4.17 isi-sdk-8-0-0.1.5 python-dateutil-2.6.0 urllib3-1.21.1

This is problematic, for example, if headers that are required for installing a package with system-level dependencies (e.g. sasl.h for the sasl package) are located in a non-standard location. In that case, you can specify the location to pip via --global-opts which forces installation via sdist.

I tried to generate Go bindings using Swagger/OpenAPI and it generated bad code (ErrorUnknown) for the HardwareTapesDevices API type. I subsequently discovered swagger-go and using that, it has found several more conformance issues of which this is the first:

2019/03/19 12:01:01 validating spec ./isi_8.0.0.7.config
The swagger spec at "./isi_8.0.0.7.config" is invalid against swagger specification 2.0. see errors :
- type in definitions.HardwareTapesDevices.properties.media_changers is required
- "definitions.HardwareTapesDevices.properties.media_changers.items" must validate at least one schema (anyOf)
- items in definitions.HardwareTapesDevices.properties.media_changers.items is required
- items in definitions.HardwareTapesDevices.properties.media_changers.items.properties.paths is required
- type in definitions.HardwareTapesDevices.properties.tapes is required
- "definitions.HardwareTapesDevices.properties.tapes.items" must validate at least one schema (anyOf)
- items in definitions.HardwareTapesDevices.properties.tapes.items is required
- items in definitions.HardwareTapesDevices.properties.tapes.items.properties.paths is required

PR to follow, once I can get the Python to mangle the JSON to match my manual fix!

Customers need a way through the SDK to access the filesytem. The customer I spoke to is
looking to automate SyncIQ policy creation and needs to be able to view the directory on OneFS. Right now you can only do it outside of the sdk using curl.

"name": "SyntaxError",
"message": "Swagger schema validation failed.
Data does not match any schemas from 'oneOf' at #/paths//platform/1/debug/stats/get/responses/200
Data does not match any schemas from 'oneOf' at #/paths//platform/1/debug/stats/get/responses/200/schema
Data does not match any schemas from 'anyOf' at #/schema/properties/handlers/items
Additional properties not allowed: descriprion at #/properties/handlers/items/properties/name
Expected type array but found type object at #/properties/handlers/items
Additional properties not allowed: properties at #/schema
Missing required property: $ref at #/paths//platform/1/debug/stats/get/responses/200

"name": "SyntaxError",
"message": "Swagger schema validation failed.
Data does not match any schemas from 'oneOf' at #/paths//platform/1/auth/access/{AuthAccessUser}/get/responses/200
Data does not match any schemas from 'oneOf' at #/paths//platform/1/auth/access/{AuthAccessUser}/get/responses/200/schema
Data does not match any schemas from 'anyOf' at #/schema/properties/access/items
Additional properties not allowed: descriptoin at #/properties/access/items/properties/id
Expected type array but found type object at #/properties/access/items
Additional properties not allowed: properties at #/schema
Missing required property: $ref at #/paths//platform/1/auth/access/{AuthAccessUser}/get/responses/200

2 spec file validation failures occur due to misspelled attribute names that are passed through to the spec file. There are 2 variations on the spelling of "description" that cause these failures:

• descriprion at /platform/1/debug/stats/get/responses/200/schema
• descriptoin at /platform/1/auth/access/{AuthAccessUser}/get/responses/200

These misspellings are fixed in OneFS in later releases, but to generate a compliant spec file for existing OneFS versions create_swagger_config.py would need to correct these when generating the spec file.

The python bindings should catch two errors.

HTTP not redirected

First, if http is not configured to redirect, then using the raw IP or fqdn for OneFS will result in an ApiException with Traceback:

Traceback (most recent call last):
...
    check_inheritance(isi_sdk.AuthApi(onefs).get_settings_acls())
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0_1/apis/auth_api.py", line 4444, in get_settings_acls
    callback=params.get('callback'))
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0_1/api_client.py", line 339, in call_api
    response_type, auth_settings, callback)
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0_1/api_client.py", line 166, in __call_api
    post_params=post_params, body=body)
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0_1/api_client.py", line 359, in request
    headers=headers)
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0_1/rest.py", line 184, in GET
    query_params=query_params)
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0_1/rest.py", line 177, in request
    raise ApiException(http_resp=r)
isi_sdk_8_0_1.rest.ApiException: (404)
Reason: Not Found
HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 18 May 2017 16:38:01 GMT', 'Content-Length': '227', 'Content-Type': 'text/html; charset=iso-8859-1', 'Server': 'Apache/2.2.31 (FreeBSD) DAV/2 mod_ssl/2.2.31 OpenSSL/1.0.2j-fips'})
HTTP response body: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /platform/3/auth/settings/acls was not found on this server.</p>
</body></html>

Instead of this traceback the user should get an error that the request was rejected and they should try HTTPS if they haven't already.

Missing port

Second, if the host string has https:// but not the port :8080 at the end of the IP or fqdn, an exception is raised by urllib3:

2017-05-19 09:09:53,611 WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x344c5d0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /platform/3/auth/settings/acls
[2017-05-19 09:09:53,611] WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x344c5d0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /platform/3/auth/settings/acls
2017-05-19 09:09:53,857 WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x344c790>: Failed to establish a new connection: [Errno 111] Connection refused',)': /platform/3/auth/settings/acls
[2017-05-19 09:09:53,857] WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x344c790>: Failed to establish a new connection: [Errno 111] Connection refused',)': /platform/3/auth/settings/acls
2017-05-19 09:09:54,099 WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x344c8d0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /platform/3/auth/settings/acls
[2017-05-19 09:09:54,099] WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x344c8d0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /platform/3/auth/settings/acls
Traceback (most recent call last):
  File "isilon-hadoop-tools/ambari_config_check.py", line 542, in <module>
    main()
  File "isilon-hadoop-tools/ambari_config_check.py", line 489, in main
    check_inheritance(isi_sdk.AuthApi(onefs).get_settings_acls())
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0/apis/auth_api.py", line 4210, in get_settings_acls
    callback=params.get('callback'))
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0/api_client.py", line 339, in call_api
    response_type, auth_settings, callback)
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0/api_client.py", line 166, in __call_api
    post_params=post_params, body=body)
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0/api_client.py", line 359, in request
    headers=headers)
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0/rest.py", line 184, in GET
    query_params=query_params)
  File "/usr/lib/python2.7/site-packages/isi_sdk_8_0/rest.py", line 161, in request
    headers=headers)
  File "/usr/lib/python2.7/site-packages/urllib3/request.py", line 66, in request
    **urlopen_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/request.py", line 87, in request_encode_url
    return self.urlopen(method, url, **extra_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/poolmanager.py", line 321, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 678, in urlopen
    **response_kw)
  File "/usr/lib/python2.7/site-packages/urllib3/connectionpool.py", line 649, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/lib/python2.7/site-packages/urllib3/util/retry.py", line 388, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='dog-pools.west.isilon.com', port=443): Max retries exceeded with url: /platform/3/auth/settings/acls (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x344c9d0>: Failed to establish a new connection: [Errno 111] Connection refused',))

This error is not helpful since it's not clear how urllib3 got the string that it tried to use. It would be easy to check the host to see if it has https:// at the front and no port, and warn the user.

{
"name": "SyntaxError",
"message": "Validation failed. /paths/platform/1/auth/mapping/identities/{MappingIdentityId}/post is missing path parameter(s) for {MappingIdentityId}",
"stack": "SyntaxError: Validation failed. /paths/platform/1/auth/mapping/identities/{MappingIdentityId}/post is missing path parameter(s) for {MappingIdentityId}
at Function.ono as syntax
at validatePathParameters (/Volumes/Git/sdk-test/node_modules/swagger-parser/lib/validate-spec.js:174:15)
at validateParameters (/Volumes/Git/sdk-test/node_modules/swagger-parser/lib/validate-spec.js:99:3)
at /Volumes/Git/sdk-test/node_modules/swagger-parser/lib/validate-spec.js:45:7
at Array.forEach (native)
at validatePath (/Volumes/Git/sdk-test/node_modules/swagger-parser/lib/validate-spec.js:40:18)
at /Volumes/Git/sdk-test/node_modules/swagger-parser/lib/validate-spec.js:25:7
at Array.forEach (native)
at validateSpec (/Volumes/Git/sdk-test/node_modules/swagger-parser/lib/validate-spec.js:20:9)
at /Volumes/Git/sdk-test/node_modules/swagger-parser/lib/index.js:161:9"
}

In the paths section, for "/platform/1/auth/mapping/identities/{MappingIdentityId}" the post operation does not map {MappingIdentityId} from the path.

            "post": {
                "description": "Manually set or modify a mapping between two personae.",
                "operationId": "createMappingIdentity",
                "parameters": [
                    {
                        "description": "Desired mapping target to fetch/generate.",
                        "enum": [
                            "uid",
                            "gid",
                            "sid"
                        ],
                        "in": "query",
                        "name": "type",
                        "type": "string"
                    },
                    {
                        "description": "Optional zone.",
                        "in": "query",
                        "name": "zone",
                        "type": "string"
                    },
                    {
                        "in": "body",
                        "name": "MappingIdentity",
                        "required": true,
                        "schema": {
                            "$ref": "#/definitions/Empty"
                        }
                    }
                ],

To resolve this validation failure, there should be an additional parameter:

                    {
                       "in": "path",
                       "name": "MappingIdentityId",
                       "required": true,
                       "type": "string"
                    },

Found in v0.1.2 isi_sdk_7_2/output.json.

The 9.0 build for PYPI appears to be missing even though the link is in the Readme.MD

https://pypi.org/project/isi-sdk-9-0-0/

Issue

Generated Java language bindings fail to install using mvn install with:

[ERROR] /Volumes/Git/swagger-codegen/isi_sdk_java_7_2/src/main/java/io/swagger/client/model/SyncJobPolicyFileMatchingPatternOrCriteriaItemAndCriteriaItem.java:[26,4] error: variable _ is already defined in enum OperatorEnum
[ERROR] /Volumes/Git/swagger-codegen/isi_sdk_java_7_2/src/main/java/io/swagger/client/model/SyncJobPolicyFileMatchingPatternOrCriteriaItemAndCriteriaItem.java:[27,4] error: variable _ is already defined in enum OperatorEnum

and so on

Analysis

The root of the problem appears to sanitation of enum names that occurs in Java code gen where characters outside of [A-Za-z0-9] are replaced with _. The operator enum is defined in the PAPI schema:

/4/sync/jobs?describe: 
                              "operator": {
                                "enum": [
                                  "==", 
                                  "!=", 
                                  ">", 
                                  ">=", 
                                  "<", 
                                  "<=", 
                                  "!"
                                ], 

Each of those values is converted to varable name _ during Java codegen.

Steps to reproduce:

Generate swagger spec file:
components/create_swagger_config.py -i <IP for 7.2.1 cluster> -u root -p a -e excluded_end_points_7_2.json -o output_7_2.json

Generate Java language bindings:
java -jar modules/swagger-codegen-cli/target/swagger-codegen-cli.jar generate -i output_7_2.json -l java -o isi_sdk_java_7_2

Attempt to install the generated bindings:
cd isi_sdk_java_7_2
mvn install

Among some other errors you will see:

[ERROR] /Volumes/Git/swagger-codegen/isi_sdk_java_7_2/src/main/java/io/swagger/client/model/SyncJobPolicyFileMatchingPatternOrCriteriaItemAndCriteriaItem.java:[26,4] error: variable _ is already defined in enum OperatorEnum

"name": "SyntaxError",
"message": "Swagger schema validation failed.
Data does not match any schemas from 'oneOf' at #/paths//platform/3/hardware/tapes/get/responses/200
Data does not match any schemas from 'oneOf' at #/paths//platform/3/hardware/tapes/get/responses/200/schema
Additional properties not allowed: tapes,media_changers at #/schema/properties/devices
Additional properties not allowed: properties at #/schema
Missing required property: $ref at #/paths//platform/3/hardware/tapes/get/responses/200

Problem is caused by an invalid PAPI GET_output_schema for /platform/3/hardware/tapes. This should be fixed in a future OneFS release (bug 176270) but for existing releases would need to be handled in create_swagger_config.py to generate a compliant swagger spec file.

Here's the problem output in the spec file. "media_changers" and "tapes" are not valid as direct children of "devices". "devices" is missing type and properties attributes.

        "HardwareTapes": {
            "description": "Get list Tape and Changer devices",
            "properties": {
                "devices": {
                    "media_changers": { ... },
                    "tapes": { ... }
                },
                "resume": {
                    "description": "Resume string returned by previous query.",
                    "type": "string"
                },
                "total": {
                    "description": "The number of devices",
                    "type": "integer"
                }
            },
            "type": "object"
        },

Found in v0.1.2 isi_sdk_8_0/output.json

The version of the swagger Python mustache we're using is rather old and has a number of issues. Specifically in this case, it fails to handle Python 2 longs, so any API calls with larger integers e.g. setting quotas fail thus:

Traceback (most recent call last):
  File "provtest.py", line 143, in provision_storage
    quota = task.create_quota()
  File "C:\Users\timw\test\provtest\modules\provisioning_task.py", line 382, in create_quota
    self.quota_id = self.api_client.create_quota(self.share_path, advisory=self.size)
  File "C:\Users\timw\test\provtest\modules\isi_api_client_8_0.py", line 59, in create_quota
    api_response = api_instance.create_quota_quota(quota_quota)
  File "C:\Users\timw\test\test_env\lib\site-packages\isi_sdk_8_0\apis\quota_api.py", line 127, in create_quota_quota
    callback=params.get('callback'))
  File "C:\Users\timw\test\test_env\lib\site-packages\isi_sdk_8_0\api_client.py", line 345, in call_api
    response_type, auth_settings, callback)
  File "C:\Users\timw\test\test_env\lib\site-packages\isi_sdk_8_0\api_client.py", line 157, in __call_api
    body = self.sanitize_for_serialization(body)
  File "C:\Users\timw\test\test_env\lib\site-packages\isi_sdk_8_0\api_client.py", line 242, in sanitize_for_serialization
    for key, val in iteritems(obj_dict)}
  File "C:\Users\timw\test\test_env\lib\site-packages\isi_sdk_8_0\api_client.py", line 242, in <dictcomp>
    for key, val in iteritems(obj_dict)}
  File "C:\Users\timw\test\test_env\lib\site-packages\isi_sdk_8_0\api_client.py", line 242, in sanitize_for_serialization
    for key, val in iteritems(obj_dict)}
  File "C:\Users\timw\test\test_env\lib\site-packages\isi_sdk_8_0\api_client.py", line 242, in <dictcomp>
    for key, val in iteritems(obj_dict)}
  File "C:\Users\timw\test\test_env\lib\site-packages\isi_sdk_8_0\api_client.py", line 238, in sanitize_for_serialization
    for attr, _ in iteritems(obj.swagger_types)
AttributeError: 'long' object has no attribute 'swagger_types'

This was fixed in upstream by the changes in commit swagger-api/swagger-codegen@89befeb#diff-027e164ab608834aec24a0ee3c3fc9c5

"name": "SyntaxError",
"message": "Swagger schema validation failed.
Data does not match any schemas from 'oneOf' at #/paths//platform/1/job/jobs/{JobJobId}/get/responses/200
Data does not match any schemas from 'oneOf' at #/paths//platform/1/job/jobs/{JobJobId}/get/responses/200/schema
Data does not match any schemas from 'anyOf' at #/schema/properties/jobs/items
Data does not match any schemas from 'anyOf' at #/properties/jobs/items/properties/paths/items
Expected type array but found type boolean at #/properties/paths/items/required
Expected type array but found type object at #/properties/paths/items
Expected type array but found type object at #/properties/jobs/items
Additional properties not allowed: properties at #/schema
Missing required property: $ref at #/paths//platform/1/job/jobs/{JobJobId}/get/responses/200

Draft 3 style required attributes appear in spec file. Swagger spec requires draft 4 style required attributes.

                "participants": {
                    "description": "The set of devids working on the job.",
                    "items": {
                        "required": true,
                        "type": "integer"
                    },
                    "type": "array"
                },
                "paths": {
                    "description": "Paths for which the job was queued.",
                    "items": {
                        "required": true,
                        "type": "string"
                    },
                    "type": "array"
                },

create_swagger_config.py seems to correctly convert a number of these required attributes for this definition, just not the above 2.

Attrs converted to v4 style:

            "required": [
                "impact",
                "type",
                "state",
                "priority",
                "create_time",
                "policy",
                "retries_remaining",
                "total_phases",
                "id"
            ],

Using python 2.7.8 under babun (cygwin) I get python failures. The identical command works fine on CentOS 7, python 2.7.9. Not urgent, just FYI.

{ isilon_sdk } master » python components/create_swagger_config.py -i [REDACTED] -o onefs_8.0_swagger.json

Please provide the username used to access [REDACTED] via PAPI: [REDACTED]
Password:
Traceback (most recent call last):
  File "components/create_swagger_config.py", line 1228, in <module>
    main()
  File "components/create_swagger_config.py", line 1104, in main
    excludeEndPoints)
  File "components/create_swagger_config.py", line 903, in GetEndpointPaths
    numEndPoints = len(endPointListJson["directory"])
KeyError: 'directory'
{ isilon_sdk } master » python -V                                         ~/swagger2/isilon_sdk 1
Python 2.7.8

So, I have some code which I will include below which uses a limit to restrict the number SMB openfiles the call does at a time. The first time I run it, I get the proper number along with a resume key. However, when I try to run the call again with the resume key, I get a empty data set. This does not seem to be the issue with other calls such as listing snapshots.
openfiles.txt

Here is the output:

Not a resume call
{'openfiles': [{'file': 'C:\ifs\data\PDF\007.pdf',
'id': 7247,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF\Aesops_Fables_T.pdf',
'id': 7373,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF\catalog_aso.pdf',
'id': 7539,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF\Around_the_World_in_80_Days_T.pdf',
'id': 7485,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF\4233.pdf',
'id': 7287,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF\Alices_Adventures_in_Wonderland_T.pdf',
'id': 7411,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF\aoa.pdf',
'id': 7449,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF\boxman.pdf',
'id': 7503,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF',
'id': 6749,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'},
{'file': 'C:\ifs\data\PDF\Globes.pdf',
'id': 7577,
'locks': 0,
'permissions': ['read'],
'user': 'Administrator'}],
'resume': '2-2-MTAA10-NDI5NDk2NzI4MwAA',
'total': 24}
DEB: RESUME CALL
{'openfiles': [], 'resume': None, 'total': 24}

"name": "SyntaxError",
"message": "Swagger schema validation failed.
Data does not match any schemas from 'oneOf' at #/paths//platform/3/protocols/nfs/netgroup/put/parameters/1
Data does not match any schemas from 'oneOf' at #/paths//platform/3/protocols/nfs/netgroup/put/parameters/1
Expected type object but found type string at #/schema/properties/type
Missing required property: type at #/
Missing required property: $ref at #/paths//platform/3/protocols/nfs/netgroup/put/parameters/1

A malformed schema is generated for definition NfsNetgroup. The "properties" nested directly under another "properties" is not valid.

        "NfsNetgroup": {
            "properties": {
                "description": "NFS netgroup cache settings.",
                "properties": {
                    "bgwrite": {
                        "description": "To-disk backup interval for the netgroup cache.",
                        "type": "integer"
                    },
                    "expiration": {
                        "description": "Time between updates of netgroups in the cache.",
                        "type": "integer"
                    },
                    "lifetime": {
                        "description": "Length of time an un-accessed netgroup remains in the cache.",
                        "type": "integer"
                    },
                    "retry": {
                        "description": "Retry interval for netgroup updates if the remote provider is unresponsive.",
                        "type": "integer"
                    }
                },
                "type": "object"
            },
            "type": "object"
        },

The root of the problem is likely a malformed PAPI PUT_input_schema for /platform/3/protocols/nfs/netgroup/put/parameters/1. This should be fixed in a future OneFS release (bug 176259) but would need to be addressed in create_swagger_config.py to create valid spec files from existing OneFS releases.

The bad schema suspected to result in the bad output in the spec file is:

"PUT_input_schema": {
  "settings": {
    "description": "NFS netgroup cache settings.",
    "properties": {
      "bgwrite": {
        "description": "To-disk backup interval for the netgroup cache.",
        "type": "integer"
      },
      "expiration": {
        "description": "Time between updates of netgroups in the cache.",
        "type": "integer"
      },
      "lifetime": {
        "description": "Length of time an un-accessed netgroup remains in the cache.",
        "type": "integer"
      },
      "retry": {
        "description": "Retry interval for netgroup updates if the remote provider is unresponsive.",
        "type": "integer"
      }
    },
    "type": "object"
  },
  "type": "object"
}