cloudera / cm_api

Cloudera Manager API Client

Cloudera Manager RESTful API Clients

Cloudera Manager is the market-leading management platform for CDH. As the industry’s first end-to-end management application for Apache Hadoop, Cloudera Manager sets the standard for enterprise deployment by delivering granular visibility into and control over every part of CDH – empowering operators to improve cluster performance, enhance quality of service, increase compliance and reduce administrative costs.

This project contains all the source, examples and documentation you need to easily build a Cloudera Manager client in Java or Python.

All source in this repository is Apache-Licensed.

This client code allows you to interact with Cloudera Manager to:

• Manage multiple clusters
• Start and stop all or individual services or roles
• Upgrade services running on your cluster
• Access time-series data on resource utilitization for any activity in the system
• Read logs for all processes in the system as well as stdout and stderr
• Programmatically configure all aspects of your deployment
• Collect diagnostic data to aid in debugging issues
• Run distributed commands to manage auto-failover, host decommissioning and more
• View all events and alerts that have occurred in the system
• Add and remove users from the system

cm_api's Issues

Python api binding endpoint for role missing process endpoints.

I couldn't find anything in the python binding for cm api that correlates to the following endpoint:

/clusters/{clusterName}/services/{serviceName}/roles/{roleName}/process/configFiles/{configFileName }

Can this be added?

Diagnostic data collection in python doesn't return the download URL

collect_diagnostic_data_45
return a job command, after calling fetch (which should refresh the job information) the resultDataUrl on the apiCommand isn't set.

Cluster creation via importTemplate does not create hostTemplates in Cloudera Manager

I am using cloudera manager python API to create a cluster via import_cluster_template method.

I have a cluster template with well defined "hostTemplates" which is being used in the "instantiator" section to map it with the different hosts. I am able to successfully import the template and create the cluster services, but when I look into the "Hosts Templates", I see completely empty.

Having the hosttemplate also imported as templates would help me expand the cluster using cm_api. Let me know if I am doing anything wrong.

More broken documentation

What does create_cluster do in the Python API when a cluster of the same name already exists?

The PyDoc comments you put in the source code are also useless. Can you please write something which is actually usable?

http://cloudera.github.io/cm_api/apidocs/v5/path__clusters.html

Use Encrypted password for ApiResource instead of cleartext password

Can I pass an encrypted password to the cm_api python ApiResource?
What is the recommended way to securely send the password as this user will receive admin level privileges once instantiated?

Documentation of Configure from Scratch Flow

Hi, I'm working on automating the setup of a CDH cluster after installing packages.
I would like to avoid using the wizard manually, and instead execute some python code to add nodes and to configure services and roles. The provided examples are good to understand how the API should be used, but I think it would be better to have a full example detailing the overall flow that it is done thru the wizard.

This will enable automatic setup of the cluster without relying on human interaction.

Thanks!

No service type 'SPARK' available for cluster with version 'CDH 6.3.0'

Hi,
I am able to install HDFS, YARN and Zookeeper. But facing issue while installing spark.
when I run
python deploycloudera.py -ihivepass -fCloudera@123 -nCloudera@123 -rCloudera@123

Traceback (most recent call last):
File "deploycloudera.py", line 1079, in
main()
File "deploycloudera.py", line 1028, in main
spark_service = deploy_spark(CLUSTER, SPARK_SERVICE_NAME, SPARK_SERVICE_CONFIG, SPARK_MASTER_HOST, SPARK_MASTER_CONFIG, SPARK_WORKER_HOSTS, SPARK_WORKER_CONFIG, SPARK_GW_HOSTS, SPARK_GW_CONFIG)
File "deploycloudera.py", line 689, in deploy_spark
spark_service = cluster.create_service(spark_service_name, "SPARK")
File "/usr/lib/python2.7/site-packages/cm_api/endpoints/clusters.py", line 161, in create_service
service_type, self.name)
File "/usr/lib/python2.7/site-packages/cm_api/endpoints/services.py", line 44, in create_service
ApiService, True, data=[apiservice])[0]
File "/usr/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 137, in call
ret = method(path, data=data, params=params)
File "/usr/lib/python2.7/site-packages/cm_api/resource.py", line 148, in post
self._make_headers(contenttype))
File "/usr/lib/python2.7/site-packages/cm_api/resource.py", line 73, in invoke
headers=headers)
File "/usr/lib/python2.7/site-packages/cm_api/http_client.py", line 193, in execute
raise self._exc_class(ex)
cm_api.api_client.ApiException: No service type 'SPARK' available for cluster with version 'CDH 6.3.0'. (error 400).

What is the service name for spark in CDH 6.3.0?

get_impala_queries does not return all records

When I run a get_impala_queries from python it returns just 2 records, even if I use the same date range and filter.
When I filter it in the CM UI, the first two records appears immediately, and then after a second the rest of those queries.
Is it possible to get the next result somehow?

Builder for `ApiService`

When creating a new service your code have to go like

ApiService svc = new ApiService();
svc.setClusterRef(new ClusterRef(cluster.getName()));
svc.setName("hdfs");
svc.setType("HDFS");
List<ApiRoles> roles = new ArrayList<>();
svc.setRoles(roles);
ApiRole role = new ApiRole();
role.setName("mynamenode");
role.setType("NAMENODE");
roles.add(role);

etc.

It'll be nicer to have a nicer API like

ApiServiceBuilder.newHdfs("myhdfs1", cluster).
    withRole(ApiRoleBuilder.newNameNode("mynamenode", host2)).
    withRole(ApiRoleBuilder.newDataNode("myDataNode", host1)).
    build();

Will you accept pull requests? I'm having something I can make a pull request from.

Typo in cmps options -s SEPERATOR (should be SEPARATOR)

Don't think I've got access to push to this repo to fix it.

Java Client is using deprecated Jackson2 object references in the class ApiObjectMapper

The project uses an older version of Jackson2 and references the deprecated AnnotationIntrospector$Pair class.
This causes problems if you are trying to use this client with a project that has a Jackson2 version greater than 2.2

The class ApiObjectMapper should reference new AnnotationIntrospectorPair( ...) not Pair(...)

To get this to work you also need to update the cxf.verision to 2.7.8
Below are the changes to allow usage of Jackson2 versions 2.3 or later:

pom.xml

  <properties>
    <cxf.version>2.7.8</cxf.version>
    <guava.version>14.0</guava.version>
    <jackson2.version>2.3.4</jackson2.version>
    <joda.version>2.1</joda.version>
    <junit.version>4.11</junit.version>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <test.redirectToFile>true</test.redirectToFile>
    <privateClassPath>com.cloudera.api.shaded</privateClassPath>
  </properties>

ApiObjectMapper constructor

  public ApiObjectMapper() {
    // Print the JSON with indentation (ie. pretty print)
    configure(SerializationFeature.INDENT_OUTPUT, true);
    // Allow JAX-B annotations.
    setAnnotationIntrospector(
       new AnnotationIntrospectorPair(
            getSerializationConfig().getAnnotationIntrospector(),
            new JaxbAnnotationIntrospector(getTypeFactory())));
    // Make Jackson respect @XmlElementWrapper.
    enable(MapperFeature.USE_WRAPPER_NAME_AS_PROPERTY_NAME);
    // Print all dates in ISO8601 format
    setDateFormat(makeISODateFormat());
  }

git tags not used

Hello,
I'm working on cm_api python client package for Debian.
I already have good results, but I would like to use git tags to ease packaging operations.

Could you please use them ?

Thanks,

Cloudera Manager GUI shows configuration validation when services are created using REST API.

Hi,

I am trying to create services like HDFS, MapReduce in Cloudera Manager using REST API. Services are getting created properly along with configurations, however Cloudera Manager GUI shows configuration validations errors, e.g

1. HDFS Checkpoint Directory: Missing required value: HDFS Checkpoint Directory
   2.JobTracker Local Data Directory: Missing required value: JobTracker Local Data Directory.
   Please see the attached image.

If we call REST GET method for the service configuration, above configurations are set properly.

Please suggest how to get rid of validation errors.

Thanks

Abhijeet

python3 compatibility

I installed cm-api for python3 using pip without issues

pip install cm-api

However, following the example on the website , I immediately get an error:

>>> from cm_api.api_client import ApiResource
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python\lib\site-packages\cm_api\api_client.py", line 23, in <module>
    from cm_api.http_client import HttpClient, RestException
  File "C:\Program Files\Python\lib\site-packages\cm_api\http_client.py", line 182
    except urllib2.HTTPError, ex:
                            ^
SyntaxError: invalid syntax

The error is obvious, python3 does not allow ','(comma) as syntax to declare an alias of an exception.

It seems cm-api is NOT compatible with python3 since this commit: dcc11df
I could not find a reason for reverting the compatibility.

So either this lib should be removed from the pip3 repositories or this commit should be 'unreverted'?

ApiService from python binding is missing first_run function.

When installing a new service to an existing cluster using the CM API python binding there should be a way to execute all commands listed in the services SDL under serviceInit. Having a first_run function seems like the logical place to handle this.

How to bootstrap Standby Namenode?

Use in my code:
service.bootstrap_hdfs_stand_by(str(newNameNode.name))
EX:
service.bootstrap_hdfs_stand_by("hdfs-NAMENODE-1234567890")

Error message:
<ApiList>(1): [<ApiCommand>: 'BootstrapStandbyNameNode' (id: 996; active: False; success: False)]

How can I learn more info on what is happening so I can debug?

there is a api that can get the hdfs disk usage summary?

anyone can help me ?

Python 3 compatibility

I see that there were some commits to add python3 support but they were reverted (broke python2?)

I'm sure eventually this will happen, would be great to know what the plans were (we've moved everything else we use to python3 and it's not ideal to have to program in two variants of the same language). Thanks

Is a Swagger spec available?

The documentation says that the Java and Python libraries in this repo are based on Swagger. Is there a Swagger JSON or YAML spec that could be used to generate a client in another language?

My current CM install doesn’t expose API Explorer/Swagger UI under the Support menu; just has API Documentation.

Even more broken documentation

The documentation for create_cluster does not state any failure conditions. Hence, it's broken. Can you please release something which works according to what is documented?

The instructions to call getNextException are also useless, because it doesn't refer to any particular module and the global symbol getNextException doesn't exist. I don't quite understand what moves you to write such shitty documentation and code.

The current documentation for create_cluster for the version argument is also completely wrong; it says one layer down that you can pass it for example "4", except that too is a lie. You can only pass it something like "CDH4" (this is also on your "blog"). So, basically every single API call I find is either broken in its documentation or in its code. How exactly do you expect people to use this?

$ python create_cluster.py 
Traceback (most recent call last):
  File "create_cluster.py", line 18, in <module>
    cluster = api.create_cluster(CLUSTERNAME, "CDH"+str(CDH_VERSION))
  File "/data/home/cloudera-api-caller/development/env/lib/python2.7/site-packages/cm_api/api_client.py", line 108, in create_cluster
    return clusters.create_cluster(self, name, version)
  File "/data/home/cloudera-api-caller/development/env/lib/python2.7/site-packages/cm_api/endpoints/clusters.py", line 45, in create_cluster
    data=[apicluster], api_version=api_version)[0]
  File "/data/home/cloudera-api-caller/development/env/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 137, in call
    ret = method(path, data=data, params=params)
  File "/data/home/cloudera-api-caller/development/env/lib/python2.7/site-packages/cm_api/resource.py", line 138, in post
    self._make_headers(contenttype))
  File "/data/home/cloudera-api-caller/development/env/lib/python2.7/site-packages/cm_api/resource.py", line 63, in invoke
    headers=headers)
  File "/data/home/cloudera-api-caller/development/env/lib/python2.7/site-packages/cm_api/http_client.py", line 161, in execute
    raise self._exc_class(ex)
cm_api.api_client.ApiException: Batch entry 0 insert into CLUSTERS (OPTIMISTIC_LOCK_VERSION, NAME, VERSION, MAINTENANCE_COUNT, CLUSTER_ID) values ('0', 'Cluster 1 - CDH4', '4', '0', '4') was aborted.  Call getNextException to see the cause. (error 400)

cxf version problem

In order to be compatible with the spring 4.x version, I modified the cxf version from 2.7.7 to 3.3.0.
An exception occurred during runtime.

Code：
InputStreamDataSource inputStreamDataSource = apiRoot.getClustersResource().getServicesResource(clusterName)
.getClientConfig(serviceName);

Exception：
Caused by: javax.ws.rs.client.ResponseProcessingException: Problem with reading the data, class org.apache.cxf.jaxrs.ext.multipart.InputStreamDataSource, ContentType: application/octet-stream.

What should i do？

cm_api.api_client.ApiException: (error 404)

Traceback (most recent call last):
File "hdfs.py", line 37, in
Test()
File "hdfs.py", line 29, in Test
result = api.query_timeseries(query, from_time, to_time)
File "/usr/lib/python2.6/site-packages/cm_api/api_client.py", line 295, in query_timeseries
return timeseries.query_timeseries(self, query, from_time, to_time, by_post=by_post)
File "/usr/lib/python2.6/site-packages/cm_api/endpoints/timeseries.py", line 75, in query_timeseries
ApiTimeSeriesResponse, True, params=params, data=data)
File "/usr/lib/python2.6/site-packages/cm_api/endpoints/types.py", line 139, in call
ret = method(path, params=params)
File "/usr/lib/python2.6/site-packages/cm_api/resource.py", line 110, in get
return self.invoke("GET", relpath, params)
File "/usr/lib/python2.6/site-packages/cm_api/resource.py", line 73, in invoke
headers=headers)
File "/usr/lib/python2.6/site-packages/cm_api/http_client.py", line 183, in execute
raise self._exc_class(ex)
cm_api.api_client.ApiException: (error 404)

Issue with creation of Zookeeper service

I am trying to create zookeeper service but init is failing saying "ZooKeeper is already initialized." (in the logs ZooKeeper data directory already exists at /var/lib/zookeeper (or use --force to force re-initialization))

When I use ServicesResourceV2.createServices(service) and ServicesResourceV2.startCommand(serviceName) together service is not starting up.

But When I execute start command after some time.Its just works fine.
Please suggest.

Thanks.
Mahesh

Unable to use Cloudera Manager Python Shell

Hello,

I'm trayng to use cmps, and i have error when i'm trying to authenticate .
help

Regards,

API v17 client libs not pushed

CM 5.12.0 looks to have moved up to v17 of the API, but client libs here remain stuck at v16.

Python: ApiHost.set_rack_id() fails with Type Error

calling set_rack_id()fails with a type error because of an errror in ApiHost._put(). Typical trace is:

  File "./set_rackid.py", line 90, in <module>
    h.set_rack_id(new_rack_id)
  File "/mnt/development/Projects/bugfix/cm_api/python/src/cm_api/endpoints/hosts.py", line 187, in set_rack_id
    self._put()
  File "/mnt/development/Projects//bugfix/cm_api/python/src/cm_api/endpoints/hosts.py", line 100, in _put
    return self._put('', ApiHost, data=self)
TypeError: _put() takes exactly 1 non-keyword argument (3 given)

Update dynamic resource pools and impala admission control

Would it be possible to update dynamic resource pools for YARN and Impala Admission control via cm_api?

Documentation is useless

http://cloudera.github.io/cm_api/apidocs/v6/ns0_apiUser.html is supposed to contain documentation. The only goal I want to achieve is to do a HTTP PUT request via e.g. curl for testing to set the password of the admin user. Unfortunately, you have documented absolutely nothing useful to obtain this goal.

For example, for the fields pwHash, pwSalt and pwLogin it says: NOTE: Only available in the "export" view.

You should write down what the valid range of values is for those inputs without requiring me to reverse-engineer your "product" and you should write down all other conditions which need to be true before this call works, if any.

Your Python API is terrible too, because it doesn't actually do anything unless you specify the version of the API you want to use. This is an optional parameter in your API, but you actually require it, without documenting it. In other words: what are you smoking?

Document which API versions work on which Cloudera Manager versions

Is there a document of which API versions are available on which Cloudera Manager versions?

Now that 6.0 has removed API v1 - 5 I need to update the Nagios Plugins for Cloudera and shouldn't have to backwards and forwards test every single CM myself via /versions to find out which API versions are compatible with which versions. I could just guess to move to v6 but I think this should be documented and would be best done by someone still working for Cloudera.

Java API to put the Management Service in Maintainence Mode

Hi,

I am unable to find documentation around the Java API to put the Management Service in Maintainence Mode.

I can do a start/stop from the API though.

Regards
Chandra

API: /clusters/{clusterName}/services/{serviceName}/yarnApplications

API: /clusters/{clusterName}/services/{serviceName}/yarnApplications
CDH: 5.12

I want to use the cm api , check this job is still running or not running.

Steps：

1. Spark streaming job running is 2 days.

2. Shell：
   JOB_NAME=SparkStreamingJob
   curl -u $CDHUSER:$CDHPASSWORD $CDHURL/api/v17/clusters/JYDWCluster/services/yarn/yarnApplications > cdh.json
   cat cdh.json | grep -A 4 $JOB_NAME > job.log
   RUNNINGNUM=cat job.log | grep "state" | grep "RUNNING" | wc -l

3. In most cases, this job is existed in the cm api result.
   But ,once in a while, this job is not existed in the cm api result, is strange.
   I suspect it's a bug.

api_client ApiResource error

Hello All,

I am using python version 2.6.6 and cloudera/hadoop version 2.6.0-cdh5.8.2. Getting error "importerror no module named cm_api.api_client "
from cm-api.api_client import ApiResource in line 2

Is there any version issues or what could be the reason ? Could someone please help.

Python package should not require `readline`

I believe this is often provided by default, and can cause lots of issues when the user is forced to build it. Could we remove this bit of code?

https://github.com/cloudera/cm_api/blob/master/python/setup.py#L31-L34

high availability for autodeploy example

Hi, I'm working on adding high availability on the autodeploy example from @justinhayes's fork. Do you have any ideas on how to implement it?

I'm thinking on adding a standby name-node for HDFS plus journal nodes to the current roles. Also for YARN jobhistory roles and standby resource manager. However I have no clue on how to do this... Any directions or sample code for that?

How do you accept the licenses during the initial configuration?

I can use the Java API client or the Python API client. Do either provide coverage of the initial acceptance of licenses and selection of Cloudera Express (as opposed to the Data Hub or Enterprise editions)? I normally use the web UI to accept the Java license, Cloudera terms, and choose the edition. I'd like to automate this process.

timeseries.py shows "has no attribute 'isoformat'" bug

Can any body help?thanks.

It is failing to install the latest cm-api(version 20.0.0) with pip

the following errors are showing to stop the installation while I run pip install cm-api:
Collecting cm-api
Using cached https://files.pythonhosted.org/packages/ae/e3/c79c1c36309af73c77cd61157baa547a1d24edb5ca761a40909480785cd9/cm_api-20.0.0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/private/var/folders/y1/kk6g44757wj5z327cctdl1hh0000gq/T/pip-install-v_gSi_/cm-api/setup.py", line 31, in
fobj = open(setup)
IOError: [Errno 2] No such file or directory: '/private/var/folders/y1/kk6g44757wj5z327cctdl1hh0000gq/T/pip-install-v_gSi_/cm-api/python/setup.py'

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/y1/kk6g44757wj5z327cctdl1hh0000gq/T/pip-install-v_gSi_/cm-api/

why there is no SSH free password configuraction

the nodes need ssh free password. why there is no this configuraction

missing property in ApiHdfsReplicationArguments definition causing get_replication_schedules() to fail

I'm getting following error when trying to use get_replication_schedules on the hive service with the latest python release of the cm_api.

>>>hive_service.get_replication_schedules()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/services.py", line 1495, in get_replication_schedules
    api_version=3)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 357, in _get
    api_version)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 380, in _call
    api_version)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 143, in call
    return ApiList.from_json_dict(ret, method.im_self, ret_type)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 421, in from_json_dict
    items = [ attr.from_json(resource_root, x) for x in dic[ApiList.LIST_KEY] ]
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 97, in from_json
    return self._atype.from_json_dict(data, resource_root)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 288, in from_json_dict
    obj._set_attrs(dic, allow_ro=True)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 225, in _set_attrs
    v = attr.from_json(self._get_resource_root(), v)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 97, in from_json
    return self._atype.from_json_dict(data, resource_root)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 288, in from_json_dict
    obj._set_attrs(dic, allow_ro=True)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 225, in _set_attrs
    v = attr.from_json(self._get_resource_root(), v)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 97, in from_json
    return self._atype.from_json_dict(data, resource_root)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 288, in from_json_dict
    obj._set_attrs(dic, allow_ro=True)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 223, in _set_attrs
    attr = self._check_attr(k, allow_ro)
  File "/dhcommon/dhpython/python/lib/python2.7/site-packages/cm_api/endpoints/types.py", line 236, in _check_attr
    (name, self.__class__.__name__))
AttributeError: Invalid property bandwidthPerMap for class ApiHdfsReplicationArguments.

It seems that bandwidthPerMap is missing from the definition of ApiHdfsReplicationArguments in m_api/endpoints/types.py. Adding it to the attributes fixes the problem.

ClouderaManagerClientBuilder does not handle default port with enableTLS=true

If enableTLS=true and port not set, default port remains 7180 which is HTTP default port.
HTTPS default port (7183) should be introduced for that case

Configuring CDH, cm_api cluster.add_hosts(all_hosts) gives error stating "Host not found" ApiException

We are deploying CDH 5.10.0 through the Python client of cm_api in RHEL 6.8 hosted in Azure. We are receiving the following error during the configuring the CDH.

Connected to CM host on stt-mc-te.23qwyritdroeuryrw234jsdfjl.bx.internal.cloudapp.net and updated CM configuration
Traceback (most recent call last):
File "/data/opt/deploytmp/ConfigureCDH.py", line 1037, in
main()
File "/data/opt/deploytmp/ConfigureCDH.py", line 936, in main
CLUSTER = init_cluster(API, CLUSTER_NAME, CDH_VERSION, CLUSTER_HOSTS, CM_HOST)
File "/data/opt/deploytmp/ConfigureCDH.py", line 427, in init_cluster
cluster.add_hosts(all_hosts)
File "/usr/lib/python2.6/site-packages/cm_api-8.0.0-py2.6.egg/cm_api/endpoints/clusters.py", line 240, in add_hosts api_version=3)
File "/usr/lib/python2.6/site-packages/cm_api-8.0.0-py2.6.egg/cm_api/endpoints/types.py", line 362, in _post api_version)
File "/usr/lib/python2.6/site-packages/cm_api-8.0.0-py2.6.egg/cm_api/endpoints/types.py", line 380, in _call api_version)
File "/usr/lib/python2.6/site-packages/cm_api-8.0.0-py2.6.egg/cm_api/endpoints/types.py", line 137, in call ret = method(path, data=data, params=params)
File "/usr/lib/python2.6/site-packages/cm_api-8.0.0-py2.6.egg/cm_api/resource.py", line 138, in post self._make_headers(contenttype))
File "/usr/lib/python2.6/site-packages/cm_api-8.0.0-py2.6.egg/cm_api/resource.py", line 63, in invoke headers=headers)
File "/usr/lib/python2.6/site-packages/cm_api-8.0.0-py2.6.egg/cm_api/http_client.py", line 161, in execute raise self._exc_class(ex)
cm_api.api_client.ApiException: Host 'stt-mc-te.23qwyritdroeuryrw234jsdfjl.bx.internal.cloudapp.net' not found. (error 404)

We checked with reverse DNS with the below python command and it gave the required results.

python -c "import socket; print socket.getfqdn(); print socket.gethostbyname(socket.getfqdn())"

Can someone please advice what might be the issue here.

Ignore nameservice in getHdfsUsageReport call

@bcwalrus,

I'm using Java client to access CM API. I'm trying to get hdfs usage report by the following:

ApiHdfsUsageReport hdfsUsageReport = servicesResource.getHdfsUsageReport(service.getName(), "", "2014-01-22", "2014-01-24", ApiTimeAggregation.DAILY);

Since our cluster doesn't set up HA so I don't have name service info. Therefore I just provide an empty string as shown in the above method call. However, this will cause API error because it considers the nameservice as a blank string rather than ignore it. I'ev attached the exception stack trace at the end of this message.

Is there anyway we could create an overridden method which could ignore nameservice parameter? Before this issue is fixed (or share with me an alternative if you could kindly provide one), I am all stuck with the Java client API and could not use it.

Please help. Thanks.

---

This is the error log from the API logging in CM server log:

GET /api/v4/clusters/CDH4-cluster/services/hdfs1/reports/hdfsUsageReport?nameservice&from=2014-01-22&to=2014-01-24&aggregation=DAILY

2014-01-24 16:11:47,495 WARN [1835383056@scm-web-12510:api.ApiExceptionMapper@141] Unexpected exception.
java.lang.RuntimeException: HeadlampOperationException(msg:Could not access index for service hdfs1/)
at com.cloudera.api.dao.impl.HeadlampDaoImpl.getHdfsUsageReport(HeadlampDaoImpl.java:60)

Add examples of requests to documentation of REST API

Add examples or move documentation to swagger ui - currently is pretty hard to recognize how request that i want to do should look, also error responses are not informative

cloudera-scm-server not starting

Hello ,

Using setup-master.sh , its failing at below step :
_+ sudo service cloudera-scm-server-db start
Creating SCM configuration file: /etc/cloudera-scm-server/db.properties
waiting for server to start...............................................................could not start server
Could not start database server, giving up

• sudo service cloudera-scm-server start
  Starting cloudera-scm-server: [ OK ]
• sudo sed -i.bak $'-es%server_host=localhost%server_host=indlin2394.corp.amdocs.com\nindlin2394%' /etc/cloudera-scm-agent/config.ini
  sed: -e expression #1, char 62: unterminated `s' command
• sudo service cloudera-scm-agent start
  Starting cloudera-scm-agent: [ OK ]_

cloudera-scm-server is not coming UP .
Logs attached herewith .
Logs&Files.zip

i am using below python version .
[root@indlin2394 cloudera-scm-server]# python -V
Python 2.7.13

Kindly help .

NumberFormatException deserializing a Timeseries response

Cloudera Manager version: Cloudera Enterprise 6.2.0
Swagger api java client: 6.2.0, 6.3.0

In the Swagger client library for Java I've had some situations in which a timeseries request would fail at client side with a NumberFormatException.
Digging I've found that the issue happens when trying to deserialize a "NaN" to a BigDecimal.
This is an example that triggers the issue:

...
"data" : [ {
   "timestamp" : "2019-08-14T14:00:00.000Z",
   "value" : 2917.75436282436,
   "type" : "SAMPLE",
   "aggregateStatistics" : {
       "sampleTime" : "2019-08-14T13:50:42.000Z",
       "sampleValue" : 6882.2668112798265,
       "count" : 322211,
       "min" : 1566.891751506337,
       "minTime" : "2019-08-14T13:36:42.000Z",
       "max" : 10739.0,
       "maxTime" : "2019-08-14T13:13:42.000Z",
       "mean" : 2917.75436282436,
       "stdDev" : "NaN"
   }
 } ]
...

The JSON deserializer tries to bind this message to an ApiTimeSeriesAggregateStatistics which has its metric values fields defined as BigDecimal.
So far I've seen this happening only in the stdDev field.
Having a "NaN" suggests the server side is using Doubles. Is there any reason for the client side to be expecting BigDecimal when it seems the server side is really using Doubles?

As a workaround (not a perfect one) I'm setting a custom instance of GSON to the ApiClient having a custom TypeAdapter for BigDecimal that detects NaN and returns null.

Ruby client

A ruby client for the API would be great. Any work underway on that?

cant --> can't

There was a spelling/grammar error in python/src/cm_shell/prettytable.py

It was bothering me ¯_(ツ)_/¯
My editor settings also took out the extra spacing in the file :)

Let me know if there is anything to change. I'll link a PR in a minute