hawkular / hawkular-client-python

Python client for Hawkular

Hawkular

Hawkular is a modular systems monitoring suite, that consists of several sub-projects for storing of metrics, alerting on incoming events and more. Those projects are developed in their own GitHub repositories and integrated in this project.

Project website: http://www.hawkular.org (its code: https://github.com/hawkular/hawkular.github.io)

Sub-projects are currently:

• Hawkular-metrics: metric storage and retrieval engine

• Hawkular-bus: asynchronous bus to connect the various parts

• Hawkular-alerts: alerting on events.

• Hawkular UI Console (a hawt.io 2 plugin)

  • Hawkular-ui-services: common services and ngResource wrappers for Hawkular REST Apis.

• Hawkular-charts: Charts and other Angular visualization components used to graphically render data in Hawkular.

• Hawkular build tools: Helpers and definitions to build Hawkular

About this repository

In this repository we are assembling the individual pieces, sub projects and UI into the overall Hawkular instance.

In the root pom.xml you can actually set the particular versions of the components, but we can’t guarantee the various permutation of component versions to work together.

Building

To build Hawkular, clone this repository and build from the top level.

$ git clone https://github.com/hawkular/hawkular.git
$ cd hawkular
$ mvn install

Once those steps are achieved, .zip and .tgz archives will be available in dist/target directory.

Tip

If you build with mvn install -Pdev, an uncompressed directory will be created in dist/target. A default user will be created, the username is jdoe and the password is password. This can be convenient when you are working on the project as you won’t have to unzip/untar and register a new user. The uncompressed directory can be found in dist/target/hawkular-${version}/hawkular-${version}/ and run with bin/standalone.sh as normally starting a WildFly server.

Please have a look at the developer documentation for more information.

License

Hawkular is released under Apache License, Version 2.0 as described in the LICENSE document.

   Copyright 2015-2016 Red Hat, Inc.

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.

hawkular-client-python's Issues

Improve tenants creation process

Support TenantDefinition structure as dicts in the create_tenant()

query_metric_definition requires metric id

contrary to the README client.query_metric_definition() required metric id

from hawkular.metrics import HawkularMetricsClient, MetricType


host = 'hawkular-services-openshift-infra.origin.cloud1.hawkular.org'
path = 'hawkular/metrics'
port = '443'
token = 'redacted'
tenant_id='promgen'

client = HawkularMetricsClient(token=token, tenant_id=tenant_id, scheme='https', host=host, path=path, port=port)

if __name__ == "__main__":
    tenants = client.query_tenants()
    print(tenants)

    defs = client.query_metric_definition(MetricType.Counter)
    print(defs)

output

Traceback (most recent call last):
  File "pyme/main.py", line 16, in <module>
    defs = client.query_metric_definition(MetricType.Counter)
TypeError: query_metric_definition() takes exactly 3 arguments (2 given)

feature request: query_metric(start/end params) to support native Python datetime

Python prides itself for English like/expressive syntax. Currently using Java-esque epoch time for start and end time range in query_metric(..., start=<start_timestamp>, end=<end_timestamp>) seems to a bit awkward to work with (at least in 2.7). I have to add the following code to get metrics within the last 5 minutes.

def unix_time_millis(dt):
    return '{:.0f}'.format((dt - epoch).total_seconds() * 1000)
...
now = datetime.utcnow()
end = unix_time_millis(now)
start = unix_time_millis(now - timedelta(minutes=5))
data = client.query_metric(MetricType.Counter, metric_id, start=start, end=end)

It would be nice if query_metric() can support native Python datetime, time, timedelta.

single vs plural noun for metric types

When introspecting metric definition the Python client returns metric type of (in json) counter, gauge or string (single noun). If I want to build dynamic query to fetch metric data based on the returned definition I would have convert the returned type gauge into MetricType.Gauge whose string representation is gauges (plural). It would be convenient to pass the same returned type to query_metric(type, metric_id).

Migrate from dicts to classes

Considering that Hawkular-Metrics API will live and change, handling different dicts is becoming an issue and support in the IDEs is better for classes and objects.

GroupConditionInfo conditions list should be an instance attribute instead of a class attribute

When Instantiating a GroupConditionInfo object its conditions list is, in fact, a class attribute instead of an instance attribute.

The issue was solved in FullTrigger class by explicitly instantiating the relevant attributes in a constructor, a solution that can be applied to this object as well (with slight changes).

This issue has to do with the defaults mechanism of ApiOject, which IMHO should change.

Create 0.5.5 and 0.6.0

• 0.5.5 should include old version of alerts + metrics fixes from 0.5.4
• 0.6.0 should include alerts 2.0.0 support

No support for String values in metrics

I want to enable a push of data like docker, rpms versions etc, will need to have String values support in metrics.

HTTP Error 400: Admin token is not specified when metrics server >= v0.24

I believe Hawkular-Admin-Token was introduced recently.

  File "/workspace/git/.pyme/lib/python2.7/site-packages/hawkular/metrics.py", line 325, in query_tenants
    return self._get(self._get_tenants_url())
  File "/workspace/git/.pyme/lib/python2.7/site-packages/hawkular/client.py", line 263, in _get
    return self._http(url, 'GET')
  File "/workspace/git/.pyme/lib/python2.7/site-packages/hawkular/client.py", line 243, in _http
    self._handle_error(e)
  File "/workspace/git/.pyme/lib/python2.7/site-packages/hawkular/client.py", line 296, in _handle_error
    raise ee
hawkular.client.HawkularMetricsError: HTTP Error 400: Admin token is not specified. Use 'Hawkular-Admin-Token' header

Support stats endpoint

Support fetching aggregates and downsampled buckets by requesting from the /stats endpoint

Aggregate stats from a project

I'd like to use the client to get aggregate stats for projects from our OpenShift platform. However query_metrics_stats() insists on a metric_id while I rather just want to filter by tags to get my result. I reckon a simple if statement enabling to use metric_id=None could resolve this but of course being new to this stuff leaves some uncertainty. Anyway, the following change in metrics.py did the trick for me:

    def _get_single_id_url(self, previous_url, id):
        if id:
            return previous_url + '/{0}'.format(HawkularBaseClient.quote(id))
        else:
            return previous_url

Does this make sense or am I supposed to use the client in a different way to retrieve my stats?

Alerts module imports are broken in 0.5.0

Withon Python 2.7.12 this happens if you try to import everything:

>>> from hawkular import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'DampeningFullTrigger'
>>>

Add datapoint tagging

Next release (0.16.0) will reintroduce datapoint tagging. Add back this feature for writing datapoints.

Authentication support

Support for authentication methods, such as username/password and token authentication should be supported.

List Triggers doesn't work on Hawkular Alerts 2.0

Hello.

Calling List Trigger method with Hawkular Alerts 2.0 generates a bad request error.

HawkularMetricsError Traceback (most recent call last) <ipython-input-8-0b9fdd9cbf61> in <module>()
----> 1 alertsClient.list_triggers() /Users/user/.pyenv/versions/2.7.13/lib/python2.7/site-packages/hawkular/alerts.pyc in list_triggers(self, ids, tags) 133 tags = ','.join(tags) 134 url = self._service_url('triggers', {'tags': tags, 'ids': ids}) --> 135 triggers_dict = self._get(url) 136 return Trigger.list_to_object_list(triggers_dict) 137

/Users/user/.pyenv/versions/2.7.13/lib/python2.7/site-packages/hawkular/client.pyc in _get(self, url, **url_params) 261 url = '{0}?{1}'.format(url, params) 262 --> 263 return self._http(url, 'GET') 264 265 def _service_url(self, path, params=None):

/Usersuser/.pyenv/versions/2.7.13/lib/python2.7/site-packages/hawkular/client.pyc in _http(self, url, method, data, decoder, parse_json) 241 242 except Exception as e: --> 243 self._handle_error(e) 244 245 finally:

/Users/user/.pyenv/versions/2.7.13/lib/python2.7/site-packages/hawkular/client.pyc in _handle_error(self, e) 294 # Keep the original payload, couldn't parse it 295 ee.msg = err_json --> 296 raise ee 297 298 elif isinstance(e, URLError):

HawkularMetricsError: HTTP Error 400: Bad arguments: Unknown Query Parameter(s): [ids]

Support for multi-tenant operations

There are going to be cross tenant apis and we would need to have some support for those in clients, currently created with one tenant in mind.

Add SSL support

Add support for HTTPS requests.

Easier tenant handling

In OpenShift, each namespace_id is mapped to a tenant, so we'll need a simple way to fetch data from different tenants. In Golang client this was with a provided function, perhaps something like that could be provided here also to override given tenant?

Also, instantiation should be allowed without a given tenant

Username/authentication is broken in Python 2.7

In the refactoring of HawkularMetricsClient to use the common client.py, there was a change that broke the compatibility with Python 2.7. This affects the Alerts module also.

Offending line

b64 = base64.b64encode(bytes(self.username + ':' + self.password, encoding='utf-8'))

Since Python 2.7 does not accept more than one parameter to the bytes (unlike Python 3.x, which requires it). This wasn't catched because there's no tests for username/password combination.