dynatrace / dynatrace-api

Overview

The Dynatrace API examples demonstrate how to access various parts of Dynatrace, such as timeseries, problem feed and infrastructure topology as an API consumer.

Dynatrace API documentation

The full Dynatrace API documentation can be found here: https://help.dynatrace.com/api-documentation/v1/

API key

In order to use the Dynatrace API, you need an API key for your Dynatrace tenant. You can generate a key by following these steps

1. Go to your Dynatrace environment: https://{tenant}.live.dynatrace.com
2. Expand the side-bar menu on the left side of the screen and go to Settings and then Integration
3. Choose the Dynatrace API section
4. Click on Generate token to create a new API key
5. Enter a description label and submit the request
6. Expand the created key via clicking on the "Edit"-label, copy the token and use it in your Dynatrace API examples

Using the API with Dynatrace Managed environments

For Dynatrace Managed environments you need to use an URL with the following structure:

https://<hostname>/e/<tenant-id>/api/v1/

e.g.

https://sample.dynatrace-managed.com/e/12345678-1234-1234-1234-abcdef12/api/v1/

License

This module is provided under BSD-3-Clause license. Please check out the details in the LICENSE.txt

dynatrace-api's Issues

Getting Error(No module named 'dynatrace.environment_v1') when I add endpoint on Ping plugin

Getting Error(No module named 'dynatrace.environment_v1') when I add endpoint on Ping plugin:
Not sure if i am doing something wrong, Please advise ASAP

Ping
Version: 1.012 Type: ActiveGate extension Id: custom.remote.python.thirdparty_ping

AWS Cloudwacth - Dynatraces

Use the code to see the synthetic cloudwacth of aws in dynatraces it already appears monitored, however it does not generate any type of alert when there is unavailability. What should I do for dynatraces to alert it?

TypeError: must be unicode, not str

I am using Python 2.7.5 & Getting error while executing configurations/save.py

[dynatrace@xxxxx63 ~]$ python save.py
managementZones save list: 200
-1171957049078145000

Traceback (most recent call last):
File "save.py", line 34, in
main()
File "save.py", line 31, in main
saveList('managementZones')
File "save.py", line 25, in saveList
save('/api/config/v1/' + list_type + '/', entry['id'], tr.json())
File "save.py", line 15, in save
text_file.write("%s" % json.dumps(content))
TypeError: must be unicode, not str

[Third-party synthetic] Port check AG-extension breaks if "Test target host" is unreachable

@dlopes7 When the port check extension can't access the provided host name/IP, then it throws an exception and fails to send events/results.

Here's the end of the stacktrace:

File "C:/Program Files/dynatrace/remotepluginmodule/plugin_deployment/custom.remote.python.thirdparty_port\port_extension.py", line 59, in query
     step_success, step_response_time = test_port(target_ip, int(port))
   File "C:/Program Files/dynatrace/remotepluginmodule/plugin_deployment/custom.remote.python.thirdparty_port\port_extension.py", line 110, in test_port
     result = sock.connect_ex((ip, port))
 socket.gaierror: [Errno 11001] getaddrinfo failed

[Third-party synthetic] Port check - events are sent before test results are sent/monitor is created

The third-party synthetic port check extension sends events for failed ports before it sends the test result. The first test result implicitly creates the monitor. So, if a port fails on the first run, the event will result in an uncaught exception/400. As a result the monitor will never be created.

It should probably create the monitor before it checks any of the ports.

dynatrace-api/third-party-synthetic/active-gate-extensions/extension-third-party-port/src/port_extension.py

Line 73 in 98568a3

self.dt_client.report_simple_thirdparty_synthetic_test_event(

@dlopes7

Dynatrace operator failing to fetch the Token to rollout the oneagent installation in kubernets

We tried to setup AKS monitoring using the dynatrace by following classic full stack installation.

1. we created dynatrace operator using
   kubectl apply -f https://github.com/Dynatrace/dynatrace-operator/releases/download/v0.7.0/kubernetes.yaml

2)We created A access Token with all the defined scope permissions and created the secret using the command

kubectl -n dynatrace create secret generic dynatracesecret --from-literal="apiToken=dtxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

3. Applied the dynakube crd with below configurations enabled.

apiVersion: dynatrace.com/v1beta1
kind: DynaKube
metadata:
  name: mydynatrace
  namespace: dynatrace
spec:
  apiUrl: https://xxxxxx.dynatrace-managed.com/1xxxxxxxxxxxxxx/api
  tokens: dynatracesecret
  skipCertCheck: false
  networkZone: azurezone

  oneAgent:
    classicFullStack:
      image: ""

      tolerations:
        - effect: NoSchedule
          key: node-role.kubernetes.io/master
          operator: Exists
        - effect: NoSchedule
          key: node-role.kubernetes.io/control-plane
          operator: Exists

  activeGate:
    # Enables listed ActiveGate capabilities
    capabilities:
      - routing
      - kubernetes-monitoring
      - dynatrace-api

    image: ""

    resources:
      requests:
        cpu: 500m
        memory: 512Mi
      limits:
        cpu: 1000m
        memory: 1.5Gi

But the oneagent rollout is not happening and we could see the below error
The error stating as below
lastTransitionTime: "2022-08-09T19:39:54Z"
message: 'error when querying token on secret dynatrace:xxxxxxx: invalid
character ''<'' looking for beginning of value'

I decrypted the secret key and cross verified the API token, which is correct and not sure why this error is coming. Due to this the oneagent nodes are still not rolled out in the cluster.

{"level":"info","ts":"2022-08-09T19:55:54.663Z","logger":"dynakube-controller","msg":"problem with token detected","dynakube":"xxxxxxx","token":"APIToken","msg":"error when querying token on secret dynatrace:xxxxx: invalid character '<' looking for beginning of value"}
{"level":"info","ts":"2022-08-09T19:55:54.663Z","logger":"dynakube-status","msg":"updating DynaKube CR","cause":"token conditions updated","dynakube":"xxxxxxxx"}

Note: - we have Istio as service mesh on top of kubernetes.

DNS third-party monitor AG extension doesn't send events

@dlopes7 I'm hoping you've got the time to fix this ;)

The DNS extension fails to send events when it fails, because it provides the wrong syntheticEngineName, because the default is set to "Ping" as can be seen in this screenshot.

Error(Expecting value: line 6 column 1 (char 5))

Error during setting up an endpoint using all of the

extensions

[Third-party synthetic] Port/Ping/DNS check, only option is to alert on the first failure

We began to implement the ping/port plugin to resolve a problem with AIX agents that take a full 5 min of downtime to alert. The majority of system cashes are rebooting in that time and not catching server outages. Server availability shows 100% after the crash. The result is the unix team is not getting server down alerts, the applications see impact however the unix team needs these events to know a system went down.

We are using this plugin to address this however have the reverse problem with the plugin. The plugin defaults to executing 1 min and does not allow for configuring the consecutive failures. One ping or port failure keeps a problem open for 15 min. We need the ability to make this less sensitive. We've held off deploying port checks to the 5000 unix servers until this is resolved. The current 100+ monitors sends about 10 alerts a day.

In file: dynatrace-api/ExcelExport/dt-excel.py, column_cells[0].column should be column_cells[0].column_letter

In line 108 of file dynatrace-api/ExcelExport/dt-excel.py, column_cells[0].column returns a column number but ws.column_dimensions[] should be indexed with a string.

python dt-excel.py

Status: 200
Status: 200
Traceback (most recent call last):
File "dt-excel.py", line 108, in
ws.column_dimensions[column_cells[0].column].width = length+1
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/openpyxl/utils/bound_dictionary.py", line 26, in getitem
setattr(value, self.reference, key)
File "/opt/rh/python27/root/usr/lib/python2.7/site-packages/openpyxl/descriptors/base.py", line 44, in set
raise TypeError('expected ' + str(self.expected_type))
TypeError: expected <type 'basestring'>

python -V

Python 2.7.13

403 - "Token is missing required scope. Use one of: ExternalSyntheticIntegration"

Hi Wolfgang,

I have studied Your Code, and adapted it to my Dynatrace instance and api-token.

When I do a POST i get the below response back :

{
"error": {
"code": 403,
"message": "Token is missing required scope. Use one of: ExternalSyntheticIntegration"
}
}

Can't find any documentation on this anywhere.

Please advice

/Per Bastholm - Red Ocean DK.

Dynatrace Synthetic plugin DNS/Port/Ping no error condition handling

We began to implement the ping/port plugin to resolve a problem with AIX agents that take a full 5 min of downtime to alert. The majority of system cashes are rebooting in that time and not catching server outages. Server availability shows 100% after the crash. The result is the unix team is not getting server down alerts, the applications see impact however the unix team needs these events to know a system went down.

We are using this plugin to address this however have the reverse problem with the plugin. The plugin defaults to executing 1 min and does not allow for configuring the consecutive failures. One ping or port failure keeps a problem open for 15 min. We need the ability to make this less sensitive. We've held off deploying port checks to the 5000 unix servers until this is resolved. The current 100+ monitors sends about 10 alerts a day.

dynatrace_ third-party-port_check error

Hi.

we detected a new error using third party port_check.

Error(Error making request to https://iki30752.live.dynatrace.com/api/v1/synthetic/ext/tests: <Response [400]>. Response: {"error":{"code":400,"message":"The property 'locationResult/startTimestamp' is only allowed to lie 10 minutes in the future at max, therefo)

 

ping monitoring from multiple Locationg

@myieye and @dlopes7

I am trying to monitor the ping availability of the network device from different Locations,

For example if I deploy ping extension on 2 different Environmental ActiveGates located in different datacenters , Is there a way i can monitor the ping response of the device from both the location ActiveGates in a single monitor, like how we do synthetic single url monitoring from multiple ActiveGate locations

Was looking for if there a way we can add/choose multiple Active Gates while adding the ping end point as shown in the attached screen shot or if we add 2 different ping monitors for the same device choosing 2 different Active gates will both the monitors ActiveGate reports to the single ping monitor or it creates 2 different ping monitors

Thank You!

Only first line checked when using pattern match in third party commands SSH

dynatrace-api/third-party-synthetic/active-gate-extensions/extension-third-party-linux-commands/src/remote_client.py

Line 112 in 44a70ef

return False, f"Pattern {pattern} did not match the output"

Where the return statement is means that no matter how many lines of output there is only one line will be evaluated.

[AWS Cloudwatch] No test result returned.Error: Protocol error (Runtime.callFunctionOn): Target closed error

Recent usage of the dynatrace-canary-exporter.js throws a "No test result returned.Error: Protocol error (Runtime.callFunctionOn): Target closed error" on the AWS side. Data does get passed properly to Dynatrace Synthetic

Using the following documentation: https://aws.amazon.com/blogs/apn/how-to-monitor-amazon-cloudwatch-synthetics-from-dynatrace/

Added async to the following functions and it worked properly - sorry it was done indiscriminately. Had to get this working quickly for a customer. Was hoping you would be able to test:

async function getCanaryHandlerName()
async function canaryConfig()
async function buildDtTestResult(stepResults, startTime, canaryWasSuccessful)
async function getRequestUrl(urlOrOptions)
async function handleCanaryRequest(request, urlOrOptions, onStepResult)
async function initDtTestResultHandlers(page, onStepResult)
async function afterCanary()
async function beforeCanary()

` try {
async beforeCanary();
} catch (error) {
log.error('DT: An unexpected error occured in the before canary handler', error);
}

      try {
          originalHandlerReturnValue = await originalHandler.call(exports);
      } catch (error) {
          log.info(`DT: Original handler threw an error (${error})`);
          originalHandlerError = error;
      } finally {
          // Prevent unexpected DT-specific errors from breaking the canary
          try {
              try {
                  async afterCanary();
              } catch (error) {
                  log.error('DT: An unexpected error occured in the after canary handler', error);
              }
              await sendResultsToDynatrace(_stepResults$, startTime, !originalHandlerError);
          } catch (error) {
              log.error('DT: An unexpected error occured while building and posting exernal-monitor result to Dynatrace', error);
          }
      }`

Activegate Proxy not working

Trying to use the port plugin module. I'm getting an error when attempting to connect to the API using the plugin. Our activegates use a proxy to connect out to the SaaS solution, there does not appear to be a way to add this proxy to the configuration.

The property 'scheduleIntervalInSeconds' must not be null or empty

py externalSyntheticExample.py Synthetic request - Response time:0.658543s Status code:200 Success:True <Response [400]> {"error":{"code":400,"message":"The property 'scheduleIntervalInSeconds' must not be null or empty."}}

I have only changed:
YOUR_DT_API_URL
YOUR_DT_API_TOKEN

payload var visibly contains:
"scheduleIntervalInSeconds": 60

python3.6.5

[Third-party synthetic] image URLs don't work

For all 3 third-party synthetic extensions the image URLs don't work. They can be fixed by removing /tree from the path.

E.g.

dynatrace-api/third-party-synthetic/active-gate-extensions/extension-third-party-dns/src/dns_extension.py

Line 66 in 98568a3

icon_url="https://raw.githubusercontent.com/Dynatrace/dynatrace-api/tree/master/third-party-synthetic/active-gate-extensions/extension-third-party-dns/dns.png",

vs
https://raw.githubusercontent.com/Dynatrace/dynatrace-api/master/third-party-synthetic/active-gate-extensions/extension-third-party-dns/dns.png

@dlopes7

Failed in the extension DNS - Failed executing the DNS test: Text input is malformed

The dns extension gives continuous errors, after an hour that it works correctly it disconnects, in the logs the error that I attach in the image is shown

Ping and Port extensions continually open a new problem every hour.

In the event a problem is opened by the ping and port extensions which lasts more than 1 hour, the extension closes the problem and opens a new one.
As a result, if a problem is open for 24 hours you are likely to see 24 problems generated, and if we have alerts configured that equates to 24 alerts sent in a day for the same problem.
The extensions should only alert once for one problem and keep the problem open until it is resolved.