alexxnica / online-analytics

An application for reporting OpenShift cluster usage analytics

Data Analytics Integration

This application syncs OpenShift user activity data with external analytics systems.

Usage

Using Woopra with default IDs

oc new-app -n openshift-infra -f ansible/roles/oso_analytics/files/user-analytics.yaml \
-p WOOPRA_ENABLED="true" \
-p WOOPRA_ENDPOINT="http://www.woopra.com/track/ce" \
-p WOOPRA_DOMAIN="YOURDOMAIN" \
-p USER_KEY_STRATEGY="[name/uid/annotation]" \
-p USER_KEY_ANNOTATION="[desired annotation]" \
-p LOG_LEVEL=5
-p CLUSTER_NAME="<int/stg/prod/..>"

The CLUSTER_NAME parameter is used to distinguish different environments (such as INT, STG, or a local test cluster). Its value by default is "kubernetes".

Note the 2 new flags USER_KEY_STRATEGY and USER_KEY_ANNOTATION. Key strategy refers to the keying method used for users in Woopra:

• name will key users by their user.Name
• uid will key users by their user.UID
• annotation will key users by an annotation specified in USER_KEY_ANNOTATION (this flag is only required if using strategy=annotation)

The annotation previously used in Devpreview was openshift.io/online-managed-id

Prometheus metrics enablement

The following flags enable Prometheus metrics gathering:

...
-p METRICS_PORT="8080" \
-p METRICS_COLLECT_RUNTIME="true" \
-p METRICS_COLLECT_WOOPRA="true" \
-p METRICS_COLLECT_QUEUE="true" \

The metrics are available at METRICS_PORT/metrics

METRICS_COLLECT_RUNTIME enables various runtime, Go, and process metrics provided by Prometheus

METRICS_COLLECT_WOOPRA enables Woopra latency metrics at /analytics_woopra_latency_seconds

METRICS_COLLECT_QUEUE enables analytics on the controller's internal events processing queue available at /analytics_queue_size_events, which returns the current size of the queue, and /analytics_events_handled, which returns the total number of events processed since the start of the controller.

Local development and testing

Using a Local Endpoint

oc new-app -n openshift-infra -f templates/user-analytics.yaml \
-p WOOPRA_ENABLED="false" \
-p LOCAL_ENDPOINT_ENABLED="true" \
-p LOG_LEVEL=5

Note! This will log to glog.V(5). A user must be logged in for an analytic to be counted.

Generate analytics by logging in and creating a project w/ basic app.

$ oc login
Authentication required for https://10.240.0.2:8443 (openshift)
Username: foo
Password:
Login successful.

You don't have any projects. You can try to create a new project, by running

    oc new-project <projectname>

$ oc new-project test
Now using project "test" on server "https://10.240.0.2:8443".

You can add applications to this project with the 'new-app' command. For example, try:

    oc new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-ex.git

to build a new example application in Ruby.
$ oc new-app centos/ruby-22-centos7~https://github.com/openshift/ruby-ex.git

Building

Build and test with make:

• make will run vendor dependencies and then install
• make build will compile the binary for the application
• make test will compile and run the unit tests
• make test-integration will compile and run the integration tests against an OpenShift master

Analytic Events

The following events are observed by a controller and sent to an analytics provider via Basic Authenticated GET w/ encoded URL.

User

user_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	user_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

user_deleted

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	user_deleted	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Deleted	ce_timestamp	obj.DeletionTimestamp	in milliseconds

pod_failed

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	pod_failed	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Failed	ce_timestamp	pod.Status.Condition.LastTransitionTime	in milliseconds
Failed	ce_reason	pod.Status.Condition.Reason	in milliseconds

ReplicationController

replicationcontroller_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	replicationcontroller_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds
Replica Count	ce_replica_count	obj.Spec.Replicas	The number of replicas of a pod

PersistentVolumeClaim

persistentvolumeclaim_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	persistentvolumeclaim_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds
Capacity	ce_capacity	obj.Spec.Capacity	The requested storage capacity
Access Modes	ce_access_modes	stringify(obj.Spec.AccessModes	The requested access modes for storage

persistentvolumeclaim_bound

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	persistentvolumeclaim_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Bound Date	ce_timestamp	obj.Condition.LastTransitionTime	in milliseconds
Request Capacity	ce_requested_capacity	obj.Spec.Capacity	The requested storage capacity
Request Access Modes	ce_requested_access_modes	stringify(obj.Spec.AccessModes	The requested access modes for storage
Actual Capacity	ce_actual_capacity	obj.Status.Capacity	The actual storage capacity of the backing volume
Actual Access Modes	ce_actual_access_modes	stringify(obj.Status.AccessModes	The actual storage capacity of the backing volume

deploymentconfig_failed

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	deploymentconfig_failed	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Failed	ce_timestamp	?	?

build_failed

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	build_failed	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Failed	ce_timestamp	?	?

Pod

pod_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	pod_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

ReplicationRontroller

replicationrontroller_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	replicationrontroller_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

PersistentVolumeClaim

persistentvolumeclaim_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	persistentvolumeclaim_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

Secret

secret_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	secret_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

Service

service_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	service_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

Namespace

namespace_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	namespace_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

Deployment

deployment_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	deployment_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

Route

route_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	route_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

Build

build_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	build_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

RoleBinding

rolebinding_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	rolebinding_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

Template

template_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	template_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds

ImageStream

imagestream_created

	parameter	v3 field	description
ID	cv_id	user.annotation["openshift.io/online-managed-id"]	User Identifier for analytics
Event	event	imagestream_created	Analytic event identifier
Name	ce_name	obj.Name	v3 object name (but not identifier/UID)
Namespace	ce_namespace	obj.Namespace	object's project/namespace
Created	ce_timestamp	obj.CreationTimestamp	in milliseconds