snowplow / snowplow-php-tracker

Snowplow event tracker for PHP. Add analytics into your PHP apps and scripts

PHP Analytics for Snowplow

Overview

Add analytics into your PHP apps and scripts with the Snowplow event tracker for PHP.

With this tracker you can collect event data from your PHP based applications, games and frameworks.

Quickstart & Testing

Make sure docker & docker-compose are installed.

• git clone [email protected]:snowplow/snowplow-php-tracker.git
• cd snowplow-php-tracker
• docker-compose run --rm snowplow composer.phar install
• docker-compose run --rm snowplow script/tests.sh

Find out more

Technical Docs	Contributing

Copyright and license

The Snowplow PHP Tracker is copyright 2014-2022 Snowplow Analytics Ltd.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

snowplow-php-tracker's Issues

Current implementation relies on shell_exec()

It comes from "ramsey/uuid" on Uuid.php.

Is it how it is going to be? To use it on production application server one will have to enable 'shell_exec()'method? New version uses 'passthru()' instead but in my opinion it is still not a way to go.

function_exists from Tracker.php line 154 should check function from the namespace in with is used? Whatever it supposed to do it does not work and (constant?) UUID_TYPE_TIME just does not exist.

Unit tests should not do any network connections at all.

The way to go with unit tests is:

• Refactor in the way to encapsulated all the code about interaction with remote server (curl, etc) in objects so the only responsibility of this objects will be those connections and nothing else. (this is about Emitters). At the end of refactoring there is no one variable or method with "debug" in name.
• Rewrite the tests in the way that tests uses mocks (https://phpunit.de/manual/current/en/test-doubles.html) of those objects instead of real objects. Since objects are only about remote connections there is no need for us to test it since curl library has its own tests in PHP core. In test if right methods are called on mocks the right quantity of time and with proper values of parameters.

$emitterCurlMock = //Here lots of lines that would set up mock!
// use: disableOriginalConstructor()
// set how what methods mock have and how many times they should be called
// Use returnCallback() to return somthing so code outside would "think" it deals with real object
// So at this point we know that right mock methods has been called right amount of time

$subject = Snowplow\Tracker\Subject()
$trucker = new Tracker($emitterCurlMock, $subject, "nodejs-tracker")

//Here we run our processing:
$trucker->trackUnstructEvent($json, $context, $timestamp)

//Here we get out from our tracker what data would be send (is not send since we use mock).
// So for instance if we send some JSON in a body check if this json is right:
$this->assertEquals($trucker->useApiToGetJsonThatWouldBeSend(), "{this is the json}");

Add background emitter using Golang Tracker

The idea is:

• Create a daemon written in Golang which should run all the time
• This daemon will embed the Golang Tracker and include some form of persistence (e.g. boltdb)
• The PHP Tracker will "fire and forget" all its events to the daemon

Update copyright notices to 2014-2019

Add event specific Mountebank Mocks

Add ability to write events to local socket

As opposed to using http or beanstalkd.

http://php.net/manual/de/function.socket-write.php

Idea is that a Go collector similar to https://github.com/LarsFronius/eventingestion_socket/blob/master/main.go could pick up those events and post on to Kinesis or Kafka.

/cc @jonalmeida @LarsFronius @ehartsuyker

Add setNetworkUserId to Subject class

Send in the network user ID aka sp cookie in the header of the request send to the Snowplow Collector, so that the collector can append nuid=blah

Fix typo POST_REQ_SCEHMA

Add support for dvce_sent_tstamp

This is related to: https://github.com/snowplow/snowplow/issues/2457

Add Packagist downloads button to README

https://github.com/satooshi/php-coveralls/blob/master/README.md

Add Coveralls code coverage to project

Ask Fred about this one

Add Quickstart section to README

UUID Library in composer is abandoned and has been replaced

The referenced library in composer.json (rhumsaa/uuid 2.8.*) has been abandoned and replaced by ramsey/uuid. This causes dependency issues for us (and most likely many others).

https://github.com/ramsey/rhumsaa-uuid/
https://github.com/ramsey/uuid

Add Asynchronous Curl Emitter as Emitter option for Tracker

https://segment.io/docs/libraries/php/#fork-curl-consumer

Add asynchronous request queue options

Add ability to attempt to resend failed event log files with file emitter

Explore mocks using Mountebank

https://groups.google.com/forum/#!topic/snowplow-user/Jxmj6Kaj074

Update payload_data schema to 1-0-2

Add Packagist version button to README

As per https://github.com/satooshi/php-coveralls/blob/master/README.md

Add CHANGELOG

Backfill with 0.1.0 and 0.2.0.

For 0.2.0, we want every ticket that is included in the 0.2.0 Milestone.

For an example see: https://github.com/snowplow/snowplow-ruby-tracker/blob/master/CHANGELOG

Potential high load crash under Windows with FileEmitter

Windows might crash under high load when using the FileEmitter. Using a simple implementation suggested by the documentation my windows machine slowly degrades when running a high load (in the range of dozens of logs a second) - up to the point where it completely hangs.

As I understood it from a senior dev, php in windows cannot spawn truly seperate processes, and slowly eats more and more resourceses when more processes are spawned.

Just a warning for those implementing this on windows - I managed to crash our servers with it

Lack of common interface for emmiters leading to mixed responsibility of objects.

Instead of binding all emitters with one base class, common interface should be used. Current solution is mixing responsibility across the objects. Like SocketEmitter inheriting buffer for curl...

Add dedicated Vagrant setup

Add copyright notices to each source file

Add ability to end background processes created by file emitter

At the moment the file emitter creates background processes on a timer; would be safer to be able to leave the workers running until we did not need them anymore and kill them from the Tracker itself.

Add Versionseye version checker button to README

Never done this before, can wait till 0.2.0

https://github.com/satooshi/php-coveralls/blob/master/README.md

Add Socket Emitter as Emitter option for Tracker

More performant than using requests.

See: https://github.com/segmentio/analytics-php/blob/master/lib/Segment/Consumer/Socket.php

Bump PHPUnit version to 4.5.0

Bump PHPUnit dependency to 4.5.0 to satisfy VersionEye.

Add ability to validate requests being sent from the event log of Mountebank

Change the test suite around so that we:

• Validate event payloads based on the data we get back from Mountebank

Essentially we can ingest all of the events sent to our local Mountebank server back into the test suite. If we could then test that the Payload we get back from Mountebank is what we assumed we sent out we can have end to end validation of the Request.

Add an automatic option for $tracker->flushEmitters(true)

As per the doco, would be nice to have an auto-flush.

uniqid doesn't generate a valid UUID

The tracker generates a UUID via uniqid, but that string won't validate in the Enrich process. EMR job fails here.

It would appear that the PHP tracker will need to include a UUID generator so that events can be accepted and processed.

Add Mountebank testing of tracker payload and collector response

Basic idea is to verify the request payload and the response.

Update PHPUnit dependency to 4.4.*

Add setIpAddress to Subject class

So that we can pass through the original (client) IP address. The querystring param already exists: &ip=

Add Travis support to project

Lack of fail over for emitters.

Lets say I want to track unstructEvent link:
$trucker->trackUnstructEvent($json, $context, $timestamp);

Now let say I want to do some actions in case it fails. There is no way to do that.

• No Exception that could be catched
• No false flag in return

Nothing. The same for all other calls.

It is like speed would be all we care about here and reliability is nothing.
But if speed is what we are focus on should we even write it in PHP? I would prefer slow but readable and safe solution...

After adding Exceptions please remove all 'print_r' from code. What do do with error messages that would be properly set in exceptions should be part of application that uses snowplow-tracker.

Curl Emitter has an infinite loop

Spotted by @https://github.com/mlively.

PR: https://github.com/mlively

Start sending payload_data version 1-0-1

Basically: once we add the useragent field (#17), the payload_data is no longer a valid 1-0-0. We need to bump the version being sent to 1-0-1, which will include the new useragent parameter.

Add License button to README

Add Code Climate button to README

Add Apache 2.0 license file

Remove rmccue/requests library as it is only used for Synchronous emitter

This dependency is only used for one emitter and can be very easily replaced by the standard inbuilt CURL library available to PHP.

Update contexts schema to 1-0-1

Which allows an empty [] array of contexts.

Add setDomainUserId to Subject class

As a complement to setNetworkUserId. Note that this one is querystring based, &duid=

For batched POSTs, correct nuid must be sent

The basic issue is: if you have a batch of 100 events, all with different nuids, in a batch, and that batch is all posted with the same nuid header, then the wrong nuid will be set against almost all of the events.

So if nuid is set for some events, we need to be maintaining a batch per individual nuid. Those batches can safely be POSTed with a common nuid in the header.

Does that make sense? @fblundun can explain more.

This is NFCCWD (Not For Changelog, Close When Done).

Add setUseragent to Subject class

So that you can pass through the user's original useragent. This is useful for people running PHP web frameworks (i.e. everybody).

Add File-Based Emitter as Emitter option for Tracker

https://github.com/segmentio/analytics-php/blob/master/send.php

Update PHPUnit dependency to 4.2.*

Add README

Follow same format as other trackers.

Add option to include page url with all events

We'd like to be able to have the current page URL included with our PHP structured events and transactions, just like it is with the JS tracker.

It seems like a logical place to include it would be as part of Subject, like Useragent and the other per-request fields live already.