GithubHelp home page GithubHelp logo

scale8 / scale8-tag-manager-and-analytics Goto Github PK

View Code? Open in Web Editor NEW
298.0 5.0 17.0 4.02 MB

Website analytics, JavaScript error tracking + analytics, tag manager, data ingest endpoint creation (tracking pixels). GDPR + CCPA compliant.

Home Page: https://scale8.com

License: GNU Affero General Public License v3.0

Dockerfile 0.02% Shell 0.01% JavaScript 0.25% TypeScript 96.49% Twig 0.28% Batchfile 0.13% Java 2.82%
app website privacy gdpr tag-manager analytics google-tag-manager google-analytics statistics charts

scale8-tag-manager-and-analytics's Introduction

Scale8 - Tag Manager & Data Manager

Scale8 - An open-source and privacy friendly alternative to GA & GTM | Product Hunt


๐Ÿ‘€ *MIT License is applied to Platforms to enable network distribution. The rest of the project remains under AGPL v3.

A simple, yet powerful Google Tag Manager alternative that is fully open-sourced and privacy-friendly. Our in-built Google Analytics alternative doesn't use any cookies and is fully GDPR, CCPA and PECR compliant. We've made it quick and easy to try a demo or simply host it yourself in production. We'll also be offering a cloud hosted version in the near future.

The Data Manager provides pluggable backends to send data anywhere, and we've got a pipeline to increase both connectors and features. In just a few clicks, design a schema, configure a back-end and create an endpoint to send data to. The Tag Manager uses this same underlying tech to provide analytics.

Our mission has been to create an intuitive Tag Management tool that can be fully extended via custom platforms (built by our community!). We're a small team that is passionate about building great open-source software and hopefully reducing the market share of much larger companies. We are not a fan of increasing corporate or government surveillance, and we fundamentally believe that your website's users deserve better - this however needs to be balanced with the business requirement to monetise a property and personal data being shared.

Running the project

Here is a one-line example of how to get started if you already have Docker Compose installed.

~> curl -L https://github.com/scale8/scale8/raw/master/docker-compose.yml | docker-compose -f - up

Once the command above has started the services, point your browser at http://127.0.0.1:8080. You will be prompted to enter some basic details to configure the project.

Or if you have checked out the project already and installed the dependencies via ~> yarn install:all and wish to run it locally: -

~> yarn run:all

Platform structure

We've architected the entire platform to support docker and limited where possible complex build processes and mounting of extra volumes. We're scripting hard to provide complete one-click production ready setups for both AWS, Google Cloud and custom kubernetes setups.

Component Description
API Contains all the business logic and provides GraphQL / RESTFul endpoints that power the UI
Common A small common library that is shared between project components
Edge Designed to scale horizontally, the edge server is responsible for delivering platforms at the edge and also tracking data in
Platforms Custom platforms created by the community and Scale8 that extend the functionality of the Tag Manager
Router A simple router, created using Nginx and designed for spinning up the self-hosted version quickly
UI Provides a static build of the UI using Next.js & React

Development - Install dependencies and build

  • Install all project dependencies: ~> yarn install:all
  • Build all projects: ~> yarn build:all

Analytics-only mode

As not everyone has the requirement to use a full Tag Manager, we've also provided an option to generate an extremely light version that only requires a few lines of JavaScript.

We have also provided full support for tracking Single Page Applications (SPAs) and hash-routing. These can be enabled when you install your application.

Features overview

Tag Manager

  • Tags: Create tags that automatically load, or include them on the fly in your application.
  • Rule Groups: Tags can be associated with one or more Rule Groups that can be run in parallel.
  • Rules: Rule groups can contain multiple rules with the first rule winning. Rules can also be repeated once they've won.
  • Events: Attach events to rules as triggers. Once attached they will listen for your chosen event to occur before allowing a rule to proceed.
  • Conditions & Exceptions: Attach conditions and exceptions to rules. These will test your data containers / data layers.
  • Actions: Once events and/or conditions are satisfied, then trigger one or more actions. App-state enables actions to trigger other actions.
  • Action Groups: Action groups contain actions and provide a useful container to separate logic.
  • Action Group Distributions: Run experiments (A/B tests) or monitor just a percentage of traffic. Easily distribute your actions across your users.
  • Installed Platforms: Install 3rd party platforms submitted by our community. These platforms extend Events, Data Containers and Actions.
  • Installed Platform Revisions: A revision is locked to a platform revision. Upgrade and deploy with confidence as connect platforms change.
  • Environments: Create as many environments as your team requires, matching your existing workflows.
  • Revision Control: All changes happen within a new revision. Preview and test without complex release cycles. Deploy with confidence.
  • Debugging: Debug your application with ease. Remote debugging coming soon!
  • Analytics: A simple and intuitive set of analytics tools. Event tracking with Event Group support.
  • Analytics-only Mode: Run an incredibly lightweight snippet of JavaScript code to gain full analytics data.

Data Manager

  • Ingest Endpoints: Generate new endpoints to send data to in just a few clicks.
  • Data Structure: Compose a data structure that is compatible with any backend we offer. Design once, use everywhere.
  • Revisions: Everything sits under revision control. Release with confidence.
  • Usage Monitoring: Track endpoint requests and data usage.
  • Environments: Test your latest data structure without compromising your production setup. Roll-back in seconds.

All products

  • Team Support: Invite team members and manage access levels.
  • Full API Support: Automate everything - The UI is static and calls the GraphQL API for everything.

Core concepts

We've done our best to simplify the process of organising and managing tags across one or more web properties. The Tag Manager is based on an event-driven, rule-based engine whereby a rule is triggered when Events and Conditions and Exceptions are all met, resulting in one or more Actions being taken.

Events

A triggered event within the browser can be easily managed by our simple interface. A clicked link, a page coming into focus, or a submitted form are all examples of events you may wish to listen to.

Conditions & Exceptions

A condition is a test made on a data container and aims to assert that some key belonging to a data container object is aligned with the expected value. An exception aims to make the same test, but instead exclude the rule should the test pass.

Actions

Upon meeting the rule's required events and condition, an action will be fired. Learn more about actions here.

Tech stack

TLDR;

  • UI: Next.js, TypeScript, React, Apollo, GraphQL, Material UI.
  • Edge: Micronaut, Java, Maven.
  • Router: Nginx - Used for self-hosted version only and not required for all setups.
  • API: InversifyJS, Node, TypeScript, Express, MongoDB, Apollo, GraphQL, Jest.
  • Platforms: TypeScript, Webpack, Jest.

A longer explanation

We originally started off with a TypeScript front-end and Scala driven back-end. To provide the code transparency we wanted, coupled with a decision to open-source the project required us to move to almost purely TypeScript with just the edge server in Java. We fundamentally believe the future of this project sits with the open-source community now and there are more TypeScript and Java engineers than Scala engineers.

Just before releasing we've moved from CRA to Next.js too. We've also introduced inversion of control (IOC) in our TypeScript back-end. Our goal is for this project to be as extensible as possible. A plug and play model provides us with the opportunity to provide multiple services and prevent lock-ins.

container.bind<BaseStorage>(TYPES.BackendStorage).to(MongoDBStorage).inSingletonScope();
container.bind<BaseDatabase>(TYPES.BackendDatabase).to(MongoDb).inSingletonScope();
container.bind<BaseLogger>(TYPES.BackendLogger).to(ConsoleLogger).inSingletonScope();
container.bind<BaseEmail>(TYPES.BackendEmail).to(Mailer).inSingletonScope();
container.bind<BaseConfig>(TYPES.BackendConfig).to(EnvironmentConfig).inSingletonScope();

We've taken a similar approach with our Java code too.

@Replaces(StorageInterface.class)
@Singleton
@Requires(property = "backend-storage", value = "google")
public class GoogleStorage implements StorageInterface {
    ...
}

At present, we provide only MongoDB and Google's BigQuery for analytics. MongoDB is only suitable for small projects and for the purpose of demo / testing. We have full Clickhouse support in our yet to be released cloud version, and we'll be adding this support in the self-hosted version shortly along with Redshift and Postgres.

We've also removed our hourly aggregation pipelines from the self-hosted version. This added an unnecessary degree of complexity for the vast majority of use cases. There is a plan to potentially re-introduce this later and should anyone require it, please let us know.

Documentation

All of our documentation is designed for the cloud version of the product, however we'll be adding more detailed documentation for the self-hosted version soon. We've hidden away some features such as payment processing and SSL termination for custom domains that are not relevant to the self-hosted version.

The entire UI is powered by GraphQL using Apollo and the API documentation is automatically generated and easily navigated.

Support us!

All we ask is that you star or watch this on GitHub if you like the project. We'd love people to share and blog about this too! All issues will be responded to quickly in GitHub, and we're excited to see where this goes next.

Authors

Author GitHub LinkedIn
Christopher Beck Christopher Beck on GitHub Christopher Beck on LinkedIn
Alessandro Barzanti Alessandro Barzanti on GitHub Alessandro Barzanti on LinkedIn

scale8-tag-manager-and-analytics's People

Contributors

a-barzanti avatar cwbeck avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

scale8-tag-manager-and-analytics's Issues

Admin Panel

  • Check current behaviour
  • Remove approvals
  • List all orgs + change org to self billing. (API work required).
    • Check no sub exists. If it does throw and error.

Test & Update - Payment System

  • Update pricing and plans to reflect changes.
  • Test different journeys with failing card payments.
  • Check EU compliance (VAT).
  • Process for updating (changing pricing). Grandfathering setup? (i.e changing a $5 plan to a $6 plan.)
    • Plans have to persist, but need a set of plans that are available to select. Not all plans will be available to select in the future.

End of free trial.

  • Make sure if negative, "your trial ended 'n' days ago".
  • If no days left <=0, then redirect to settings of org and show message prompting for update of plan to paid.
    • Only do this at the account level. So if moving to tag manager and account is not on a plan after free trail, prompt.

Document minimum resources for selfhosting

Is your feature request related to a problem? Please describe.

Scale8 looks impressive. I'm currently setting up new infrastructure and now I'm curious how many resources each of the Scale8 services need and what specification you'd recommend.
And maybe there's even something to document on how to tweak a few things in case a lot of resources are needed (GitLab has a reference that looks like this in case this would help as inspiration).

Describe the solution you'd like
Some documentation on resources.

Describe alternatives you've considered
Not having any documentation regarding resources

PS: I'm currently building new infrastructure and I'm considering to try Scale8.
OpenID Connect seems to be the only missing feature and I saw that you have it already on the roadmap which is great. ๐Ÿ‘

Feature: Add mapped volumes to docker-compose

Is your feature request related to a problem? Please describe.
If the containers are removed, recreated or migrated then all tag/analytics information is lost.

Describe the solution you'd like
Volume mappings which allows for mapped folders and data to be hosted as per the user's requirements for db/login/tag and platform info migration if/when necessary, something like:

mongodb:
  volumes:
    - ./data:/data

Describe alternatives you've considered
Not sure what other container folders may need to be mapped so various configs persist

Additional context
Add any other context or screenshots about the feature request here.

Terraform GCP.

Move GCP under Terraform. (GCS + BigQuery currently missing).

Self-hosted version provides http src as `127.0.0.1:3000`

Describe the bug
I created a reverse proxy to access the gui via my sub.domain.com. The documentation I found doesn't really give info for this, however the tags being generated for applications still show src=http://127.0.0.1:3000/edge/xxxxxxxxxxxxxxxxxxx. I have tried changing the PROXY_LOCATION and the S8_EDGE_SERVER to my sub.domain.com but the generated tags still show the src listed above.

How do we change the tag so that it will work via reverse proxy? Doing it manually doesn't seem to work

To Reproduce
Steps to reproduce the behavior:

  1. Create service via docker-compose (including trying to use https://mysub.domain.com as proxy location and edge server)
  2. Create reverse proxy to access via 'mysub.domain.com`
  3. Create User
  4. Log in
  5. Create application
  6. View generated tag

Expected behavior
Generated tag would follow a template which allowed for reverse proxies, something like:

<script src="https://sub.domain.com/[platformID]/tm.js" async></script>

Additional context
Hardware

  • Synology NAS (DS920+)
  • Nameserver is Cloudflare

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.