GithubHelp home page GithubHelp logo

cregev / finala Goto Github PK

View Code? Open in Web Editor NEW

This project forked from similarweb/finala

0.0 1.0 0.0 2.92 MB

A resource cloud scanner that analyzes and reports wasteful and unused resources to cut unwanted expenses.

License: Other

Dockerfile 0.20% Makefile 0.91% Go 78.10% Shell 1.30% HTML 2.85% JavaScript 14.44% CSS 2.21%

finala's Introduction

Finala

Build Status

A resource cloud scanner that analyzes and reports about wasteful and unused resources to cut unwanted expenses. The tool is based on yaml definitions (no code), by default configuration OR given yaml file and the report output will be saved in a given storage.

Currently it is implemented for AWS resources (RDS, EC2 instances, DynamoDB, ElasticCache, documentDB, ELB and etc) and can be easily extended.

Example: Unused RDS report

alt Resources

Example: Summary of all resources

alt Summary

+-------------------------------------------------------------------------------------------------+
| ID           | REGION    | INSTANCE TYPE | MULTI AZ | ENGINE | PRICE PER HOUR | PRICE PER MONTH |
+-------------------------------------------------------------------------------------------------+
| arn:aws:rds: | us-east-1 | db.m3.medium  | true     | mysql  | 0.18           | 129.6           |
| arn:aws:rds: | us-east-1 | db.t2.medium  | false    | mysql  | 0.068          | 48.96           |
+-------------------------------------------------------------------------------------------------+

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

How To Use

All the logic is contained inside config.yaml.

  1. Setup your Cloud provider (currently AWS only) credentials and accounts you want to analyze.
providers:
  aws:
  - name: <ACCOUNT_NAME>
    # Environment variables will be used in case if these variables are absent
    access_key: <ACCESS_KEY>
    secret_key: <SECRET_KEY>
    session_token: "" # Optional variable, on default this variable not set
    regions:
      - <REGION>
  1. Let it run!

Optional: There are defaults but, You can specify your own resources to analyze and change the metrics thresholds.

For example:

If you want to test RDS resources that had zero connections in the last week:

rds:
    - description: Database connection count
        metrics:
        - name: DatabaseConnections
            statistic: Sum
        period: 24h 
        start_time: 168h # 24(h) * 7(d) = 168h
        constraint:
        operator: "=="
        value: 0

Prerequisites

  1. AWS access key and secret key (with readonly access)
  2. Optional: Docker
  3. Browser (for the Dashboard UI)
  4. Tested NodeJS V12.13

Installing

  1. Build from source
$ git clone [email protected]:kaplanelad/finala.git
$ make build

To run (with static AWS credentials defined in config.yaml):

$  ./finala aws -c ${PWD}/config.yaml

With environment variables:

$ export AWS_ACCESS_KEY_ID=...
$ export AWS_SECRET_ACCESS_KEY=...
$ export AWS_SESSION_TOKEN=...
$ export AWS_SECURITY_TOKEN=...

$  ./finala aws -c ${PWD}/config.yaml

With aws-vault:

$ aws-vault exec aws-account-profile -- ./finala aws -c ${PWD}/config.yaml

With Docker and environment variables (untested):

$ export AWS_ACCESS_KEY_ID=...
$ export AWS_SECRET_ACCESS_KEY=...
$ export AWS_SESSION_TOKEN=...
$ export AWS_SECURITY_TOKEN=...

$ docker run -it --rm \
  -e AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID}" \ 
  -e AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}" \
  -e AWS_SESSION_TOKEN="${AWS_SESSION_TOKEN}" \
  -e AWS_SECURITY_TOKEN="${AWS_SECURITY_TOKEN}" \
  -v ${PWD}/config.yaml:config.yaml:ro \
  dockerhub_id/image_name

For config example

  1. Download the binary https://github.com/similarweb/finala/releases

Release New Version

To release a new version run the command:

$ make release

Development

To run Finala from the source code run the command:

$ go run main.go aws -c ./config.yaml

To run Finala UI with run the command:

$ cd ui
$ npm install
$ npm run dev

Then browse to: http://127.0.0.1:8081/static/

Dynamic parameters

By default all the data will save in sqlite in local folder

-c, --config string                      config file path
    --disable-clear-storage              Clear storage data
    --disable-ui                         Disable UI dashboard view
-h, --help                               help for finala
    --storage-connection-string string   Storage connection string. Default will be DB.db (default "DB.db")
    --storage-driver string              Storage driver. (Options: mysql,postgres,sqlite3,mssql) (default "sqlite3")
    --ui-port int                        UI port. default 9090 (default 9090)

Running the tests

$ make test

$ make test-html

Use-cases:

The full working example can be found in config.yaml.


  1. Find EC2 instances has less that 5% CPU usage.
ec2:
    - description: EC2 CPU utilization 
        metrics:
        - name: CPUUtilization
            statistic: Maximum
        period: 24h 
        start_time: 168h # 24h * 7d
        constraint:
        operator: "<"
        value: 5
  1. Find RDS DB's that had zero connections in the last week. Maybe we can destroy the DB :)?
rds:
    - description: Database connection count
        metrics: 
        ### Start: Cloudwatch metrics ###
        - name: DatabaseConnections
            statistic: Sum
        period: 24h  
        start_time: 168h # 24h * 7d
        ### End: Cloudwatch metrics ###
        constraint:
        operator: "=="
        value: 0
  1. Find ELB's that had zero traffic (requests) in the last week. Maybe we can destroy the ELB :)?
elb:
    - description: Loadbalancer requests count
        ### Start: Cloudwatch metrics ###
        metrics:
        - name: RequestCount
            statistic: Sum
        period: 24h 
        start_time: 168h # 24h * 7d 
        ### End: Cloudwatch metrics ###
        constraint:
        operator: "=="
        value: 0   
  1. Display the actual capacity usage VS the capacity requested (in percentage).
dynamodb:
    - description: Provisioned read capacity units
        ### Start: Cloudwatch metrics ###
        metrics:
        - name: ConsumedReadCapacityUnits
            statistic: Sum
        - name: ProvisionedReadCapacityUnits
            statistic: Sum
        period: 24h 
        start_time: 168h # 24h * 7d
        ### End: Cloudwatch metrics ###
        constraint:
        formula: ConsumedReadCapacityUnits / ProvisionedReadCapacityUnits * 100 # specify any formula 
        operator: "<"
        value: 10

Built With

Contributing

All pull requests and issues are more then welcome! Please see Contribution guidelines.

finala's People

Contributors

dependabot[bot] avatar isan-rivkin avatar kaplanelad avatar rrun1et avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.