gompact / gofixtures

Golang lib/command line tool to load fixtures to databases, useful for testing

gofixtures

version: 3.0.0

A small command line tool written in Go lang, that loads fixtures and insert it's records to a database. Currently supports YAML and JSON files through CLI, and supports PostgreSQL only.

Supported File Types

• JSON
• YAML
• CSV

Supported Databases

• PostgreSQL
• MySQL
• SQLServer
• MongoDB
• Redis
• Cassandra
• Firebase

Install

This will install gofixtures to your $GOPATH/bin

$ go get github.com/schehata/gofixtures/v3

Usage

To start using gofixtures you need to have a configuration file, that includes information on which db driver should it use, and what are the credentials It could be written in YAML or JSON, here is an example for .gofixtures.yaml which will be automatically loaded by gofixtures:

import (
    "github.com/schehata/gofixutres/v3"
    "github.com/schehata/gofixutres/v3/entity"
)

func main() {
    config := entity.Config {
      entity.DBConfig{
        Driver: "postgres"
      }
    }
    gf, err := gofixutres.New(config)
    if err != nil {
      log.Fatal(err)
    }
    err = gf.LoadFromFiles(string[]{"./fixtures/countries.yml"})
    if err != nil {
      log.Fatal(err)
    }
}

example of ./fixtures/countries.yml:

table: countries
records:
  - name: Egypt
  - capital: Cairo
  - name: Germany
    capital: Berlin
  - name: Netherlands
    capital: Amsterdam

3. the previous yaml file inserts three records into table countries
4. gofixtures will parse each record and insert it into the database

Notes on CSV Support

• Column names are read from the first row
• Filename will be used as a tablename
• For now only comma ',' is allowed as a separator, will work on providing CLI flags to change that as needed

Usage from Command Line

You can use the gofixtures CLI tool to quickly load data into datastores from the command line, check the CLI docs.

gofixtures's Issues

Ability to print SQL statements for debugging using a cli flag

• Create a cli flag that allows the user to read SQL statements in the CLI before they being sent to the db.

• If the flag is called --debug we will have to think how would this affect mongodb for example ?

Add Support for csv

Add a new parser that allows reading from CSV files and feed its data to the database.

Testing ZenHub Issues

Create global configuration file Instead of db configuration only

change the db configuration file to be more flexible and be be able to include different configuration types. for example configurations could be added for the CSV parser.

the yaml or json file could be a map with keys of a type name, for example

db:
   user: ....
csv:
   separator: ...

sub epic

group queries by table name, use transactions and roll back on error

group queries by table name, use transactions and roll back on error

Add support for JSON fixtures

Add new function to clear database tables

This could be very useful for testing, as gofixutres could be used as a library to setup testing data and then clear the database at the end of each test. in order to do that a clear function is needed, as a library function and as a cli command.

For that purpose the CLI should have commands instead of just assuming loading data each time, so these commands could be

• load : which loads data from files
• clear : which clears database
• version: which shows current version of the gofixtures cli

Create sql tables if they doesn't exist using cli flag

Currently gofixtures will fail to insert data if the targeted table doesn't exit.

Solution:

• add a cli flag that allows the user to enforce creation of the table if it doesn't exist
• would it be possible to guess the type of column from the data ?

Details:

• Create cli flag called "autoTables" which takes a boolean value, true or false
• if its true, check if table exists, if not create it.

testing zenhub epic

Improve the CLI logs appearance, and use prefixes

• Add Prefix for different CLI log types, specially that in the future there will be debug messages.
• Use Colord outputs for different logging types

e.g:

[Warning]: some warning
[Info]: info messages
[Error]: ...

Fake Mode, Ability to just print sql statement without real insert to the db

• Allow users to call gofixtures and print sql statements of data read from the file without really inserting anything to a db.

this could be useful for someone, although i am not sure if there is a real case for this.

• maybe just create a new fake Postgres DAL, which is just a struct that implements the datastore interface and does nothing but printing the sql statements.

Restructure the project structure so it could be used inside unittests of other projects

Currently the project is focused around being a binary. It would be best if the cli moved inside a cmd package which can be installed if needed, otherwise developers would be able to import gofixtures inside there tests and use it load fixtures or clear them.