pedrolucasom / datamigration

:computer: Spring Batch Application to migrate data from .csv files :bookmark_tabs: to a configured database with Docker :whale:.

💻 Spring Batch Application to migrate data from .csv files 📑 to a configured database with Docker 🐳.

Topics

1. About SpringBatch
2. About the Project
   • Implemented Job
   • Prerequisites
   • Run
   • Usage
   • Stop
3. Author
4. Contributing
5. Show your support
6. License

1. About SpringBatch

It is a framework that uses the Java Virtual Machine and the Spring Ecosystem to build batch applications. By definition, batch systems are systems that realize a process of a finite amount of data without interaction or interruption.

To learn more about this framework, view this article on the Notion: SpringBatch Article

2. About the Project

Implemented Job

It's a job responsible to execute two steps that will import people and banks from a .csv file to a configured datasource mysql.

These are the files that will be imported:

• people.csv
• bank.csv

If some person of the file people.csv is invalid with name, email or birthDate wrongs, the person isn't inserted on the datasource. In this case, the application will build a file in the path /files/output with the name invalid-people.csv and put these invalid records represented by id.

Prerequisites

• docker

Run

With the docker started, execute this command at the project root:

docker-compose up -d --build

Usage

Seeing the valid records in the datasource

Person records:

docker-compose exec database_app mysql -u root -papp#1234 -e "select * from app_batch.person;"

Bank records:

docker-compose exec database_app mysql -u root -papp#1234 -e "select * from app_batch.bank;"

Seeing the invalid records in the generated file

Navigate to /files/output/ inside of project root and open the file invalid-people.csv. You will see the invalid records id if exists.

Stop

To stop correctly:

docker-compose down -v

Remember to execute this command each time that you want change the parameter value.

3. Author

👤 Pedro Lucas

• Twitter: @PedroLucasOM
• Github: @PedroLucasOM
• LinkedIn: @PedroLucasOM

4. Contributing 🤝

Contributions, issues and feature requests are welcome!
Feel free to check issues page.

5. Show your support

Give a ⭐ if this project helped you!

6. License 📝

Copyright © 2021 Pedro Lucas.