This project consists of a synthetic data generation agent that uses the Anthropic API to analyze and generate new data based on a given sample CSV file. The generated data is saved in a new CSV file.
- agents.py: The main script for reading the sample CSV file, analyzing it using the Anthropic API, and generating new synthetic data.
- prompts.py: Contains the prompt templates used for data analysis and generation.
- Dockerfile: Docker configuration file for setting up the container environment.
- requirements.txt: Lists the Python dependencies required for the project.
- Docker
- Anthropic API Key
Ensure your project directory has the following structure:
/project-root
│── agents.py
│── prompts.py
│── Dockerfile
│── requirements.txt
│── data/
└── fineTuningSampleDataset.csv
-
Anthropic API Key: Ensure you have your Anthropic API Key ready as it will be needed during execution.
-
Sample CSV File: Place your sample CSV file (e.g.,
fineTuningSampleDataset.csv) in thedatadirectory.
Navigate to the project directory and build the Docker image using the following command:
docker build -t synthetic-data-agent .
Run the Docker container with the necessary volume mounting and execute the script:
docker run -it -v "/path/to/project-root/data:/app/data" synthetic-data-agentReplace /path/to/project-root with the actual path to your project directory on the host machine.
- API Key: When prompted, enter your Anthropic API Key.
- CSV File: Enter the name of your CSV file (e.g.,
fineTuningSampleDataset.csv). - Number of Rows: Enter the number of rows you want to generate (e.g., 65).
The script will read the sample CSV file, analyze it, and generate the specified number of new rows. The generated data will be saved in a new CSV file (new_dataset.csv) in the data directory.
This script contains the following functions:
- read_csv(file_path): Reads the sample CSV file and returns its contents as a list of lists.
- save_csv(data, output_file, headers=None): Saves the provided data into a CSV file. If headers are provided, they are written to the file first.
- analyzer_agent(sample_data): Sends the sample data to the Anthropic API for analysis and returns the analysis results.
- generator_agent(analysis_result, sample_data, num_rows=30): Sends the analysis results and sample data to the Anthropic API to generate new data and returns the generated rows.
This script contains the prompt templates used for data analysis and generation:
- ANALYZER_SYSTEM_PROMPT: Prompt for the analyzer agent.
- GENERATOR_SYSTEM_PROMPT: Prompt for the generator agent.
- ANALYZER_USER_PROMPT: User prompt for the analyzer agent.
- GENERATOR_USER_PROMPT: User prompt for the generator agent.
The Dockerfile sets up the Python environment with the necessary dependencies and copies the project files into the container.
Lists the Python dependencies required for the project, including the anthropic package for interacting with the Anthropic API.