yatendra2001 / ai_buddy

Your personal free-to-use AI assistant, built with gemini & flutter.

AI Buddy

Your personal completely open-sourced and free-to-use AI assistant, built with gemini & flutter.

Features

AI Buddy is a cross-platform mobile application that leverages the power of AI to provide users with a personal assistant capable of understanding and responding to various queries.

Chat with PDF	Ask Image	Text Chatbot

Architecture

The architecture of the AI Buddy app is designed to efficiently process and interact with PDF documents. Here's a breakdown of the workflow:

Overview

The "Chat with PDF" feature of the AI Buddy app allows users to interact with the content of a PDF document in a conversational manner. The app extracts information from the PDF, processes it, and provides relevant responses to user queries.

Workflow Diagram

Workflow Steps

1. PDF Extraction: The PDF document is processed to extract its pages.

2. Chunking: The extracted pages are then split into manageable chunks for processing.

3. Batching: Chunks are grouped into batches (e.g., 1 batch = 100 chunks) for parallel processing.

4. Embedding Generation: Each batch is sent to an Embeddings API with the task type set to 'Document'. The API generates a list of vector embeddings for each chunk.

5. Local Storage: The generated embeddings for each batch are split into individual chunk embeddings. These embeddings are stored locally using Hive (a key-value database).

6. User Interaction: When a user inputs an instruction (e.g., "Give me an intro to the book"), the app sends the query to the Embeddings API with the task type set to 'Query' to generate an embedding for the query.

7. Semantic Search: The app performs a semantic search by taking the embedding of the user's query and comparing it across the document embeddings stored in Hive.

8. Ranking Results: The chunks are ranked based on the Euclidean distance between the query embedding and document embeddings. The top-ranked chunks are used as the context for the language model (LLM).

9. Response Generation: The language model generates an answer based on the context provided by the top-ranked chunks.

Technologies Used

• Flutter: For cross-platform mobile application development.
• Hive: For local storage of chunk embeddings.
• Gemini Embeddings API: For generating vector embeddings of text.
• Gemini (LLM): For generating responses based on context.
• Riverpod: For managing states across the app.

Getting Started

Prerequisites

Need the latest flutter version & a gemini api key from here

  flutter upgrade

Installation

1. Clone the repository from GitHub:

git clone https://github.com/yatendra2001/ai_buddy.git

2. Get all dependencies:

flutter pub get

3. Run the app:

flutter run

Contributing

Contributions are what makes the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

Contribution to the project can be made if you have some improvements for the project or if you find some bugs. You can contribute to the project by reporting issues, forking it, modifying the code and making a pull request to the repository.

Please make sure you specify the commit type when opening pull requests:

feat: The new feature you're proposing

fix: A bug fix in the project

style: Feature and updates related to UI improvements and styling

test: Everything related to testing

docs: Everything related to documentation

refactor: Regular code refactoring and maintenance

To know more extensively about how to contribute to this project, read our Contribution Guide.

License

Distributed under the AGPL-3.0 license. See LICENSE for more information.

Contact

Twitter: @iamyatendrak

LinkedIn: linkedin

Make sure to leave us a ⭐️

ai_buddy's Issues

[Feat]: Enable Chat Interaction with YouTube Videos

Is your feature request related to a problem?

Users would benefit from being able to input YouTube video links for chatting.

Describe the solution that you'd like

Implement a feature where users can input YouTube video links. The app will extract the video transcripts and allow users to query and interact with this content through the chat interface.

Describe alternatives that you have considered

An alternative is to allow users to input video transcripts manually but not preferred due to usability concerns.

Additional Information

N/A

Acceptance Criteria

• Users can submit YouTube video links for processing.
• The app extracts transcripts from these videos.
• Users can interact with the extracted content through the chat interface, similar to how they would with PDF inputs.

[Feat]: Enhance Chat Functionality with Link Support

Is your feature request related to a problem?

Currently, the app supports chatting based on text and PDF inputs. Users have expressed a desire to include links to articles or blog posts as input for chatting.

Describe the solution that you'd like

Implement a new feature where users can input a link to an article or blog post. This feature would scrape the data from the provided link, process it (chunking + embedding), and then allow users to query this data, similar to the existing chat with PDF feature.

Describe alternatives that you have considered

An alternative could be to allow users to manually input the text from the article or blog post, but this would not be as user-friendly or efficient.

Additional Information

N/A

Acceptance Criteria

• Users can input a link to an article or blog post.
• The app successfully scrapes and processes the content of the link.
• Users can interact with the processed content through the chat interface.

[Bug]: App Freezes on Unsafe Language Input

What happened?

When a user inputs text in a language that the model deems unsafe, the model returns an error. Instead of displaying this error to the user, the app currently freezes, leaving the user waiting indefinitely for a response.

Steps to reproduce

1. Open the app and use any feature, for this example let's say chat with ai.
2. Input text in a language considered unsafe by the model. Like 
2. Submit the text for processing.
3. Observe that the app does not proceed or display an error message, effectively freezing.

Supporting info to reproduce

Check image:

Relevant log output

No response

[Feat]: Develop a Web Version of the Mobile App

Is your feature request related to a problem?

To expand the accessibility of the app, there's a need to create a web version using Flutter Web.

Describe the solution that you'd like

Create a web version of the AI Buddy app with a UI similar to the app, utilising Flutter Web. The project structure should be organized into a monorepo to maintain separate apps (mobile, web) under the same repository.

Describe alternatives that you have considered

N/A

Additional Information

N/A

Acceptance Criteria

• A web version of the AI Buddy app is developed using Flutter Web.
• The new UI is user-friendly and designed specifically for web.
• The project is structured as a monorepo with separate apps for mobile and web. I'll help you with this.
• The web app is fully functional and offers the same features as the mobile version.

[Feat]: Integrate Chat Functionality with Podcasts

Is your feature request related to a problem?

Users have requested the ability to input podcast links (including Spotify, Apple, or YouTube) for chatting.

Describe the solution that you'd like

Develop a feature that allows users to submit podcast links. The app will extract transcripts from these links and process them for chatting, enabling users to interact with podcast content through the chat interface.

Describe alternatives that you have considered

An alternative could be to require users to provide transcripts manually, but this would significantly reduce usability and accessibility.

Additional Information

N/A

Acceptance Criteria

• Users can submit links to podcasts from major platforms (Spotify, Apple, YouTube).
• The app extracts transcripts from these podcasts.
• Users can chat with the app based on the content of the podcasts.
• The feature maintains the user-friendly interface of the existing chat functionalities.