apssouza22 / chatflow

Leveraging LLM to build Conversational UIs

Hey 👋, This is Alexsandro Souza

I am passionate about software development and agile methods. I love solving team and company problems from a tactical and strategic point of view.

I help teams and company to achieve more. Improving code, processes, flows, architecture, communication and human resources.

Beyond my day-to-day work, I am giving back to positively impact the community with articles, videos, lectures and free courses.

Some of my Github Stats

My most interesting projects

Architecture and Desing

• Service Mesh
• Big data Pipeline
• Microservices with Java
• Security Architecture for Distributed Systems
• gRPC for production
• Trading system with Java
• Video chat using Webrtc
• Modern API management using Protobuf

AI/DL/ML

• Smart drone
• Conversational UI using AI Agents
• Neural Network with Javascript from scratch
• Neuroevolution - ANN and Genetic Algo
• CNN for developers
• Computer vision with OpenCV
• Reiforcement Learning wiht vanilla Js

chatflow's Issues

No input after answering all questions

After answering the questions, there should be an input field for user to try to communicate on his/her topic. But there are no such input visible. It may cost losing us many users:

Create individual chat history for each user

Move the code to github enterprise

No proper namespace for Redis keys

The following demonstrates that data_vector:9051 is available in the top-level namespace of Redis.

> KEYS *
...
data_vector:9051
...
> HGETALL data_vector:9051
...

That wrong, because on a real server Redis may be simultaneously used by other software and that key data_vector: is without a unique prefix.

After being tricked by the logic of existing software, I also created a subroutine that creates keys like:

predict_cache:Show 10 of my GitHub repos.

that is without a unique prefix, too.

It was created by:

async def create_predict_cache():
    value_field = TextField("value")

    # Create index
    await redis_conn.ft(INDEX_NAME + "_predict_cache").create_index(
        fields=[value_field],
        definition=IndexDefinition(prefix=["predict_cache:"], index_type=IndexType.HASH)
    )

Here I was tricked by my intuitive understanding of API that .ft(...) provides a namespace INDEX_NAME + "_predict_cache" but this intuition contradicts to the fact that

HGETALL "predict_cache:Show 10 of my GitHub repos."

retrieves data without a unique prefix having been specified.

The next step is to understand what is the argument of .ft(), does it provide any useful value (because it has no expected value of providing a prefix), and then what to replace it by.

add user and app information to the UI

Add to the topbar user and active application information. Currently we can not really tell what app is currently active in the session.

Here where the active app is set

Use llmlingua for re ranking

Llmlingua can help with multiple docs rerank and reduce tokens

https://github.com/microsoft/LLMLingua

Implement Upload Files From The Chat

Add the option to upload image using natural language. The user should just write "upload file" and the file upload option would appear to him

Store Apps ina DB

Currently the apps are stored using file, we should change it to store in the postgres db.

To-Do: Improve app authentication/authorization

Change The Doc Response Structure

Edit Docs Endpoint

Create a endpoint to edit documentation. Today we have only create and delete endpoints
See https://github.com/apssouza22/chatflow/blob/main/server/src/api/docs.py

test

UI, Improve Open Chat Button

Pull request about unclear code

Do something about this pull request: #42

Replace the current predict cache file storage with Redis

Currently the prediciton cache is storage in memory we should use Redis instead

Move all system prompts to the application UI

Sometimes "Login failed" with `undefined` instead of `/api/v1`, despite the frontend was running

Sometimes we have "Login failed" with undefined instead of /api/v1.

Restarting the frontend solves this problem, but sometimes the problem appears despite the frontend has been already restarted in the past.

We need further investigation how to reproduce this bug.

Store Message History in Redis instead of file

Currently we store the message history in the file system. Would be good to store in the redis itself.

Create QA environment

Enable images and videos to be shown as a lightbox when clicked.

Improve Design, remove chat icon, remove chat bubble, use chats full width

Simplify onboarding

The software asks a long series of questions for new users.

Many users would leave it in the middle. We need to make users able not to answer all these questions.

redis.exceptions.ResponseError: chat_docs: no such index

In the first run of the software as described in README.md:

INFO:     127.0.0.1:43206 - "POST /api/v1/docs/search HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/porton/.local/lib/python3.9/site-packages/uvicorn/protocols/http/h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/porton/.local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/home/porton/.local/lib/python3.9/site-packages/fastapi/applications.py", line 284, in __call__
    await super().__call__(scope, receive, send)
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 91, in __call__
    await self.simple_response(scope, receive, send, request_headers=headers)
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 146, in simple_response
    await self.app(scope, receive, send)
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/home/porton/.local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/home/porton/.local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/home/porton/.local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/home/porton/.local/lib/python3.9/site-packages/fastapi/routing.py", line 241, in app
    raw_response = await run_endpoint_function(
  File "/home/porton/.local/lib/python3.9/site-packages/fastapi/routing.py", line 167, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/porton/Projects/chatflow/server/src/api/docs.py", line 31, in find_docs
    return await doc_search_service.full_search(search_req)
  File "/home/porton/Projects/chatflow/server/src/core/docs_search/doc_service.py", line 34, in full_search
    results_vector = await self.vector_search.search_vectors(search_req.tags, vector_bytes)
  File "/home/porton/Projects/chatflow/server/src/core/docs_search/vector_search.py", line 22, in search_vectors
    return await self.conn.ft(self.index).search(query, query_params={"vec_param": vector})
  File "/home/porton/.local/lib/python3.9/site-packages/redis/commands/search/commands.py", line 889, in search
    res = await self.execute_command(SEARCH_CMD, *args)
  File "/home/porton/.local/lib/python3.9/site-packages/redis/asyncio/client.py", line 518, in execute_command
    return await conn.retry.call_with_retry(
  File "/home/porton/.local/lib/python3.9/site-packages/redis/asyncio/retry.py", line 59, in call_with_retry
    return await do()
  File "/home/porton/.local/lib/python3.9/site-packages/redis/asyncio/client.py", line 492, in _send_command_parse_response
    return await self.parse_response(conn, command_name, **options)
  File "/home/porton/.local/lib/python3.9/site-packages/redis/asyncio/client.py", line 539, in parse_response
    response = await connection.read_response()
  File "/home/porton/.local/lib/python3.9/site-packages/redis/asyncio/connection.py", line 810, in read_response
    raise response from None
redis.exceptions.ResponseError: chat_docs: no such index

Amend docker-compose to run the app entirely

Now docker-compose.yml uses several ports that may glitch with host software (e.g. if host runs PostgreSQL and/or Redis, they will collide). So to run it I needed to shut down my PostgreSQL, what is at least inconvenient.

I tried to fix it by adding

networks:
  mynetwork:
    driver: bridge

at the end of docker-compose.yml and commenting out port forwarding. But to run the app, I needed to uncomment every commented out port mappings, because the server software and the frontend are running not in Docker.

I recommend, for cleanness, to add to docker-compose.yml means to run all the software entirely in Docker, and to make it the recommended way to run in README file.

Use SSE to stream the response

https://www.vidavolta.io/streaming-with-fastapi/

"Something went wrong"

When executing it on localhost as described by the README.md file, it among useful output produces errors:

Create Admin Users

CONTEXT instead of CONTENT

There, apparently should be f'CONTEXT\n----------------------------\n {context}' instead of f'CONTENT\n----------------------------\n {context}' in
server/src/core/llm/prompt_handler.py

Set CI pipeline with github actions

Build the docker imagine and run the service to later add integration tests

Keep the session on the browser and load the data on reload

Currently the hole history is lost everytime we reload the page. We need to keep the chat history on the browser using local storage

A typo in AI reasonings

In Given the contents below, please answer which content option it should be used to replay to the user input. there should be reply instead of replay.

server/src/core/llm/prompt_handler.py

Store User Data ina DB

Currently users are stored in the file system. We should persist users in the DB

Create integration test for the backend API

Create a simple integration test to test the rest endpoiints.

Basically we should create python tests to call our API and validate the responses.

We have few REST endpoints that are used by the UI
http://apps.newaisolutions.com/api/docs

Later we will run those as part of the pipeline