canner / wrenai

WrenAI makes your database RAG-ready. Implement Text-to-SQL more accuratly and securely.

CannerCMS has pivoted to CannerData - The Data Access Layer for Analytics Data Applications.

Canner now provides solution designs for data teams so that data owners, stewards, and consumers can easily collaborate in one interface. It is an ease-of-use, scalable, secure platform for querying and analyzing information across multiple databases, file systems, and other storage formats. Unlike current solutions that data is often manually integrated and transferred across different tools and sources, resulting in time-consuming and error-prone processes.

wrenai's Issues

Automatically create docker image when PR merged

Each time PR is merged, it will trigger the CI to build an image with a nightly tag. and it has a restriction that the PR must include module/ai-service. the result will be like

Support the Web APIs for Langfuse Integration

will put the updated API specs here soon.

errors when running `yarn build`

Description

Following errors thrown when I try to run yarn build

Frontend issues

wren-ui/src/pages/component.tsx

Type '{ joinType: JOIN_TYPE.ONE_TO_ONE; fromField: { model: string; field: string; }; toField: { model: string; field: string; }; name: string; properties: { description: string; }; }' is not assignable to type 'RelationFieldValue'.
  Property 'type' is missing in type '{ joinType: JOIN_TYPE.ONE_TO_ONE; fromField: { model: string; field: string; }; toField: { model: string; field: string; }; name: string; properties: { description: string; }; }' but required in type 'Pick<RelationsDataType, "type" | "name" | "fromField" | "toField" | "properties">'.ts(2322)

also error at MetadataDrawer of metrics

Object literal may only specify known properties, and 'name' does not exist in type 'Metadata'.ts(2353)

wren-ui/src/components/tableFormControls/*.tsx all -Controls component

Module '"@/components/table/CalculatedFieldTable"' has no exported member 'getCalculatedFieldTableColumns'.

Backend issues

wren-ui/src/apollo/server/mdl/mdlBuilder.ts

'Relation' is declared but its value is never read.ts(6133)

src/apollo/server/resolvers/projectResolver.ts

./src/apollo/server/resolvers/projectResolver.ts:114:8
Type error: This expression is not callable.
  Each member of the union type '{ <S extends CompactTable>(predicate: (value: CompactTable, index: number, array: CompactTable[]) => value is S, thisArg?: any): S[]; (predicate: (value: CompactTable, index: number, array: CompactTable[]) => unknown, thisArg?: any): CompactTable[]; } | { ...; }' has signatures, but none of those signatures are compatible with each other.

  112 |     const id = project.id;
  113 |     const tableDescriptions = dataSourceColumns
> 114 |       .filter((col: BQColumnResponse) => col.table_description)
      |        ^
  115 |       .reduce((acc, column: BQColumnResponse) => {
  116 |         acc[column.table_name] = column.table_description;
  117 |         return acc;

Embed the telemetry point in the pipelines

• https://langfuse.com/docs/sdk/overview

Test semantics descriptions and refine it if necessary

We need to test whether semantics descriptions(/v1/semantics-descriptions/) would improve the accuracy of the ask pipeline(text-to-sql). If not, we need to improve the semantics descriptions api. Text-to-SQL benchmark chart also needs this.

Support Mistral Provider

We're going to support Mistral. More details about the implementation will be updated soon. 😝

Refine ask pipeline - version 1

• refine ask pipeline based on diversity, cost and latency
• can filter out logically duplicate results to prevent confusion
• executable sql queries
• fix llm treating relationship as column
• improve follow-up performance by designing another prompt
• improve wren-engine sql execution efficiency and reduce cost

docker build optimization

currently we use the naive approach to build the docker image, in the future we should optimize docker build process in order to reduce docker image size and reduce docker build latency.

references

• Blazing fast Python Docker builds with Poetry

Creating a workflow for Image build and deployment

hey there! I believe it's essential to establish a workflow for building images that can be deployed anywhere. Detailed information will be added to this issue later. For now, let's create this as a reminder to tackle it soon. 😸

Enable Release Workflow Commits on Protected Main Branch

Following issue #59, we've run into a problem where commits cannot be pushed to the main branch due to the enabled branch protection, preventing any direct pushes to main. A potential solution could involve creating a GitHub app specifically for the release workflow to update the version in pyproject.toml. This app would have permissions to bypass the branch protection, allowing commits to be pushed as part of the release process.

Langfuse integration

Logging for easier troubleshooting...

reference: https://github.com/Delgan/loguru

Refine ask details pipeline - version 1

• rewrite prompt using few-shot learning

Support Anthropic Provider

We're going to support Anthropic. More details about the implementation will be updated soon. 😎

ask_details pipeline evaluation process

We need to come up with some rules to evaluate the ask_details pipeline. Something we can think about:

1. What is a good ask_details pipeline from user's perspective?
   • Summery text in each step is easily understandable?
   • SQL query in each step is executable?
   • SQL query should semantically correspond to respective summary text?
2. Is there any public benchmark we can reference with?

related issue: #4

Support Amazon Bedrock Provider

We're going to support Amazon Bedrock. More details about the implementation will be updated soon. 😉

Refine evaluation process

In the current evaluation process, there are some issues for further refinement

• reproducibility: there is no dvc.lock related file to make sure the pipelines are reproducible in evaluation.(we can directly reference this to spider-benchmark repo; however, if it's not that important as of now, we can skip this right now.)
• evaluation report:
  • adding cost/latency
  • adding pipeline metadata so that we can easily understand what parameters cause the differences among different reports

Support Web API to get the current LLM provider

Text to SQL(ask) pipeline performance benchmark

• What are performance objectives we are optimizing for?
• What are we comparing with?

related issue: #4

Automatically re-run the CI after pushing the new commits to PR

Currently, we trigger the CI in PR via labeling the tag ci/ai-service. however, we need to unlabel and re-label it to trigger the CI again if we push any new commits. so, I think when pushing commits and checking the label exists, it will be triggered automatically. It will be awesome. 👍

Filter out duplicate results

Can filter out logically duplicate results to prevent confusion

Handling text generation edge cases

• generated tokens reach token window limitation according to respective llm model specs.
• (nice to have) add moderation layer to make our app more safe. reference: https://platform.openai.com/docs/guides/text-generation/how-can-i-make-my-application-more-safe

Embed the Langfuse key in the Docker image

• https://medium.com/@brian978_dev/effortlessly-secure-passing-secrets-from-github-to-your-docker-image-f1df3b6d0e49
• https://docs.docker.com/compose/use-secrets/