A FastAPI + Streamlit financial RAG application over company filings.

The backend is instrumented with Langfuse Cloud. Each financial request creates a trace with:

• metadata-extraction span
• filtered-retrieval span
• generation generation
• ragas-faithfulness evaluation span

Company and quarter are attached to the trace as metadata and tags so they can be filtered in Langfuse. Langfuse supports filtering observations by metadata and filtering traces/observations by tags. See the official docs: https://langfuse.com/docs/observability/features/metadata and https://langfuse.com/docs/observability/features/tags.

The application reads these values from environment variables and does not store credentials in source control:

LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_BASE_URL=https://cloud.langfuse.com

Copy .env.example to .env and put your Langfuse project credentials there.

For answered questions, the backend evaluates the generated answer against the retrieved contexts using the RAGAS Faithfulness metric and sends the numeric score to the active Langfuse trace under the score name faithfulness.

RAGAS needs an evaluator LLM. The default implementation uses OpenAI:

OPENAI_API_KEY=...
RAGAS_EVALUATION_MODEL=gpt-4o-mini

If the evaluator key is not configured or the evaluation fails, the financial answer still succeeds and the API reports the evaluation as unavailable/error. A RAGAS evaluation failure never turns a valid financial answer into a 500 response.

Langfuse's documentation confirms that Ragas metrics can be used as evaluator functions and that scores attach to Langfuse traces. https://langfuse.com/integrations/frameworks/ragas

cp .env.example .env
# edit .env with your Langfuse credentials and OPENAI_API_KEY

docker compose up --build

Open http://localhost:8501.

The frontend intentionally has one financial query box. If company or quarter is missing, the app asks the user to add the missing information to the same box.

POST /v1/financial-query

{
  "query": "What was Apple's revenue in Q3 2025?"
}

The response includes the answer, citation, and—when RAGAS evaluation succeeds—the faithfulness evaluation result.

GET /health reports whether Langfuse credentials are configured.

This project uses the Langfuse Python SDK v4 and follows the official Langfuse Agent Skill instrumentation guidance.

For each /v1/financial-query request, the backend creates one financial-query trace with nested observations for metadata extraction, metadata-filtered Qdrant retrieval, deterministic answer generation, and RAGAS faithfulness evaluation. Company and quarter are propagated as trace metadata and tags. The RAGAS judge uses langfuse.openai.AsyncOpenAI, so its judge-model calls are automatically captured as nested Langfuse generation observations with model/usage/latency information.

The RAGAS result is attached to the root trace as a numeric Langfuse score named faithfulness. Telemetry and evaluation failures are non-fatal: they never turn an otherwise valid financial answer into an API failure.

Set LANGFUSE_SECRET_KEY, LANGFUSE_PUBLIC_KEY, LANGFUSE_BASE_URL, and OPENAI_API_KEY in .env. On startup, /health reports whether Langfuse authentication succeeded. The backend flushes at request boundaries for prompt visibility during development and calls shutdown() during FastAPI shutdown to drain buffered telemetry.

Financial questions no longer have to contain an explicit Q1-Q4 token. The API resolves calendar time expressions before applying Qdrant metadata filters:

• February 2025 -> Q1 2025
• January, February and March 2025 -> Q1 2025
• March and April 2025 -> Q1 2025, Q2 2025
• Q1 and Q3 2025 -> Q1 2025, Q3 2025
• 2025-08-15 -> Q3 2025
• bare 2025 -> Q1 2025, Q2 2025, Q3 2025, Q4 2025

For a full calendar year or any request involving several quarters, the retriever searches each quarterly metadata subset separately and merges the results. For the currently supported additive metrics (revenue, net income and operating income), the answer layer sums quarterly figures only when evidence is available for every requested quarter. If any quarter is missing, it returns a no_data response identifying the missing quarters instead of presenting a partial total as a full-year figure.

Date and month expressions that cross a quarter boundary return a figure for each involved quarter plus a total across those quarters. Explicit fiscal-year forms such as FY2025 retain their existing fiscal-year metadata semantics.

The aws/ directory deploys this exact multi-quarter-compatible application to AWS without changing the vector database architecture. The populated backend/qdrant_storage/ directory is copied into the backend image, so the Fargate backend opens the same local Qdrant collection used by the local Docker application.

• Docker running locally
• AWS CLI authenticated to the AWS account you will demo from
• One existing ECR repository (the scripts use separate backend-* and frontend-* tags in the same repository)
• Langfuse credentials
• OpenAI API key for the RAGAS faithfulness judge

You do not need to create ECS, a VPC, subnets, or an ALB manually. CloudFormation creates those resources.

aws sts get-caller-identity
export AWS_REGION=eu-west-2

Use the region containing your ECR repository. If the repository is named something other than financial-rag, set its existing name:

export ECR_REPOSITORY=YOUR_EXISTING_ECR_REPOSITORY

Copy the example and replace the placeholders locally. Never commit the populated file.

cp aws/secret.example.json /tmp/financial-rag-secret.json

The file must contain:

{
  "LANGFUSE_SECRET_KEY": "...",
  "LANGFUSE_PUBLIC_KEY": "...",
  "LANGFUSE_BASE_URL": "https://cloud.langfuse.com",
  "OPENAI_API_KEY": "...",
  "RAGAS_EVALUATION_MODEL": "gpt-4o-mini"
}

Create the AWS Secrets Manager secret:

aws secretsmanager create-secret \
  --region "$AWS_REGION" \
  --name financial-rag/demo \
  --secret-string file:///tmp/financial-rag-secret.json

If that name already exists, update it instead:

aws secretsmanager put-secret-value \
  --region "$AWS_REGION" \
  --secret-id financial-rag/demo \
  --secret-string file:///tmp/financial-rag-secret.json

Save the ARN:

export APP_SECRET_ARN=$(aws secretsmanager describe-secret \
  --region "$AWS_REGION" \
  --secret-id financial-rag/demo \
  --query ARN --output text)

From the project root:

./aws/build_and_push.sh

The script prints:

BACKEND_IMAGE_URI=...:backend-...
FRONTEND_IMAGE_URI=...:frontend-...

Export those exact values:

export BACKEND_IMAGE_URI='123456789012.dkr.ecr.eu-west-2.amazonaws.com/my-repo:backend-...'
export FRONTEND_IMAGE_URI='123456789012.dkr.ecr.eu-west-2.amazonaws.com/my-repo:frontend-...'

The backend image contains the existing backend/qdrant_storage directory. There is no Qdrant Cloud connection and no Qdrant API key.

./aws/deploy.sh

CloudFormation creates:

Internet
   |
Application Load Balancer :80
   |
ECS/Fargate task (exactly one)
   |-- Streamlit :8501
   `-- FastAPI :8000
          |
          `-- /app/qdrant_storage (baked into backend image)

The frontend and backend run in the same ECS task, so Streamlit calls FastAPI at http://127.0.0.1:8000. FastAPI itself is not exposed publicly.

At the end, deploy.sh prints the ALB URL.

aws ecs describe-services \
  --region "$AWS_REGION" \
  --cluster financial-rag \
  --services financial-rag \
  --query 'services[0].{running:runningCount,desired:desiredCount,pending:pendingCount,events:events[0:5]}'

aws logs tail /ecs/financial-rag \
  --region "$AWS_REGION" \
  --follow

A healthy backend reports "vector_store": "embedded-qdrant" from /health.

This configuration intentionally fixes DesiredCount at 1 and configures ECS not to run old and new tasks simultaneously during deployment. Do not enable horizontal autoscaling while using embedded qdrant_storage.

The database is part of the Docker image. If you ingest or re-index filings, rebuild the backend image and redeploy it. Runtime writes to the task-local Qdrant copy are not durable across task replacement. For this read-mostly demo, every task therefore starts from the known indexed snapshot.

You can leave your existing local demo running while building/pushing the AWS images. Building images and pushing them to ECR does not stop running local containers.