molly-carroll-tw / ig-poc-twai21

AI21 RAG PoC: Interview Genie

Ongoing testing: experiments using Contextual Answers RAG TSM + Jamba Instruct model

ig-poc-twai21's Issues

create public facing content, blog, webinar, etc.

Prioritize 1-2 UI elements to show data based on criteria

This will depend on chosen evaluations in pipeline

slide building

update and finish PoC deck for meeting with Z

Presentation + deep dive discussion will involve these topics: the business problem, business value of concept, fitness of the solution, implementation details and demo of the results, conclusions, limitations, next steps
No need for code details -- focusing on functionality of solution using AI21 AI technology
Deck link - WIP

Revise evaluation pipeline to output more formatted CSV + use labeled speakers transcript

Decide on MVP-level ranking of 1-2 traits based on transcript. Criteria for prod dependent on HR communication

Verify ability to use Jamba Instruct or Jamba for implementation especially chat

Find out how to use with Python SDK or if not able, what to do otherwise

Code Streamlit UI setup to utilize app data

Create slide deck and prep content for RAGCamp session on AI21 PoC

prompt engineering to improve results

improve evaluation results and citation with Jamba model by changing the prompt template

Build query pipeline passing uploaded documents into prompt context

Use Python SDK to include document matches in prompt context when the "evaluate" function is triggered to use in comparison with role-specific rubric to determine if candidate fulfills rubric line item or not.

When "evaluate" is triggered (in the future, by user via UI button after uploading their documents) the appropriate rubric criteria for the role acts as a query used by Jamba LLM against context documents

For PoC, context documents are: candidate resume, interview transcript

Write and test Python script to ingest documents to RAG engine vector store

Revise RAG pipeline to calculate & parse criteria requested by TW HR