Context

• In #5 we have converted the CPT guideline criteria into a JSON Decision Tree in which the leaves of the tree correspond to single criteria such as "Patient is over 45 years old".
• We can now loop through the decision tree, and query our RAG pipeline for each criterion to determine if it is true.
• We can then use the And/Or logic held within the Decision Tree to determine whether the criteria as a whole are met.

Acceptance Criteria

• Write a function that takes a single criterion from the CPT guideline decision tree and queries the RAG pipeline to determine whether it's true or not.
• Obtain and store supporting evidence for the conclusion for each criterion.
• Write a function that loops through the Decision Tree and combines the results from the individual criteria to determine if the criteria overall are met.
• For now, just print the output to the console.

Context

• Having set up a very basic RAG query pipeline in #2 , we now want to parse the CPT code of the requested treatment from the document.

Acceptance Criteria

• Parse the CPT code of the requested treatment from the medical record.

Technical Details

• This could either be performed by querying with an LLM or with a Regex.

Context

• As per the task instructions, the first step of the pipeline is to determine whether a conservative treatment has been attempted and if so, whether it was successful in treating the symptoms.

Acceptance Criteria

• Implement function(s) that query the RAG pipeline to determine whether a conservative treatment was attempted and successful.
• For each question, return a boolean answer (yes/no) and evidence.
• The evidence should link back to a part of the original report.
• For now, just print the output from this step of the pipeline to the console.

Context

• Having built an API in #7 to submit requests to the Pre Auth pipeline, the next natural extension would be to build a User Interface (web app) where a user can submit requests to the pipeline and view the results.

Acceptance Criteria

• Build a simple web page using a HTML templating engine with Fast API.
• User can submit a new request to the pipeline.
• User can view the report generated by the pipeline.

Context

• If I have time to do #7 and #8 , it'd be nice to deploy the web app so people can interact with it for testing.

Acceptance Criteria

• Create AWS Elastic Beanstalk environment
• Upload code
• [Optional] Configure Circle CI to automatically deploy on PR merge.

Context

• Let's lay the foundations for the pipeline by installing and setting up LlamaIndex, a Python package we'll use to build our RAG & query pipeline.

Acceptance Criteria

• Install LlamaIndex
• Setup LlamaIndex (API Keys for Open AI etc.)
• Implement the most basic ingestion, indexing and query pipeline demonstrated in Starter Tutorial.
• Sanity check that asking basic questions about the medical records gives accurate answers.

Context

• Let's perform some basic setup for the project / repo

Acceptance Criteria

• Install and setup Poetry for package management.
• Install and setup flake8 for linting.
• Install and setup pytest with 1 dummy test for now.
• Configure Github Action to run tests on push.

Context

• As a natural extension we can build a REST API to submit new Prior Auth requests and then fetch the results.
• As part of this we can store the requests and results in a mock database (writing to disk).

Acceptance Criteria

• Implement POST pre-authorization-report endpoint where a PDF can be submitted and which starts the pipeline.
• Implement a GET pre-authorization-report endpoint to fetch the report(s) from the DB.

Context / Explanation

• When it comes time to assess whether the criteria listed in the CPT guidelines are met we will need to break the guidelines down into simple Yes/No questions (e.g. "Is the patient over 45 years old") before querying our RAG pipeline.
• We will also need to know how to logically combine the Yes/No answers from each question into an overall answer using the And/Or logic that is inherent in the guidelines.
• We could do this manually for the purpose of this task but it would be better for the pipeline to automatically handle this operation so that it can be applied to other CPT codes and guidelines.
• To do this, we will use an LLM to convert the CPT guidelines into a JSON decision Tree which encodes the individual conditions and how they are related with And/Or logic.

Example:

Example Input (part of the guidelines provided):

- Colorectal cancer screening, as indicated by 1 or more of the following:
   - Patient has average-risk or higher, as indicated by ALL of the following:
      - Age 45 years or older
      - Symptomatic (eg, abdominal pain, iron deficiency anemia, rectal bleeding)
   - X
   - Y

Example JSON Decision Tree Output:

{
 "logic": "OR",
 "conditions": [
   {
      "condition": "Patient has average-risk or higher",
      "logic": "AND",
      "conditions": [
          {
            "condition": "Age 45 years or older",
          },
          {
           "condition": "Symptomatic (eg, abdominal pain, iron deficiency anemia, rectal bleeding)"
          }
       ]
   },
   {...},
   {...}
] 
}

Acceptance Criteria

• Write a function that can parse CPT guidelines PDF whilst maintaining nested bullet point structure/text formatting.
• Write a prompt (perhaps with a few examples) for converting the guidelines into the desired JSON format.
• Test that the prompt works reliably at least on the Colonoscopy guidelines provided.
• Write a Pydantic model for the decision tree - will make it easier to work with in python.

Context

• In previous tickets we've now implemented each step of the pipeline as an individual function.
• Now we need to combine them into a single pipeline function.

Acceptance Criteria

• Write a function:

pre_authorization_pipeline(medical_record_file_path: str | Path) -> PreAuthorizationPipelineResult

Where PreAuthorizationPipelineResult is:

class PreAuthorizationPipelineResult(BaseModel):
    are_criteria_met: bool.    # Overall decision
    cpt_code: str.  # CPT code parsed from PDF
    criteria: str.  # The formatted criteria
    criteria_results: list[CriterionResult]. # The result for each criterion

class CriterionResult(BaseModel):
    criterion_id: str
    criterion: str
    criterion_question: str | None = None
    is_criterion_met: bool | None = None
    reason: str
    evidence: str | None = None
    information_required: str | None = None

• Raise a CPTGuidelinesNotAvailable exception if the report requests a procedure for which we don't have the CPT guidelines available.
• For now, the main.py file should just allow you to specify a medical record, call the pipeline and print the result.

Context

• Need to write a README to explain the approach and how to run it

Acceptance Criteria

• Explain how to run the project with Docker
• Explain the solution at a very high-level