Eyes on the patient, not the keyboard β the visit documents itself.
CoDoc is an ambient clinical copilot. It listens to the doctorβpatient conversation, understands it in real time, and turns it into a real, standards-compliant medical record β with zero typing.
Built at the YC Γ Medplum Agentic Healthcare Hackathon (Aug 2026) on Medplum, Deepgram, and Claude.
- Listens β Deepgram Nova-3 streaming with speaker diarization and medical keyterm boosting
- Understands β Claude infers who the clinician is, extracts symptoms (with the patient's verbatim words), vitals, allergies, meds, and intake details (DOB, phone, address, pharmacy)
- Registers patients by voice β say your name: returning patients get their full history pulled up; new patients get a chart created on the spot
- Catches what humans miss β cross-checks the live conversation against the stored record (e.g. flags an amoxicillin order against a documented penicillin allergy)
- Acts on command β "CoDoc, refer her to orthopedics" β drafts the order note and queues a real FHIR
ServiceRequest+Taskfor the specialist; prescriptions becomeMedicationRequests - Closes the visit β one click signs a SOAP note into the chart as a
DocumentReference - Three-sided product β PCP runs visits, the specialist works a referral queue (accept β patient appears in their panel), and the patient sees their own record, conversations, and referral status in plain language
Everything is written to Medplum as conformant FHIR (11+ resource types), live, as the conversation happens.
| Layer | Tech |
|---|---|
| Speech β text | Deepgram Nova-3 (streaming WebSocket, diarize=true) |
| Understanding | Claude Haiku 4.5 (Messages API, forced tool call) |
| Record of truth | Medplum (hosted FHIR R4) |
| App | Next.js 16 / React 19, framer-motion, react-body-highlighter |
| Login accounts | SQLite via node:sqlite (demo auth) |
Prereqs: Node 22.5+ (uses built-in node:sqlite), npm, and a microphone.
git clone https://github.com/Nehanth/CoDoc.git
cd CoDoc
npm installCreate .env.local in the project root:
# Deepgram β https://console.deepgram.com (free credits)
DEEPGRAM_API_KEY=your_deepgram_key
# Anthropic β https://console.anthropic.com
ANTHROPIC_API_KEY=your_anthropic_key
# Medplum β https://app.medplum.com β Project β Clients β create client
MEDPLUM_BASE_URL=https://api.medplum.com/
MEDPLUM_CLIENT_ID=your_medplum_client_id
MEDPLUM_CLIENT_SECRET=your_medplum_client_secretThen:
npm run devOpen http://localhost:3000 (must be localhost or HTTPS β the mic requires a secure context).
| Username | Password | Role |
|---|---|---|
pcp |
pcp |
Primary care β runs live visits, sends referrals & prescriptions |
specialist |
specialist |
Specialist β referral queue; accepting a referral adds the patient to their panel |
any patient's name (e.g. Emily Carter) |
anything on first login | Patient β their own record, conversations, and referral status |
Patient accounts self-register on first login for any patient that exists in your Medplum project; the password you type first becomes theirs.
- Sign in as
pcp, hit Start session - Clinician: "Hi, can I get your name and date of birth?" β Patient: "I'm Emily Carter, March 14th, 1998β¦" (phone, address, pharmacy, penicillin allergy, meds)
- Patient: "I've had pain in my right knee for two months, since I started training for a 10K" β watch the chart bloom: symptoms with quotes, body map, vitals as spoken
- Clinician: "I'll start you on amoxicillin for that sinus thing" β π¨ contraindication card (penicillin allergy)
- Clinician: "CoDoc, refer her to orthopedics" β agent drafts the note, order lands in the queue
- End visit Β· sign note β SOAP note signs itself
- Log out β
specialistβ accept the referral (patient's phone is right there) β Emily appears in their Patients panel - Log out β
Emily Carterβ she sees her record, the whole conversation, and "Accepted β expect a call to schedule"
Check app.medplum.com afterwards β every resource is real FHIR.
- Login is demo-grade (plaintext SQLite, localStorage session) β hackathon scope, not production auth
- Pharmacy integration is a labeled dummy; referral receiving is intentionally out of scope (any FHIR-native system can work the Task queue)
codoc.db(accounts) is created on first run and gitignored