A Complete Exploration of iOS 26 On-Device Foundation Models, Tool Calling, Generable, Transcript & Real-World Data Integration
This repository contains a collection of real, production-grade demo implementations showcasing Appleβs iOS 26 Foundation Models (on-device LLM). It is designed as a comprehensive playground for experimenting with:
- On-device language models
- Generable (structured generation)
- Tool (agent-style function calling)
- Transcript (AI workflow timeline)
- HealthKit & EventKit real-world data integration
- Fax / Scan / PDF intelligent automation
- Dynamic schemas & configurable workflows
This project demonstrates how to build real products with Appleβs on-device AIβnot just chatbots.
Demonstrates Apple's on-device LLM capabilities introduced in iOS 26:
- Low latency (<100ms local inference)
- Fully private (no data leaves the device)
- Native integration with app-level workflows
- Built-in guardrails for safety
Includes demos for:
- Prompting
- Instruction design
- Safety boundaries
- Streaming generation
- Chat session orchestration
Generable lets developers build strong schema-driven models:
- Define Swift structs that the model must fill
- Replace βLLM hallucinated JSONβ with strict, type-safe results
- Perfect for UI forms, workflows, automation logic
Included demos:
GenerableWorkflow DemoDynamicGenerationSchemaDemoViewDynamicFaxPlanSelection DemoDynamicOCRLanguageSelection DemoDynamicPDFCompression DemoDynamicContactSelection DemoDynamicScanPreset DemoDynamicWorkflowSchema Demo
This project demonstrates how to turn Swift services into callable LLM tools.
Examples:
SearchContactToolSendFaxToolAddCoverPageToolSearchFaxToolSaveFaxDraftToolCropFaxTool
-
CorrelateWithRealWorldDataToolβ Calls real HealthKit + EventKit- Reads sleep data
- Reads heart rate
- Reads calendar events
- Correlates real-world stressors with user dream emotions
All tool calls are automatically shown in the Transcript UI.
Transcript is one of the most important parts of iOS 26βs Foundation Models.
We provide two production-ready implementations:
ChatGPT-style UI with:
- User bubbles
- AI response bubbles
- Tool call bubbles (with tool badges)
- Tool output bubbles
- Streaming responses
- Auto-scroll
- Message grouping
Developer-facing console to debug:
- Tool sequence
- Argument correctness
- Output inspection
- Model decision steps
Transcript entries include:
.instructions.prompt.toolCalls.toolOutput.response
We include a real production-ready version of HealthKit + EventKit integration:
- Requests authorization
- Reads sleep analysis
- Reads heart rate
- Computes pressure nights
- Fully async/await
- Reads real calendar events for βlast weekβ
- Extracts meaningful stress-related events
Used in the dream-context emotional analysis Demo.
A complete end-to-end example:
-
User asks: βWhy were my dreams so anxious last week?β
-
AI is required (via Instructions) to call:
correlateWithRealWorldData -
Tool fetches HealthKit + Calendar real data
-
Returns structured
RealWorldContext -
Model generates empathetic wellness insights based on actual user life events
-
Perfect example of combining:
- LLM reasoning
- Real Sensor Data
- Structured output
- Tool chaining
UI file:
AIHubEngine acts as the central orchestrator:
- Manages session lifecycle
- Registers all tools
- Handles streaming
- Publishes Transcript entries
- Exposes UI-friendly state
- Builds multi-module AI workflows (Fax, Contacts, Dream, PDF, Scan)
This is the βbrainβ of the entire system.
Root
βββ AIHub/
β βββ AIHubEngine.swift
β βββ Tool/
β βββ Services/
β βββ TranscriptUI/
β βββ ChatUI/
β
βββ Demos/
β βββ FoundationModel Demo
β βββ Prompt Demo
β βββ Safety Demo
β βββ ToolFaxScan Demo
β βββ WorkflowToolTranscript Demo
β βββ Dynamic Workflow Demos (Generable)
β βββ DreamContextAnalyzerView.swift
β
βββ HealthKit/
β βββ HealthKitManager.swift
β βββ CalendarManager.swift
β βββ RealWorldContext.swift
β
βββ FaxServices/
β βββ FaxService.swift
β βββ ContactsService.swift
β βββ CoverPageService.swift
β βββ Tools/*.swift
β
βββ README.md β (this file)
-
Xcode 16+
-
iOS 26+ device (HealthKit not supported in Simulator)
-
Enable in Signing & Capabilities:
- HealthKit
- iCloud (optional)
- Contacts (if you plug deeper)
- Calendars
Ensure the following keys exist in Info.plist:
<key>NSHealthShareUsageDescription</key>
<string>App uses your health data to build local AI insights.</string>
<key>NSHealthUpdateUsageDescription</key>
<string>App uses health data to improve dream & wellness insights.</string>
<key>NSCalendarsUsageDescription</key>
<string>App reads calendar events to correlate real-world stress factors.</string>-
Install to a real iPhone (Simulator won't support HealthKit)
-
Launch the app
-
Navigate through the Demos from the main menu
-
Try:
- Generating structured output
- Calling tools
- Observing Transcript
- Running Dream Context Analyzer
- Fax AI workflow demo
User β βHelp me send this fax to my accountantβ
Model β Plans workflow:
1) searchContact
2) addCoverPage
3) sendFax
4) saveFaxDraft (fallback)
DynamicFaxPlanSelection
β Changes fields based on country/user context
β Schema passed to model
β Model outputs strict Generable struct
Tool: correlateWithRealWorldData(period: "last_week")
β Reads sleep analysis
β Reads heart rate
β Reads calendar events
β Returns RealWorldContext summary
β Model produces emotional explanation
- PDF semantic segmentation (on-device)
- On-device OCR auto-language selection
- Tool router for multi-domain AI orchestration
- Local RAG (device-only embeddings)
- AI-powered scanning skills (cropping, enhancement)
- Fax analytics + workflow optimization
- Jet AI Hub: unified AI control center for all Jet apps
This project is for research, demonstration, and education. Commercial usage should ensure compliance with Apple policies, HIPAA/PII rules, and local regulations.