Changelog
The full timeline from first fine-tuning attempt to production deployment.
Phase 1: Fine-Tuning (ModelForge)
January 2026
ModelForge Created
Built an automated fine-tuning pipeline in Python. 10 modules, 38 unit tests, full CLI. Designed to parse dental policy PDFs, generate training data, and fine-tune local LLMs on Apple Silicon.
Python
ModelForge
January - February 2026
Runs 1 through 4
Four fine-tuning attempts with escalating strategies: default settings, expanded data, pipeline fixes, curated training pairs. All produced hallucinated content: fabricated emails, invented forms, confident but wrong answers.
Failed
February 2026
Run 5: RAFT
Tried Microsoft's RAFT (Retrieval-Augmented Fine-Tuning). Perfect smoke test scores with RAG active, but useless without it. Defeated the purpose of standalone fine-tuning.
Failed
February 2026
Run 6: Maximum Effort
2,198 training pairs, deeper LoRA, MLP targeting. Smoke tests passed. Real questions failed in five distinct ways. Conclusion: fine-tuning teaches style, not knowledge.
Failed
February 2026
Research Deep Dive
Reviewed Microsoft EMNLP 2024, Stanford FineTuneBench, Meta LIMA, Allen-Zhu ICLR 2025. All confirmed: fine-tuning cannot inject knowledge. RAG is the correct architecture. ModelForge parked permanently.
Research
Phase 2: The RAG Pivot
March 24, 2026
RAG Proxy Built in 18 Minutes
Core proxy, dental brain indexing, and web management interface built in a single 18-minute session with Claude. 550 lines of proxy code, 180 lines of brain indexing, 900 lines of web GUI. 20/20 curl tests passing. Deployed to the M3 Mac Studio.
Milestone
Perl
March 24, 2026
Production Deployment
Deployed to the UofT Faculty of Dentistry M3 Mac Studio. Dental brain with 94 policy documents indexed. Open WebUI configured to route through the proxy.
Production
March 29 - 30, 2026
Multi-Brain + Model Routing
Added model allow-list filtering, per-brain system prompts, and model-name routing (brain/model convention). Stood up additional clinical knowledge brains alongside the dental brain to prove the multi-domain pattern. Running on port 6969.
Feature
April 1, 2026
Brain Snapshot System
Built an export/import system for complete brain state, so a brain can be backed up, version-controlled, and restored on another machine. The dental brain alone holds 94 source documents indexed into searchable chunks.
Feature
April 5, 2026
USW Local 1998 Brain
Built a new brain domain over the USW Local 1998 collective agreement and related staff documents. Tested and registered. Demonstrates the system's flexibility beyond dental clinical policies.
Feature
April 7, 2026
Multiple Brains Online
Source documents consolidated across all brain directories. Internal and external knowledge-base brains brought online alongside the dental and USW 1998 brains.
Milestone
Phase 3: Demo-Ready
April 14, 2026
Speaker Notes Written
8 sections of speaker notes covering the full 30-minute TKF 2026 talk. Master outline with per-section timing. Q&A prep with 7 anticipated questions and answers.
Presentation
April 21, 2026
Demo Verification
5 demo questions fact-checked against source policy documents. 4 of 5 confirmed recording-quality. Ministral 3 14B confirmed natively multimodal. M3 Mac Studio production redeployed on port 7079. Web GUI formatting fixed.
Demo-Ready
April 28 - 30, 2026
Slide Deck
Slide deck built and refined. All fine-tuning examples fact-checked against the live dental RAG brain. Three slides patched with corrected policy references.
Presentation
May 1, 2026
Companion Site Launched
Built a 9-page companion website with slide viewer, failed runs deep dive, RAG setup guide, glossary, links, bio, and changelog. Self-contained for offline browsing. Designed for audience reference during and after the talk.
Milestone