Changelog

Built an automated fine-tuning pipeline in Python. 10 modules, 38 unit tests, full CLI. Designed to parse dental policy PDFs, generate training data, and fine-tune local LLMs on Apple Silicon.

Four fine-tuning attempts with escalating strategies: default settings, expanded data, pipeline fixes, curated training pairs. All produced hallucinated content: fabricated emails, invented forms, confident but wrong answers.

Tried Microsoft's RAFT (Retrieval-Augmented Fine-Tuning). Perfect smoke test scores with RAG active, but useless without it. Defeated the purpose of standalone fine-tuning.

2,198 training pairs, deeper LoRA, MLP targeting. Smoke tests passed. Real questions failed in five distinct ways. Conclusion: fine-tuning teaches style, not knowledge.

Reviewed Microsoft EMNLP 2024, Stanford FineTuneBench, Meta LIMA, Allen-Zhu ICLR 2025. All confirmed: fine-tuning cannot inject knowledge. RAG is the correct architecture. ModelForge parked permanently.

Deployed to the UofT Faculty of Dentistry M3 Mac Studio. Dental brain with 94 policy documents indexed. Open WebUI configured to route through the proxy.

Added model allow-list filtering, per-brain system prompts, and model-name routing (brain/model convention). Stood up additional clinical knowledge brains alongside the dental brain to prove the multi-domain pattern. Running on port 6969.

Built an export/import system for complete brain state, so a brain can be backed up, version-controlled, and restored on another machine. The dental brain alone holds 94 source documents indexed into searchable chunks.

Built a new brain domain over the USW Local 1998 collective agreement and related staff documents. Tested and registered. Demonstrates the system's flexibility beyond dental clinical policies.

Source documents consolidated across all brain directories. Internal and external knowledge-base brains brought online alongside the dental and USW 1998 brains.

8 sections of speaker notes covering the full 30-minute TKF 2026 talk. Master outline with per-section timing. Q&A prep with 7 anticipated questions and answers.

5 demo questions fact-checked against source policy documents. 4 of 5 confirmed recording-quality. Ministral 3 14B confirmed natively multimodal. M3 Mac Studio production redeployed on port 7079. Web GUI formatting fixed.

Slide deck built and refined. All fine-tuning examples fact-checked against the live dental RAG brain. Three slides patched with corrected policy references.

Built a faithfulness scoreboard against the real dental brain (94 policy ids). The real bug was invented policy numbers for ids that do not exist. The proxy now loads the real list and refuses those. Measured: fabrication on absent ids stopped. This is the start of the harness.

Needlestick answers were terse and wrong because the retrieved H.03 text was silently cut off before the model saw it. Raised the context floor so the injected documents actually fit. Retrieval was already finding the right file; the plumbing was the problem.

Own web UI for the brains: drag-and-drop a markdown file, it indexes immediately. You do not rebuild a chat app to update a policy. Citation footer lists the real retrieved files (the model had been inventing filenames). Corpus audit and lint land as modules in the dentistry tree.

Faculty product tree, version 0.1.0-dental. Dental brain only. Aimed at the Spark cluster, not a laptop Ollama path. Personal tools stay out. Same Perl proxy idea; own deploy, own corpus, own VERSION.

Faculty boxes. One is the front door (chat, proxy, embeddings). The other runs generation. Private interconnect between them. Clinic answers stay on Faculty hardware. Not a cloud chat app.

Post-generation check for dental: invented clinical detail that is not in the retrieved policy gets dropped. Policy-number list still applies. Streaming and thinking knobs for the Spark deploy. Operator bar: reliability over peak speed. Not in the TKF code snapshot.

Pins: when a question matches a topic, the proxy injects a known-good cheat sheet even if search grabbed the wrong file. Clinic questions (needlestick, dismiss, refund) graded against the real policies on Faculty hardware. Generation on the two-box Spark cluster.