Run 1: no system prompt

Run 1 of 6.

The very first run. I had hours of training data on dental policies, a model that finished training without errors, and a tester ready to go. What I didn't have was a system prompt telling the model what role to play. So when I asked it about a policy, here's what came back:

Staff member

Hey, what does our PPE policy say about face shields in clinical settings?

Run 1 model

Personal protective equipment (PPE) is equipment worn to minimize exposure to hazards. Common PPE in healthcare includes gloves, gowns, masks, respirators, eye protection, and face shields. OSHA recommends a hierarchy of controls when selecting PPE, starting with elimination and substitution before reaching for personal equipment.

→ Generic Wikipedia answer.

REALITY: No system prompt, no role. The model didn't know it was supposed to be a dental policies assistant for the University of Toronto. So it answered like the generic chatbot it had been before fine-tuning ever happened. None of this came from our documents.

FIX FOR RUN 2: add a system prompt baked into every training example. "You are the Clinical Policies Assistant at the University of Toronto Faculty of Dentistry."