When telemedicine expanded during the pandemic, no one proposed re-running randomised controlled trials to establish whether talking to a doctor helps. The evidence for clinical consultation was already clear. The question was whether remote delivery maintained what mattered: diagnostic accuracy, treatment adherence, and the therapeutic relationship. Studies focused on “diagnostic concordance” and “non-inferiority”—measuring whether the new delivery mechanism preserved the effectiveness of the old one. 1
Could the spine of effective practice survive a new delivery mechanism?
Career support faces the same question with AI. The spine has been established. As we saw in Part 4, Individual Placement and Support (IPS) works through sustained relationship, integration with other services, rapid employer contact, zero exclusion, and time-unlimited availability. Twenty-eight RCTs confirm this. 2
The question isn’t whether AI-augmented career support “works,” as if we’re starting from scratch. It’s whether AI augmentation maintains or undermines the principles we already know are effective.
This changes how we should evaluate. We’re not measuring outcomes to discover if something works. We’re measuring fidelity to assess whether a new delivery mechanism preserves what works.
I’ve been thinking about this a lot, because we’re in the middle of building exactly this kind of system at Stay Nimble. And getting it right matters, not just for us, but because I suspect a lot of organisations are going to try bolting AI onto career support in the next few years. Most will do it badly. The fidelity framework offers a way to tell the difference.
The non-negotiables
IPS research identifies principles, not scripts. The fidelity scale measures whether services embody these principles, not whether they follow rigid protocols. This matters because it distinguishes what must stay constant from what can evolve.
Sustained relationship: Effective support depends on someone knowing your circumstances over time. Not repeating your story to a new adviser every visit. Not navigating intake processes repeatedly. Not being treated as a fresh case each interaction. Trust develops and context accumulates. The adviser learns what’s actually going on, not just what the intake form captured.
Integration with other services: Employment barriers rarely exist in isolation. Health conditions affect work capacity, housing instability disrupts job searches, caring responsibilities constrain availability, and benefit rules create perverse incentives. IPS addresses this by embedding employment specialists directly in clinical teams. The broader principle is straightforward: career support must integrate with whatever reality the person is navigating.
Rapid employer contact: Traditional approaches require people to become “job-ready” before meeting employers, going through training courses, CV workshops, and confidence building. IPS inverts this. Contact employers within weeks, learn from real interactions, adjust based on actual feedback. The labour market teaches more than preparation programmes.
Zero exclusion: No one is “too difficult” or “not ready.” The people screened out by traditional services, those with complex health conditions, chaotic circumstances, and long benefit histories, are precisely those who need sustained support most. Excluding them optimises metrics while abandoning the mission.
Time-unlimited availability: Support continues as long as needed, not until a programme window closes. Circumstances change. Progress isn’t linear. Someone might need intensive help for three months, then light-touch check-ins for a year, then intensive help again when a job ends.The service adapts to the person rather than forcing the person into the service’s structure.
These principles are the spine. They’re what produces IPS’s consistent advantage over traditional approaches. Any delivery mechanism, whether digital, AI-augmented, hybrid or otherwise, should be evaluated on whether it maintains them.
What AI changes
AI offers capabilities that could extend each of these principles, or undermine them. The question is which.
Availability without presence: Human advisers have working hours, caseload limits, geographic constraints. An AI assistant can be available at 2am when anxiety peaks, during a lunch break before an interview, in the moment someone receives a rejection. This could extend support into gaps human services can’t fill. Or it could fragment the relationship, substituting transactional interactions for sustained engagement.
Memory without continuity: AI systems can retain conversation history, recall previous discussions, track progress over time. This could provide continuity that overloaded human advisers struggle to maintain. Or it could create an illusion of relationship without actual human connection. A system that “remembers” without genuinely knowing.
Scale without integration: AI can serve thousands simultaneously, potentially democratising access to support that was previously rationed. But integration with health, housing, and benefits systems requires human coordination, professional relationships, and institutional arrangements that AI can’t replicate. Scale might come at the cost of the integration that makes support effective.
Speed without judgment: AI can respond instantly, provide information on demand, accelerate routine tasks. But knowing when someone needs information versus when they need to be heard, when to push and when to hold back, when the presenting issue masks something deeper; these require judgment that current AI systems don’t reliably possess.
Consistency without adaptation: AI can deliver evidence-based content reliably, without the variation that human delivery introduces. But adaptation to individual circumstances, cultural contexts, and emerging situations is where skilled human practice excels. Consistency might mean rigidity.
The question for any AI augmentation is: does this extend the spine or compromise it?
The questions that matter
If we’re serious about fidelity-based evaluation, we need to ask specific questions about whether AI augmentation maintains effective principles.
Does it sustain or fragment relationship?
The risk is that AI interactions become transactional episodes disconnected from ongoing human support. Someone asks a question, gets an answer, the interaction ends. No accumulation of context. No building trust. No one knowing their story.
If AI is going to preserve fidelity, it needs to extend relationship rather than replace it. AI should maintain continuity between human sessions: checking in, following up on commitments, providing information requested during coaching conversations. The human adviser remains the anchor. AI fills gaps.
So you’d want to know:
Does the AI share context with human advisers?
Do human sessions build on AI interactions?
Does engagement with AI predict sustained engagement with human support, or substitute for it?
Does it extend access or create new barriers?
AI could democratise access, reaching people who can’t attend in-person appointments, who face stigma in traditional services, who need support outside business hours. Or it could exclude those without digital literacy, reliable internet, or comfort with technology.
If we’re serious about fidelity, AI should reduce barriers, not add new hoops to jump through. No one screened out because they don’t engage “properly” with digital tools. AI as one channel among several, not a gateway.
So you’d want to know:
Who uses AI support and who doesn’t?
Does AI reach populations underserved by traditional provision?
Are there patterns in who drops off?
Does it accelerate employer contact or delay it?
AI could speed job matching, identify opportunities, prepare applications, facilitate introductions faster than human-only processes allow. Or it could become another preparation stage. “Complete the AI modules before meeting employers” becomes a way of adding delay rather than reducing it.
Fidelity means using AI to accelerate real-world contact, not substitute for it. AI should help research employers, draft approaches, and prepare for conversations, all in service of faster human-to-human connection with the labour market.
So you’d want to know:
What’s the time from engagement to first employer contact?
Does AI interaction precede employer contact or replace it?
Does it uphold zero-exclusion or enable new forms of parking?
AI could extend support to complex cases that overwhelm human caseloads. Or it could become a parking lot, with difficult cases directed to AI while human attention goes to those closer to employment. “The algorithm will help you” as a polite way of saying “we don’t have time for you.”
Fidelity means using AI to enhance support for complex cases, not divert them. AI should flag when someone needs human attention. It should surface complexity rather than masking it. It should extend capacity without triaging people out of genuine help.
So you’d want to know:
Who receives human support versus AI-only support?
Do complex cases get more combined support, or less human attention?
Does AI identify escalation needs?
Does it enable time-unlimited engagement or impose new constraints?
AI could make sustained availability economically viable, providing ongoing light-touch support that human services can’t afford to provide. Or platform design might impose engagement windows, subscription limits, or usage caps that recreate programme time-limits in digital form.
Fidelity means using AI’s scalability to provide genuinely unlimited support. Check-ins months after employment. Availability during future transitions. The relationship not ending because a funding period closed.
So you’d want to know:
What do engagement patterns look like over time?
Do people return when circumstances change?
Is support available when they do?
What we’re trying to do with Ask.Nim
As we continue to build Ask.Nim at Stay Nimble, these are the exact questions we keep using as constraints. The goal isn’t to build a cheap replacement for human interaction; it’s to see if AI can be built to high-fidelity standards. The design choices reflect this framework.
AI extends the coaching relationship rather than replacing it. Ask.Nim is positioned as available between human coaching sessions, maintaining continuity when the coach isn’t present. Conversations can inform subsequent coaching. We’re working towards the assigned coach seeing what someone has been working through with AI. We’re figuring out what context from a human coaching session can inform Ask.Nim. The idea is for context to accumulate across both channels.
This is a deliberate, yet complex, choice. We could have built a standalone AI career adviser and positioned it as cheaper than human coaching, but that would fragment the relationship. So instead, Ask.Nim is being designed to make the human coaching more effective, not to replace it.
Human escalation is designed in. The system identifies when someone needs human support: complexity beyond AI capability, emotional content requiring human presence, or situations where advice could cause harm. Escalation paths exist, and improving. The AI isn’t a gatekeeper preventing access to people; it’s a bridge ensuring they reach the right support. A human is always available.
Access extension is explicit. People who can’t attend daytime appointments, who face anxiety about formal services, who need support at unconventional hours; these are target users, not edge cases. The design asks who doesn’t access traditional support, and whether AI can reach them.
Zero-exclusion principles apply to digital engagement. No one is screened out for “not engaging properly” with the platform. Low engagement isn’t treated as disqualification. People can use it intensively or occasionally. The service adapts to patterns rather than requiring patterns to fit the service.
These aren’t outcome claims. We’re not saying “Ask.Nim produces employment.” We’re saying: here are design commitments, and they can be evaluated through fidelity measurement. Does the AI actually maintain relationship continuity? Do complex cases actually receive enhanced support? Does access actually extend to underserved populations?
The evidence will come from observing the practice, not from measuring job placements and pretending the technology alone produced them.
Why this matters beyond us
I’m sharing our approach because a lot of AI career support is going to be built in the next few years. Most of it will be built badly. The fidelity framework offers a way to tell the difference.
If you’re evaluating an AI career tool, ask the fidelity questions. Does it sustain relationship or fragment it? Does it extend access or create new barriers? Does it accelerate employer contact or add another preparation stage? Does it support complex cases or park them? Does it enable time-unlimited engagement or impose digital constraints?
These can be assessed through service audits, user research, engagement data, and practice observation. They don’t require waiting years for outcome data that will be confounded by labour market conditions, economic cycles, and countless factors beyond anyone’s control.
And if you’re building AI career support, design for fidelity from the start.
It’s much harder to retrofit.
What we’re building toward
What we’re trying to build is simple to describe and hard to do: permanent career support infrastructure, available when needed, delivered according to evidence-based principles, using whatever combination of human and AI capability serves people best.
Not programmes with start dates and end dates. Not outcome metrics that attribute employment to interventions. Not eligibility criteria that screen out complexity. Not the fusion of support with surveillance that poisons both.
Infrastructure. Like healthcare. Like education. Like the services we decided long ago should be permanently available because needs recur throughout life.
AI doesn’t change this vision. It offers new tools for realising it, or new ways to fail. The choice is ours.
We can use AI to extend genuine support, maintain fidelity to what works, and build capacity for permanent needs with surge capability for whatever’s coming.
Or we can bolt AI onto existing dysfunction, add a digital layer to the programme churn, and wonder why technology didn’t solve problems that were never technological.
The spine is established. The evidence is clear. The fidelity framework exists. What remains is commitment: to building permanently, measuring appropriately, and using every available tool, human and artificial, in service of principles we’ve understood for twenty years.
We’ve been failing at this for 117 years. We could stop.
* * *
Next week: The final piece asks what happens when endemic need meets epidemic demand, and what we’re choosing by not preparing.
Note: I have used Perplexity to help with researching and editing this series.
For example, Mayo Clinic research during the COVID-19 pandemic evaluated the shift to telemedicine by measuring “diagnostic concordance”—finding an 86.9% match between provisional diagnoses made via video and subsequent in-person visits. See: Demaerschalk, B.M. et al., ‘Assessment of Clinician Diagnostic Concordance With Video Telemedicine Diagnoses’, JAMA Network Open, vol. 5, no. 8, 2022.
https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2795871
IPS Employment Center evidence database. https://ipsworks.org/index.php/evidence-for-ips/