“When it doesn't sound like me. I'll ask ChatGPT which version is best as well, which probably defeats the point. It always has a reason for its choice, though, and I find that reassuring.”
P17 · Law · Q12
Looks better. But is it better?
The difference between checking a fact and trusting advice.
Fluency is easy to see. Quality is harder.
Participants describe several ways of checking AI output, with different strengths and limits. They recognise an invented responsibility, test a technical claim against documentation, ask a mentor, open the company's own website. This is not a corpus of people accepting whatever they are given.
But the methods reach unevenly. Participants can often compare a claim with their own experience, although their accounts also describe missed errors. Checking whether a piece of advice will help an application requires knowing how the application will be assessed — and access to that knowledge varies across the accounts. The problem is not an absence of scrutiny. It is the uneven availability of something credible to check against.
Read the supporting evidence ↗02AWhat can be checked, and how
Participants describe several bases for judgment, ranging from external references to familiarity with their own experience and craft.
Against their own experience. Participants use what they know about their work to spot invented achievements and check technical claims.
"If it's made up something about my job, I'll know. I check it against my figures and what they've asked for."
— P03 · Business and marketing, online · Q12
"The technical stuff I can check. If something looks odd, I'll look at the docs or try it myself."
— P06 · Computing, online · Q12
"It got an event ID wrong, so I went and checked that. Still useful practice."
— P18 · Cybersecurity · Q3
Against how they actually speak.
"For language, I can judge it cuz I couldn’t write like that. I also read the output aloud to see if it sounds like something I could actually say."
— P08 · Criminology and law, online · Q12
Against craft judgement. One participant describes a route the others do not have — she can see exactly what a tool changed.
"With the editing software I can see what a feature has changed. It's easier to judge that than a whole generated idea."
— P12 · Film, TV and digital production · Q4
One participant combines several forms of checking:
"Company information, their website. Technical things, course notes or someone qualified. With wording I check that my manager would recognise the job I'm describing. If it makes me responsible for everything, something's gone wrong."
— P20 · Business management · Q12
02BWhere the checking runs out
The same participants describe a hard stop when the claim concerns recruitment itself.
"The recruitment stuff is harder. I don't know what settings they've got."
— P06 · Computing, online · Q12
"Technical stuff, test it or check the documentation. Recruitment advice is harder. I ask people who've hired analysts. I don't put much weight on a CV score."
— P18 · Cybersecurity · Q12
"When it starts telling me about ATS scores, I don't know how to check that."
— P03 · Business and marketing, online · Q12
"I don't really know. If it sounds generic I don't like it."
— P04 · History · Q12
"If you get the job maybe? I’m not sure."
— P11 · Business and tourism management · Q12
That last answer is worth sitting with. The only test she can imagine is the outcome — which arrives long after the decision it was meant to inform, if it arrives at all.
02CThe proxies that fill the gap
When an external standard is difficult to access, participants describe other signals that feel informative. An opaque score, institutional endorsement, agreement between tools and AI reviewing its own output raise different questions about the basis for trust.
A score.
"I like seeing the score go up. It makes me feel I've improved it, even if I can't really tell from reading it."
— P19 · Business management · Q4
Institutional branding.
"The university logo made it feel approved I guess so I treated the score more seriously. I don't know whether it uses the same sort of model as everything else."
— P09 · Business and computing · Q12
Agreement between tools.
"I compare the answers. If two tools say the same thing, I tend to believe it more. I know that might not mean much."
— P16 · Business management · Q12
Asking the machine to review itself.
"I know I'm asking the same thing that wrote it, but I still do it."
— P19 · Business management · Q12
The awareness is not the missing piece. Some participants recognise the weakness of a signal while continuing to find it reassuring. Alternatives may exist but feel less accessible or immediate.
02DDoing the one thing you can do
Keyword insertion continues alongside expressed scepticism; it is described as something concrete the participant can do.
"Not that certain, actually. I know they use the systems. I don't know the settings. I still put the keywords in because well at least that's something I can do."
— P03 · Business and marketing, online · Q8
"Matching what they ask for. Experience is probably the main thing …" … "I still think the keywords matter."
— P16 · Business management · Q9
02ERevision with no end in sight
Without a standard, there is no signal that a draft is finished. Several participants describe revision expanding to fill the time available.
"I couldn't choose. One gave it a better score than the other and I was trying to work out why. It was only about three hundred words. I spent most of the evening on it."
— P17 · Law · Q3
"Can it make me stop editing? That's what I'd want. Give me a couple of versions and make me choose. I'd probably open another chat, actually, so I don't know if that would work."
— P17 · Law · Q18
"Like how do you know when enough is enough cuz you can really spiral outta control. I can spend hours trying different versions until I no longer know which one is clearer."
— P09 · Business and computing · Q14
"Bits of it. Then I asked it to compare the two versions. By the end it was close to what I'd written in the first place. I do that quite a lot."
— P04 · History · Q3
"I'll feel better for ten minutes, then it gives me five alternatives and I wonder what was wrong with the first one."
— P01 · Psychology · Q19
In this account, a deadline or another person provides a stopping point:
"The deadline. Or my sister saying enough now. Usually the deadline."
— P17 · Law · Q14
One more version. One more question.
01 →Generate a version
A draft or suggestion provides something concrete to work with. The immediate benefit may be speed, structure or language.
02 →Seek reassurance
A score or another AI judgment offers a visible response. Agreement is not necessarily independent evidence of quality.
03 →Revise again
Without a clear standard, another version can reopen the decision. Some accounts describe deadlines or people interrupting this loop; not everyone enters it.
Where the story
gets less tidy.
Checking a fact about yourself is not a minor thing. Removing an invented qualification prevents a participant walking into an interview unable to defend their own CV. Several participants describe exactly that catch. These checks catch real mistakes. They just cannot tell students whether the rest of the advice will help them get a job.
Some participants have an external standard and use it. P05 checks against the NHS person specification. P20 checks against what his manager would recognise. P02 and P14 ask people who have hired in their field. A written specification or knowledgeable person can provide a more concrete basis for judgment without resolving every uncertainty.
Not everyone enters the revision loop. P15 describes leaving an application for another day rather than iterating into the night. P20 says CV editing "doesn't take that long now" and has reallocated his effort to timed tests. The loop is a pattern in the corpus, not a universal experience.
Evidence & qualifications
Selected quotations. Participant codes link to the profile index; question numbers locate each extract in the transcripts.
“The technical stuff I can check. If something looks odd, I'll look at the docs or try it myself. The recruitment stuff is harder. I don't know what settings they've got.”
P06 · Computing, online · Q12
“If it's made up something about my job, I'll know. I check it against my figures and what they've asked for. When it starts telling me about ATS scores, I don't know how to check that.”
P03 · Business and marketing, online · Q12
“Maybe ease of use? The university logo made it feel approved I guess so I treated the score more seriously. I don't know whether it uses the same sort of model as everything else.”
P09 · Business and computing · Q12
“ChatGPT every day. Canva, Grammarly. Sometimes Gemini. And the CV checker. I like seeing the score go up. It makes me feel I've improved it, even if I can't really tell from reading it.”
P19 · Business management · Q4
“I couldn't choose. One gave it a better score than the other and I was trying to work out why. It was only about three hundred words. I spent most of the evening on it.”
P17 · Law · Q3
“I'd ask the placement team if the CV was any good. Or my friend who's an engineer. AI's fine for a sentence.”
P02 · Mechanical engineering · Q12
“Company information, their website. Technical things, course notes or someone qualified. With wording I check that my manager would recognise the job I'm describing. If it makes me responsible for everything, something's gone wrong.”
P20 · Business management · Q12
“It's there if I need to get started. I don't use it instead of an appointment, really. If anything I'm more likely to leave the application for another day than spend all night talking to ChatGPT.”
P15 · Business management · Q17
Quotations retain transcript wording; omissions within sentences are marked. Methods and source notes ↗