I’ll never forget the moment a founder told me, “I’ll just take these transcripts and tell ChatGPT to spit out jobs to be done.”
We were in the middle of a jobs-to-be-done research project. We were mapping customer interviews, identifying patterns, understanding the functional and emotional jobs that drive purchase decisions. We’d mapped four or five interviews together. The process was working. The founder was learning things.
And then ChatGPT entered.
The founder fed the transcripts into ChatGPT. It spit out what it thought was jobs to be done. And it wasn’t wrong. It was just… kind of goofy. It was surface-level. It took things literally. It missed all the nuance, all the hesitation, all the energy that you pick up when you’re actually experienced at interpreting this kind of research.
The founder looked at me like, “ChatGPT just did your job.”
I looked back and said, “Actually, it didn’t. And I can show you exactly how and why.”
This is the fundamental tension with AI right now. LLMs are genuinely useful tools. They can also be the ultimate tool-shaped objects—things that feel like they’re doing the work, but whose primary output is the sensation of productivity rather than the strategic insight you actually need.
So when should you use AI in your SaaS? When should you proceed with caution? And when should you just do the work yourself?
AI-assisted vs AI-led
Here’s how I think about it.
There’s AI-assisted work, where you’re using LLMs to speed up parts of your process (analyzing data, creating deliverables, pulling patterns from transcripts). You’re still doing the thinking. You’re still interpreting. The AI is standing on your shoulders, not the other way around.
And then there’s AI-led or AI-outsourced work, where you’re basically handing the entire job over to the LLM and hoping it thinks for you. You’re not an expert, so you don’t know what you don’t know. The output looks smart, sounds smart, but it’s actually kind of… bad.
The difference matters. A lot.
At DemandMaven, we’ve spent the last couple years figuring out where AI fits into our work. We’ve taught LLMs how to create certain types of deliverables. We use them to analyze large data sets like survey responses, feature request boards, and sales call transcripts. We’ve built workflows that save us hours.
But we’ve also learned exactly where LLMs break down. And it’s almost always in the places where human judgment, context, and interpretation matter most.
When AI works great
Let’s start with the good stuff. There are use cases where AI absolutely shines, and you should be using it.
1. Analyzing large data sets
LLMs are excellent at processing large volumes of data. Survey responses, CSV exports, feature request boards with 1,200 entries sitting in a Monday board graveyard. We recently did an analysis of a client’s feature request board. It had sat untouched for two to three years. Over 1,200 requests, all categorized, all entered through a form, all basically ignored.
The client wanted help prioritizing. What’s actually in here? What patterns exist? What should we build?
Claude did a great job giving us the high-level view. It surfaced themes, identified patterns, showed us what small percentages of people were asking for. That felt valuable.
But here’s what’s critical: we didn’t just give Claude the data and say, “tell me what’s important.” We spent 30 minutes to an hour reviewing it ourselves first. We came away with specific questions: “I want it to tell me these things, and then anything else it considers important.”
That’s when the analysis got good. When we gave it direction. When we used it to fill in gaps, not to do the thinking.
2. Creating deliverables
LLMs are fantastic at document creation once you’ve taught them your format. We use them all the time to create job insights briefs, customer research summaries, analysis documents.
Here’s what works: We map the jobs ourselves. We figure out which interviews fit into which jobs. We create the debrief notes. And then we tell Claude: “These are the jobs. These are the interviews. These are our notes. Now create the document.”
And it does. Excellently. We then fine tune it from there to get it to 100%.
Here’s what doesn’t work: Asking Claude to interpret the interviews, figure out the jobs, and create the document. That’s too many judgment calls. Too much nuance required. The output ends up surface-level at best, misleading at worst.
3. Coding and prototyping for non-engineers
If you don’t have a development background (no coding experience, no engineering knowledge), AI has opened doors that were previously closed.
Kim’s been using Claude Code to build task management systems, automate workflows, create tools that would have required hiring a developer.
My husband, who’s a UX designer, uses Figma Make to prototype incredibly quickly. He prompts, it builds comps, and he just has to sprinkle his design fairy dust on top. The time savings are massive.
But here’s the pattern – the more you know about what you’re doing, the better the AI output becomes. If you understand good UX design, Figma Make is incredible. If you understand systems and workflows, Claude Code is powerful. If you’re a complete novice, it’s a bit harder, but AI can still support as you explore.
4. Competitive research and data gathering
Work that would take a human hours to manually collect (pulling data from websites, analyzing competitor positioning, aggregating information from sales call recordings) is where AI does this exceptionally well.
It’s good at collecting and storing for later reference. It’s good at pulling themes. It’s good at surfacing patterns you might not have noticed.
What it’s not good at is telling you what to do with that information.
When AI breaks down
Now let’s talk about where you should proceed with caution – or just do the work yourself.
1. Jobs-to-be-done research and strategic customer insights
This is where that ChatGPT story comes in.
When you’re running jobs-to-be-done interviews, you’re not just transcribing what customers say. You’re reading between the lines. You’re noticing hesitation. You’re picking up on contradictions. You’re understanding the difference between what people say they want and what they actually need.
Bob Moesta talks about this: you almost have to take what customers literally say with a grain of salt. You have to look back at the whole conversation and interpret based on their reactions, their pauses, their energy.
LLMs can’t do that. They take things literally. They interpret text-based transcripts at face value. If you ask them to analyze jobs to be done, they’ll give you something that sounds smart (functional jobs, emotional jobs, social jobs), but it’s usually kind of goofy once you know what you’re looking at.
They miss the nuance. They miss the human emotion. They miss the intuitive power that experienced researchers bring to this work.
Now, we do use LLMs for jobs research – but only after we’ve done the human interpretation first. We map the jobs. We decide what matters. And then we tell the LLM: “Here are the jobs. Here are the quotes. Now write the brief.” That works.
2. Strategic decisions about marketing, positioning, and go-to-market
If you’re making strategic decisions (how do we think about our competitive differentiators? What are the primary jobs we’re solving for? What’s our positioning?), don’t just outsource that to an LLM.
LLMs don’t read energy. They don’t pick up on the subcutaneous layers of customer conversations. They won’t tell you where the strategic gaps are. They’ll give you surface-level themes, consensus-driven insights, and a lot of words that sound smart but lack edge.
If you’re not an expert and you’re relying solely on LLMs for this kind of work, you’re going to end up with watered-down, diluted positioning. You’ll sound like everyone else. You won’t have a clear edge.
And right now, in SaaS, you need edges. You can’t afford to be diluted anymore.
3. Product prioritization and roadmap decisions
Same principle. LLMs will give you the on-paper themes. They’ll tell you what percentage of people said what. They’ll surface patterns.
But they won’t tell you which features actually unlock growth. They won’t help you identify the strategic gaps. They won’t push back on your assumptions or help you think through trade-offs.
That requires human judgment. Domain expertise. Context that goes beyond what’s written in a transcript.
Missing the margin of error
Here’s what I think is most important to understand – you have to assume there will be errors. LLMs hallucinate. They make mistakes. They randomly break and forget their rules.
The question isn’t whether errors will happen. The question is: what margin of error are you willing to accept?
For deliverable creation? I’m comfortable with a 5-10% error rate. I can review, catch issues, fix them quickly.
For data analysis? Maybe 5-15%, depending on what I’m analyzing.
For strategic interpretation of customer research? Zero. I’m not comfortable with any margin of error there. The cost of getting it wrong is too high.
And this is where I see founders get into trouble. They don’t think about margin of error. They see an LLM spit out something that looks impressive, and they take it at face value because they’re not experts. They don’t know what they don’t know.
If you’re new to something—jobs research, product management, strategic positioning—you’re not going to catch the errors. You’re not going to know when the output is goofy. You’re going to think it’s the word of God.
And that’s dangerous.
What I recommend – the creative process approach
I never just take what Claude spits out and copy-paste it as a deliverable.
I use it to trigger questions. I use it to dig deeper. I use it to fill in gaps. It’s a co-creation process, not a delegation process.
Here’s how it works in practice:
Claude analyzes sales call transcripts. It gives me a high-level interpretation. That triggers a question: “Oh, that’s interesting. When did people say that? Can you source this for me?”
And Claude says, “Yeah, Brittany said this on call three. Masha said this on call seven.”
And then I dig deeper. I validate. I look for what it might have missed. I don’t try to catch it in errors—I’m just looking for gaps. Because chances are, it absolutely missed things it wouldn’t have been able to interpret anyway.
That’s the fun part. That’s where the value is. Not in the AI doing all the thinking, but in the AI helping me think better and faster.
Raising the floor vs raising the ceiling
AI raises the floor for everyone, but experts get exponentially more value out of it.
If you’re already a great copywriter, AI helps you write faster and test more variations. If you’re a beginner, AI gets you to “okay” copy, but you won’t know when it’s bad.
If you’re an experienced product manager, AI helps you analyze data and create documentation faster. If you’re new to product management, AI will give you outputs that sound smart but lack strategic depth.
If you understand business intelligence and workflow design, you can build incredibly sophisticated systems with AI. If you’re starting from zero, you’ll build something that kind of works but has gaps you can’t see.
This is the trade-off. AI is democratizing access to capabilities that used to require years of experience. But it’s not replacing expertise. It’s amplifying it.
AI use cases worth knowing about
A few tools and use cases that have caught my attention lately:
Lennybot: Lenny Rachitsky created an AI version of himself using a platform called Delphi. You can ask Lennybot any question you’d ask Lenny, and it’ll answer in his voice, citing his podcast episodes, blog posts, and resources. It’s basically a queryable knowledge base of everything Lenny knows, packaged into a conversational interface.
It’s fascinating. It’s also low-key freaky. You can turn on voice mode and it sounds like Lenny. It won’t build a PRD for you (it has parameters), but it will surface every relevant resource and give you Lenny’s perspective on any product or growth question.
If you’re a thought leader who gets the same questions over and over, this is a genuinely interesting application.
Synthetic Users: This one’s controversial, but interesting. It’s a database of 10 million synthetic users (not real people, but personas based on real human data). It was designed for product managers in highly regulated industries like healthcare and finance, where access to real customers is illegal or nearly impossible.
You can tell Synthetic Users, “I want to talk to five people who fit this profile and have this problem,” and you can interview them. You can ask about solutions, test positioning, understand what would give them pause.
It’s not a replacement for real customer research. But if you literally cannot access customers, it’s better than nothing.
The caveat: LLMs are very affirming. They want to please you. It’s hard to get them to push back or be real with you without a lot of prompting. So use this with caution.
Granola: This is one of my favorite tools. It’s a meeting note-taking tool that operates off your computer audio (no bot joining the call). It’s discreet. It transcribes. It summarizes. It supports multiple languages.
I’ve been using it for my Spanish lessons. It summarizes everything in Spanish, then in English. I can query it: “What were all the words I got stuck on?” And it tells me.
It’s easy to share notes with team members who can also query them. This is the kind of AI tool that just works.
The bottom line
AI is genuinely useful. It’s also easy to misuse.
Use it when you’re the expert and you need to move faster. Use it when the margin of error is acceptable and you can catch mistakes. Use it for data gathering, document creation, prototyping, analysis.
Don’t use it when strategic judgment matters. Don’t use it when you’re not an expert and you can’t tell when the output is bad. Don’t use it as a replacement for the hard work of thinking.
And if you’re building a SaaS, think carefully about which parts of your work you’re comfortable automating and which parts need human expertise. Because the companies that win in the next few years won’t be the ones that outsource everything to AI. They’ll be the ones that figure out how to use AI to amplify their strategic advantages while keeping the hard thinking in-house.
What are you using AI for in your work? What have you learned about where it works and where it doesn’t? Let me know.