From 0/30 to 15/15: What Dogfooding Taught Me About AI Lead Quality

A lead generation product that finds irrelevant leads is worse than a product that finds nothing.

Nothing costs a few seconds.

Bad leads consume attention, create false hope, and slowly destroy trust in the product.

I learned this while dogfooding ZeroToUser, the product I am building to help founders find potential customers in public conversations on Reddit and X.

During an internal test, ZeroToUser filled my daily queue with 30 suggested leads.

The dashboard looked productive.

There were scores, explanations, source links, and plenty of things to review.

But after reading every suggestion, I reached an uncomfortable conclusion:

None of the 30 felt like a genuine opportunity for ZeroToUser.

After rebuilding the lead discovery and qualification process, I reviewed the next 15 suggestions.

All 15 felt relevant enough to investigate or consider replying to.

That does not mean ZeroToUser has achieved 100% lead accuracy. The sample is still small, and relevance is not the same as conversion.

But the difference was large enough to change what I work on next.

The main bottleneck is no longer finding relevant conversations.

The next bottleneck is turning those conversations into users.

The Illusion of a Full Lead Queue

The first version of ZeroToUser did what many social listening and lead generation tools do:

1. Search public conversations for relevant keywords.
2. Score the results.
3. Rank the highest-scoring posts.
4. Place them in a daily lead queue.

Technically, the system was working.

It found posts. It created lead records. It gave each result a score.

But a full queue is not the same as a useful queue.

The original results included people:

• Sharing general marketing advice
• Promoting their own products
• Discussing advertising performance
• Asking broad questions about startup growth
• Looking for customers in unrelated markets
• Mentioning words such as "users," "leads," or "early adopters" without needing what ZeroToUser provides

These were not necessarily bad posts.

They simply were not sales opportunities for my product.

That distinction became the center of the entire optimization.

A Relevant Topic Is Not Automatically a Relevant Lead

Consider these two posts:

Here is the exact playbook I used to find my first 50 users.

I launched my SaaS, but I still don’t know where to find my first users.

Both posts contain similar language.

Both discuss finding early users.

But they represent completely different situations.

The first author is sharing knowledge.

The second author may be actively experiencing a problem ZeroToUser can help solve.

A keyword-based system can easily treat them as equivalent.

A useful lead qualification system cannot.

The real question is not:

Does this post mention one of my topics?

It is:

Is this person experiencing a problem my product directly solves, and are they currently looking for a way forward?

That is a much higher standard.

My First Instinct Was to Change the AI Prompt

When AI-generated results are poor, the natural reaction is to blame the model.

Maybe the prompt is too broad.

Maybe the confidence threshold is too low.

Maybe the system needs a larger model.

I almost started there.

Instead, I traced the entire journey from a public post to a suggested lead.

At a high level, the process looks like this:

Discover conversations
→ Understand why each conversation matched
→ Evaluate the author’s intent
→ Decide whether it is actionable
→ Rank the best opportunities

The important discovery was that lead quality did not depend on a single AI decision.

It depended on every stage agreeing on what a useful lead actually meant.

If discovery returns the wrong conversations, a better classifier cannot recover the conversations that were never found.

If the system cannot explain why a post matched, later scores become difficult to trust.

If matching is too literal, relevant paraphrases never reach the qualification stage.

If qualification is too generous, general market research becomes a sales lead.

The problem was not one bad prompt.

It was a pipeline alignment problem.

What I Changed

I focused on four principles.

1. Search for Intent, Not Only Exact Wording

Founders describe the same problem in many ways.

Someone who needs help finding early customers might write:

How do I get my first users?

But they might also write:

We launched two months ago and still have almost no signups.

Or:

Where do you find people willing to try an early-stage SaaS?

These posts express similar underlying intent without using the same sentence.

ZeroToUser now treats Search Topics as descriptions of customer problems, not rigid phrases that must appear word for word.

The system can recognize a limited set of closely related expressions while still keeping the search focused.

The goal is not to generate hundreds of speculative keywords.

It is to understand a few common ways the same pain is expressed.

2. Require Every Match to Have a Reason

A suggested opportunity should never appear merely because the system needed to fill a quota.

ZeroToUser now expects each candidate to have a defensible reason for matching the user’s product profile.

That reason might include:

• A direct customer problem
• Active solution-seeking language
• A request for recommendations
• A complaint about an existing workflow
• A clear connection to the founder’s target audience

If the system cannot explain the match, the post should not become a lead.

This sounds like a small product detail, but it changes the user experience substantially.

A lead list becomes much more trustworthy when every item can answer:

Why is this worth my time right now?

3. Separate Discovery from Qualification

Discovery and qualification have different jobs.

Discovery should be broad enough to find relevant conversations expressed in unexpected ways.

Qualification should be strict enough to reject posts that are merely adjacent to the product.

The revised process is:

Broad but controlled discovery
→ Remove obvious noise
→ Evaluate product and audience fit
→ Confirm active intent
→ Create a lead only when the requirements are met

This allows ZeroToUser to find useful paraphrases without turning every startup discussion into a sales opportunity.

It also means the daily target is a maximum, not a requirement.

If the system finds only two strong opportunities today, it should return two.

It should not add 28 weak suggestions just to make the dashboard look busy.

4. Measure Where Candidates Disappear

Previously, I could see only the final result.

If ZeroToUser returned zero leads, I did not immediately know whether:

• No conversations had been collected
• The Search Topics matched nothing
• Candidates had been rejected as low intent
• The system had already seen the same conversations
• A later processing stage had failed

I added clearer internal diagnostics for every major stage.

Now I can distinguish between:

“We found nothing”

and:

“We found several conversations, reviewed them correctly, and none met the Lead standard.”

Those outcomes look identical in the dashboard, but they require completely different product decisions.

The first may require better discovery.

The second may simply mean there were no strong opportunities that day.

The Result

Before the optimization, my internal review looked like this:

30 suggested leads
0 felt relevant enough to pursue

After rebuilding the process:

15 suggested leads
15 felt relevant enough to investigate

The important word is "felt."

This is a founder reviewing suggestions for his own product, not a controlled benchmark with thousands of labeled examples.

I am not claiming that ZeroToUser now has perfect precision.

I am also not claiming that all 15 leads will become customers.

What changed is more practical:

I stopped wanting to dismiss the queue and started wanting to open the conversations.

That is the behavior the product needs to create.

What 15/15 Does—and Does Not—Prove

The result suggests that the direction is working.

It shows that improving the full pipeline can have a much larger effect than tuning a single score.

It also gives me enough confidence to spend less time on basic relevance and more time on what happens after a lead is found.

But it does not yet prove:

• That every future batch will be equally relevant
• That relevant leads will reply
• That replies will become website visits
• That visits will become signups
• That signups will become paying customers

Lead relevance is only the first gate.

The next challenge is conversion.

The Next Phase: Turning Relevance Into Conversion

Finding the right conversation does not automatically create a customer.

The reply still needs to be:

• Helpful
• Specific to the conversation
• Natural rather than promotional
• Appropriate for the community
• Honest about the connection to the product

Timing matters too.

A relevant conversation can still be a poor opportunity if the founder enters it with a sales pitch instead of useful context.

The next phase of ZeroToUser will focus on questions such as:

• Which reply angles lead to genuine conversations?
• When should the product be mentioned?
• What makes someone visit the website after a reply?
• When is a follow-up appropriate?
• Which signals predict a signup or paid conversion?
• How should outcomes improve future recommendations?

This is where lead generation becomes a learning loop rather than a list of alerts.

The Bigger Product Lesson

The experience changed how I think about AI products.

An AI lead generation tool is not just an AI model wrapped in a dashboard.

Its usefulness depends on the entire decision process:

Data quality
× matching quality
× intent understanding
× product fit
× timing
× user action

A strong model cannot compensate for poor inputs.

A large number of results cannot compensate for low relevance.

And an impressive score cannot make an unrelated conversation worth answering.

For early-stage founders, three relevant conversations are often more valuable than 30 keyword alerts.

That is the product I want ZeroToUser to become:

A small daily list of conversations genuinely worth your time.

Building ZeroToUser with ZeroToUser

The most useful part of this process is that ZeroToUser is now helping me find potential users for ZeroToUser.

That creates a direct feedback loop:

1. Use the product to discover conversations.
2. Review whether those conversations are actually relevant.
3. Reply manually and helpfully.
4. Track which conversations lead to interest.
5. Use the outcomes to improve the product.

It is both a product strategy and a Build in Public experiment.

The first challenge was relevance.

The next challenge is conversion.

I will keep sharing what works, what fails, and what I learn along the way.

You can follow the journey at jasper.build or try ZeroToUser.

Frequently Asked Questions

What Is AI Lead Qualification?

AI lead qualification evaluates whether a conversation matches a specific product, target audience, customer problem, and level of intent. Its purpose is to distinguish actionable opportunities from general discussions that happen to contain similar keywords.

Why Do Keyword Alerts Often Produce Irrelevant Leads?

Keywords identify words, not necessarily intent. Someone sharing advice and someone actively asking for help may use the same language while representing completely different opportunities.

Is 15/15 a Verified Accuracy Rate?

No. It is the result of a small internal dogfooding sample. All 15 suggestions felt relevant enough to investigate, but a larger labeled dataset is needed before making a formal accuracy claim.

Is a Relevant Lead the Same as a Potential Customer?

Not always. Relevance means the person, problem, and conversation are worth reviewing. Conversion still depends on timing, product fit, the quality of the reply, trust, and follow-up.

Should a Lead Generation Product Always Fill the Daily Quota?

No. A quota should be a maximum, not a fill target. Returning fewer high-quality opportunities is better than adding weak suggestions simply to make the list appear full.

Can Founders Use Reddit and X for Customer Acquisition?

Yes, especially when they participate in existing conversations with useful and contextual responses. The goal should be to help first and introduce the product only when it directly contributes to the discussion.