AI Lead Qualification: The Feedback Loop That Eats Your Best Segment

You have 400 unread DMs, three reps, and no reliable way to tell which twelve of those conversations are worth answering first. So you point a language model at the inbox and ask it to score them. Six months later the score is quietly running your pipeline, nobody can explain why the leads it likes are the leads it likes, and the segment that used to be your best one has stopped showing up at all.

This post is about the part between those two paragraphs: what to automate, what to refuse to automate, and how to find out that your qualifier has gone wrong before the wrong thing is four quarters old.

BANT did not go out of fashion. It stopped being computable.

Budget, Authority, Need, Timeline. IBM shipped it in the 1960s as a checklist for a rep on a scheduled call with one buyer who had a purchase order and a phone. It worked because those four fields were knowable by asking four questions of one person.

Two of the four assumptions are now simply false. Authority assumes a single person can say yes. Gartner's May 2025 sales survey of 632 B2B buyers found that buying groups now run from five to 16 people across as many as four functions, and that 74% of buyer teams show unhealthy conflict during the decision process. Read Gartner's definition of unhealthy conflict slowly, because it is BANT's obituary: members hold conflicting objectives, disagree on the best course of action, or are overruled by decision makers outside the team. There is no Authority field to fill in. There is a committee having an argument you are not invited to, and the person answering your DM may be overruled by someone you will never speak to.

Budget assumes money precedes need. In product-led and self-serve motions the order reverses: somebody starts using the thing, then goes and finds money for it. Asking "what is your budget" at that stage is asking a question whose answer will be invented on the spot to make you go away.

The scoreboard tells the same story. Forrester analyst Simon Daniels, writing in November 2023, put it flatly: "fewer than 1% of leads convert to closed deals, a failure rate that would normally be unthinkable." That number is not an indictment of marketing. It is an indictment of the unit of analysis. If a purchase involves eight people, and you insist on scoring individual humans as if each were a deal, then by construction at least seven of every eight qualified records are wrong, and you have built a metric that cannot be right.

For anyone working in DMs, though, there is a more immediate problem, and it kills BANT before any of the above matters. BANT is an interrogation protocol. It requires you to ask four direct questions. In a Telegram or Instagram thread, at message three, "what is your budget for this?" is not qualification. It is the message that gets you ghosted, reported, or both. The framework assumes an information-gathering ritual that the channel does not permit.

What replaced BANT is not MEDDIC. MEDDIC is BANT with more letters and the same load-bearing assumption: a rep, in a call, filling in fields. It is genuinely better for a nine-month enterprise cycle with a real discovery call, and it is completely useless as an automation target, because the fields it wants (Economic Buyer, Champion, Decision Criteria) are things a person learns over six conversations, not things a classifier reads off a message.

The thing that actually replaced BANT in working revenue teams is not a framework at all. It is a scoring function. And a scoring function has properties that a checklist never had: it can be wrong in ways that compound, it can be biased in ways nobody notices, and it will keep producing a confident number long after it has stopped meaning anything.

Fit and intent are two different questions, and one number cannot answer both

Every usable qualification model in 2026 is built on two axes that people constantly collapse into one.

Fit is what is true about the account regardless of this conversation. Company size, industry, region, tech stack, whether they have the problem you solve. Fit is explicit, slow-moving, and mostly verifiable from data that exists outside the thread. A lead's fit score should barely move from week to week.

Intent is what this person is doing right now. Replied to a DM. Asked what it costs. Pulled a colleague into the thread. Visited the pricing page twice on a Sunday. Intent is behavioural, fast, and worthless in three weeks.

The near-universal mistake is adding them together. You compute fit 0 to 50, intent 0 to 50, sum to a single 0 to 100 lead score, and route on the total. Now a score of 85 means either "perfect-fit enterprise account browsing idly" or "student with a credit card who read every page on your site tonight," and those two require opposite actions. You have thrown away the only information that told you what to do.

Keep them as a pair. The action lives in the quadrant, not the sum.

The low-fit, high-intent box deserves a moment. It is the box everyone automates hardest, because it is full of people who cost rep time and rarely buy. It is also, structurally, the only box where a new segment can first appear. Your fit model was built from accounts you have already sold to. A genuinely new type of customer will always arrive as low fit and high intent, because the model has never seen them. Automating that box to zero is how you make your fit model permanently correct about a market that is changing without you.

What a language model can actually judge from a conversation

Be precise about this, because most "AI lead qualification" copy is vague about it on purpose.

An LLM is very good at reading a messy thread and turning it into fields. It is good at normalising "we're like 40 people ish" into a headcount band, at spotting that "we're looking at Intercom too" is a competitor mention, at noticing that "before our renewal in September" is a deadline. This is extraction and classification over text, and it is the single most valuable thing the technology does for a sales team, because the alternative is a rep typing into a form, which they do not do.

What it is not good at is being a judge. There is now decent evidence for this. Norman, Rivera and Hughes published the largest systematic evaluation of LLM-as-a-Judge to date in June 2026: 21 judges from nine providers, 118 runs, roughly 541,000 individual judgments. Their findings are unkind. Agreement measured by raw exact match, which is how nearly everyone validates a judge, overstates the thing you care about: correcting for chance with Cohen's kappa deflated the numbers by 33 to 41 percentage points on MT-Bench. Judge rankings shifted by up to 14 positions depending on which benchmark you used. And two production-deployed judges showed test-retest reliability above 0.95 while simultaneously showing position bias above 0.10, which the authors call a consistency and bias paradox: the model gives you the same answer every time, and the answer is partly a function of which option you listed first.

Read that last one twice if you are about to ship a scorer. Consistency is not correctness. A model that returns 78 every single time for the same thread is not thereby right about 78.

Then there is confidence. If you are tempted to use the model's own stated certainty as a gate, note that recent calibration work across four models and 24,000 trials found expected calibration error ranging from 0.122 for the best-calibrated model up to 0.726 for the worst, with the worst-calibrated model achieving 23.3% accuracy while reporting high confidence. Confidence and accuracy are different variables. "The model said it was 90% sure" is a string, not a probability.

The design rule that falls out of this: let the model extract, let arithmetic score. The LLM's job ends at populating fields with evidence attached. What happens to those fields afterwards should be a formula a human can read, argue with, and change on a Tuesday afternoon.

The extraction layer, with a real thread

Here is the shape an inbound DM actually arrives in, in a unified inbox. Marta is invented, but nothing about how she types is. Nobody writes like a form.

14:02  marta_k: hey saw your thing in the shopify tg group
14:02  marta_k: we do support for like 6 stores rn and its a mess,
       4 of us all in one whatsapp
14:02  marta_k: does this connect whatsapp
14:31  you:     it does. how many messages a day, roughly?
14:33  marta_k: idk maybe 300? 400 on drop days. we tried intercom
       last year, way too much for what we needed
14:34  marta_k: also our gorgias renewal is in september so

The extraction step should produce this and nothing else:

{
  "headcount_band":   { "value": "1-10",        "evidence": "4 of us all in one whatsapp",   "basis": "stated" },
  "channels_in_use":  { "value": ["whatsapp"],  "evidence": "all in one whatsapp",           "basis": "stated" },
  "volume_band":      { "value": "200-500/day", "evidence": "idk maybe 300? 400 on drops",   "basis": "lead_estimate" },
  "competitors":      { "value": ["intercom","gorgias"], "evidence": "we tried intercom",    "basis": "stated" },
  "deadline":         { "value": "2026-09",     "evidence": "gorgias renewal is in september","basis": "stated" },
  "industry":         { "value": "ecommerce",   "evidence": "6 stores / shopify tg group",   "basis": "inferred" },
  "budget":           { "value": null,          "evidence": null,                            "basis": "not_stated" },
  "authority":        { "value": null,          "evidence": null,                            "basis": "not_stated" }
}

Four rules make this work, and each one exists because of a specific way it fails without them.

Every field carries an evidence span. If the model cannot quote the text it got the value from, it does not get to assert the value. This is not for the audit trail, though it helps there. It is because requiring a quote is the cheapest hallucination brake available: a model that must point at a substring cannot invent a headcount.

"not_stated" is a first-class value and is not the same as a low value. This is the single largest source of silent error in every extraction pipeline we have seen. Ask a model for a budget number and it will produce a budget number, because that is what you asked for. Marta never mentioned money. "Budget: not_stated" and "Budget: small" are different facts about the world, and the second one is a fabrication that will be indistinguishable from data three joins downstream.

Basis is a category, not a float. Stated, lead_estimate, inferred, not_stated. You cannot trust a model's 0.87, per the calibration numbers above, but you can absolutely trust it to report whether the person literally said the thing, because that is checkable in one glance. A category a human can verify beats a probability nobody can.

The extractor never sees the score, and never sees how similar leads scored. Give it context about outcomes and you have built an anchoring machine that will report what it thinks you want.

Now the uncomfortable part: read that extraction again. It says headcount 1-10. Marta runs support for six stores with four people. She is plausibly an agency, which in most ecommerce tooling is a completely different and considerably better customer than a four-person store. The extractor was not wrong, exactly. It answered the question it was asked. The question was bad. This is what extraction errors look like in practice: not lies, but correct answers to a schema that failed to anticipate the shape of the customer.

Note also what happened to "we tried intercom last year, way too much for what we needed." A naive scorer files this as competitor mention, plus 20, problem-aware. It is also a fairly loud statement about price sensitivity. Both readings are correct. One number cannot carry both, which is the argument for keeping fields as fields on the contact record and only collapsing them at the last possible moment.

Fit scoring, and why your fit model is a portrait of your past

Fit is the boring half and the half people get wrong quietly. Ten rules, legible, defensible:

On the fields the extractor actually returned, Marta scores +20 (WhatsApp) +10 (volume) +8 (competitors named) +4 (headcount 1-10) = 42, against a practical maximum near 70. Add the agency rule the schema never thought to ask about and she is at 54. Twelve points is an entire routing tier. The schema decided that, not the model.

Two things about this table are worth being honest about.

First, it is short on purpose, and short is not the same as accurate. A gradient-boosted model over 400 features will out-predict these ten rules. It will also produce a 12 for a lead a rep can see is obviously good, and when the rep asks why, the honest answer is "feature 213 interacted with feature 88." At that point the rep stops reading the score. An untrusted score is worse than no score: it adds a step to the workflow and changes no behaviour. Legibility is not a compromise on accuracy. It is what buys the score the right to exist.

Second, and more seriously: every weight in that table was derived from customers who already bought. That is what fit is. It is a compressed description of your past. It is a lagging indicator by construction, and it will be most confidently wrong about exactly the customers you have not met yet. Build in an expiry: rebuild the weights from scratch annually rather than nudging them, and flag any rule whose evidence base is under about 20 closed deals as a hunch wearing a number. Most fit tables have three rules doing real work and seven that somebody argued for in a meeting in 2024.

Intent decays, and "last 30 days" is not decay

Intent signals have half-lives. A pricing question is worth a lot today, something today, and nothing next month. The standard implementation of this insight is a rolling window: count signals in the last 30 days. That is a cliff, and cliffs produce absurdities. A lead who asked about price 29 days ago and one who asked 31 days ago differ by the entire weight of the signal, for no reason that exists in the world. A lead who asked yesterday and one who asked 25 days ago score identically, which is worse.

Use exponential decay. One parameter per signal, and the parameter means something you can argue about at a whiteboard: how long until half of this is gone?

intent(t) = sum over signals of  w_i * 2 ^ ( -(t - t_i) / h_i )

Do not guess the half-lives. You can derive them from your own inbox without waiting for a pile of closed deals: for each signal, take everyone who emitted it and later engaged again at all, and find the median gap. If half the people who ask about price and eventually re-engage do so within four days, four days is your half-life. It is a proxy, it is not causal, and it is available today, which beats a better number you will never compute.

Watch the signs. A no-show at -10 with a 14 day half-life means that missing one call costs a lead roughly two weeks of standing. Ask out loud whether that is what you meant, because nobody ever does, and a surprising amount of pipeline rots in the shadow of a punishment weight somebody typed in once.

Behavioural signals only work if you actually collect them. Page views, UTM source, and which ad click preceded the DM are the difference between a working intent model and one that only knows what people typed. Live Visitors gives you page-by-page presence and ad-click attribution without cookies, which is the raw material for the top half of that table.

The same score means two different things on day 0 and day 21

Run the formula on a real timeline. Marta emits: pricing page view and a DM reply on day 0, a pricing question on day 1, adds her colleague on day 3, names the September renewal on day 4. Then nothing.

Day 4 is the peak at 102. Day 21 is 42. Here is the part that matters and that a single stored number destroys: on day 21, forty of those forty-two points come from two signals, and both are structural. A colleague is in the thread. A renewal is dated. Every fast, cheap, enthusiasm-flavoured signal has evaporated, and what is left is the skeleton of a real buying process.

Now compare her to a brand new lead who has just viewed pricing and replied to a DM. That lead scores 40. Marta scores 42. If your CRM stores one integer, those two leads are interchangeable, and they are nothing alike. One is a stranger with a mouse. The other has a committee and a date, and is waiting for someone to talk to her.

Store the components, not the sum. Then you can ask questions the sum cannot answer: how much of this score is structural versus reactive, what is the age of the newest signal, has this lead's score ever been higher than it is now. That last one is the single most useful field nobody has: peak intent and days since peak. A lead at 42 on the way up and a lead at 42 on the way down want different messages, and the way down is where a slow first response shows up as revenue you never see.

Rank. Do not reject. (The obvious answer is the wrong one.)

Every deck you have been shown proposes the same architecture: the AI qualifies inbound, routes the good ones to reps, and drops or nurtures the rest. It is the obvious design. It is also the one decision in this system you should refuse to automate, and the reason has nothing to do with being nice to leads.

Disqualification destroys the data you need to find out whether your qualifier works.

This is an old problem with a name. Credit scoring hit it decades ago and called it reject inference. You only ever observe repayment behaviour for applicants you approved. Your model is trained on approved applicants. Your model will be applied to everybody. The population you learn from is a sample that your own past decisions selected, and it is not the population you are scoring. David Hand and William Henley asked the question directly in the title of a 1993 paper, "Can reject inference ever work?", and their conclusion was chastening: the distribution of rejected applicants cannot help you infer their outcomes unless you are willing to make additional assumptions, and those assumptions are exactly the ones your data cannot test.

Thirty-three years later, somebody ran the experiment properly. Bruno Scarone and Ricardo Baeza-Yates published "The Illusion of Improvement: Reject Inference Strategies in Credit Scoring" in June 2026, evaluating the standard fixes across a natural retraining cycle. Their finding: "models whose accuracy improves while recall collapses create an illusion of improvement that leads practitioners to believe the system is getting better when, in fact, its rejection quality, the ability to correctly screen out defaulters, is deteriorating." Extrapolation, the strategy that looked best on standard metrics, was also the one that most badly distorted the training data: on one dataset and model pairing it dragged the training set default rate from the population's 22.0% up to roughly 27%. The authors' verdict on it is that extrapolation "does not mitigate survival bias; it reverses its sign." Their conclusion is the sentence to tape to your monitor: accuracy and rejection quality "give opposite recommendations on whether to explore," which confirms "that standard evaluation metrics are misleading under selection bias."

Translate it out of credit and into your pipeline. Every quarter, your qualifier's precision improves. The leads it calls good really do convert better than the leads it calls bad. Your dashboard is green. Your conversion rate on worked leads is up. And there is no number anywhere in that dashboard capable of telling you that recall on some segment has gone to zero, because you stopped generating the observations that would have shown it. The metric that is going up is the metric that goes up when the system gets worse in the specific way it is getting worse.

So invert the architecture. The score decides position in the queue, not membership in it. Every inbound lead is in the queue. Reps work top down. The model's job is ordering, which is a job it is genuinely good at and where being wrong costs you a delay rather than an outcome.

Here is the honest cost of that recommendation, because it has one: ranking saves less rep time than rejecting. A queue of 400 is still 400 items long, and at the bottom of it are people no human will reach today. The mitigation is a real automated first touch for everything below the line, which is a categorically different act from disqualification: it keeps the thread alive, it keeps the person capable of emitting intent signals, and it keeps producing outcome data on the segment your model is skeptical about. An AI agent answering a low-scored lead in ninety seconds is not the same product as a filter deleting them, even though both save the same rep hour. One of them preserves the experiment.

There is exactly one class of exception. Automate exclusions that are facts, never exclusions that are forecasts. "This account is a competitor's employee" is a fact. "This message is the same text posted into forty groups" is a fact. "This person asked us to stop contacting them" is a fact, and also a legal obligation. "The model thinks this one will not buy" is a forecast dressed as a fact, and forecasts do not get to remove people from your data. If you can check it by looking, automate it. If you can only check it by waiting, rank it.

The failure nobody talks about: your labels are your reps' opinions

Ask what your model is actually trained on. Not what you think it is trained on. The label is almost always some version of "did this record eventually become a closed deal," or worse, "did a rep mark this qualified." Both of those are records of your own team's behaviour. Neither is a measurement of the lead.

The definitive demonstration of what goes wrong here is not from sales. Ziad Obermeyer and colleagues published it in Science in 2019: a commercial risk-prediction algorithm applied to millions of Americans was systematically assigning Black patients lower risk scores than equally sick white patients. The algorithm was not broken. It was outstanding at its job. Its job, as specified, was to predict future health care costs, chosen as a convenient stand-in for health needs. Less money is spent on Black patients at the same level of illness, so the model correctly concluded they would cost less, and therefore incorrectly concluded they were healthier. Reformulating the label to predict illness rather than cost raised the share of Black patients identified for extra care from 17.7% to 46.5%. The model was fine. The label was the bug.

Your label has the same shape. It is a proxy for lead quality, and what it actually measures is a mixture of lead quality and how your team behaves. Reply speed. Which language the rep is comfortable in. Which timezone was awake. Whether the VP said "focus on enterprise" in January. All of that is baked into every outcome you are about to train on.

Here is the loop, with numbers. Suppose you receive 1,000 inbound DMs a quarter. Seven hundred arrive in English, three hundred do not. Your reps are English-first, so the non-English threads get parked until somebody who can handle them is free. Median first response: eight minutes for English, three hours and forty minutes for everything else.

Follow it through. In Q1 the segment converts at 3.0%, which is not a fact about the segment. It is a fact about three hours and forty minutes. You train on Q1. Language correlates with conversion, so the model down-ranks the segment, and the bottom of the queue gets a nurture cadence instead of a person. Response time goes from 3h40m to 26h. Conversion falls to 1.6%. You retrain on Q2, the model is now more confident the segment is bad, and it is right, because you made it true.

Meanwhile the reps freed from those threads spend more time on the English ones, whose conversion climbs from 6.0% to 7.0%. Total deals: 51, 53, 52, 51. Flat. Nobody investigates flat. The model's precision genuinely improved. This is the illusion of improvement, in a spreadsheet you would ship to your board.

And you cannot fix it by deleting the language feature, which is the first thing everybody tries. Language was never a feature. The model reconstructs the segment from message length, emoji density, the timezone of first contact, phrasing patterns, whether the company site has an English version. Amazon found this out the expensive way. In October 2018, Reuters reporter Jeffrey Dastin revealed that Amazon had scrapped an experimental recruiting model which, trained on a decade of submitted resumes, taught itself to penalise the word "women's" and to downgrade graduates of two all-women's colleges. Nobody had put gender in the feature set. It did not need to be there: the model rebuilt it out of the vocabulary. Amazon edited the offending terms to neutral and still killed the project, because once a model has learned to reconstruct a protected attribute from proxies, patching the proxies you found is not evidence about the ones you did not.

The practical rule is the one sentence in this post most worth stealing: you may exclude a variable from the model, but you must never exclude it from the audit. Removing the feature removes your ability to see the disparity. It does not remove the disparity. It just moves it somewhere you have agreed not to look.

"A human reviews every decision" is not oversight

This is the sentence every vendor offers and every buyer accepts. As normally implemented it is worth close to nothing, and there is research explaining why in two separate ways.

The first is automation bias: people over-accept machine recommendations, and the effect gets worse under cognitive load. Your reviewer has 400 leads, a score next to each one, and eleven minutes before standup. Guess what they do.

The second is sharper and less known. Rosenthal-von der P??tten and Sach ran an experiment with 260 participants, published in Frontiers in Psychology in 2024 under the title "Michael is better than Mehmet". Participants reviewed hiring recommendations from an algorithm that, in one condition, was deliberately biased against Turkish applicants. Only 41% of participants in that condition, 54 people, reported noticing the bias at all. Here is the sting: whether you noticed was not random. Participants carrying more negative emotions toward Turkish people were the ones who more often failed to see the discrimination in front of them. The reviewer least able to catch the bias is the reviewer who already agrees with it.

Sit with the implication for your qualifier. Your model learned its bias from your team's behaviour. Your reviewer is on your team. Their priors and the model's bias are the same object, arrived at twice. There is no correction available from a reviewer who shares the error, and "a human checked it" has bought you a signature, not a check.

What actually works is unglamorous and cheap.

The blind ordering is the whole thing. Show the score first and you have measured agreement with an anchor, which is a number that will look great and mean nothing. Show the thread first and the disagreements become the most valuable data you own: they are the only place your model and your humans are both forced to commit.

The last row of that table is the one people resist. A reviewer processing 400 items an hour is a rubber stamp on a payroll. Twenty an hour, carefully, on a stratified sample, produces information. Four hundred an hour produces a log file.

Auditing for drift: three different things wearing one word

"Drift" gets used for three unrelated failures with wildly different detectability, and conflating them is how teams end up monitoring the easy one forever.

Data drift is the input distribution moving. A new ad campaign brings a different population, or your tracking breaks and a field goes null. Easy to detect, no outcomes required. The standard tool is the population stability index, borrowed from credit risk: compute the distribution of each input now against the distribution at training time. The conventional thresholds are 0.1 for attention and 0.25 for investigate. Compute it weekly, it costs nothing, and it will catch the boring disasters that account for most incidents.

Concept drift is the relationship between inputs and outcomes moving. You shipped a self-serve onboarding flow and now small accounts succeed where they used to churn. Detectable, but only with outcomes, which means you learn about it one sales cycle late.

Feedback drift is the model's own decisions reshaping the population it is later evaluated on. This is the loop from the previous section. It is not detectable from your data at any cadence, with any statistic, ever. Not because the tooling is immature. Because the observations do not exist. Every dashboard is blind to it by construction, and it is the one that eats your best segment.

There is exactly one instrument that sees feedback drift, and it is the one from the reject inference literature: deliberately act against your own model, at a small rate, on purpose. Scarone and Baeza-Yates propose controlled exploration at 2% to 5%, and report that it surfaces the feedback loop at close to zero cost.

Do the arithmetic on the example above. Three hundred non-English leads a quarter, hold out 5%: fifteen leads, routed to a human regardless of score, answered in eight minutes like anyone else. After two quarters you have thirty leads whose outcomes were generated under the good treatment. Say two of them convert, a 6.7% rate. Your model's implied rate for that population is 1.1%, which predicts 0.33 conversions across thirty leads. Under a Poisson with a mean of 0.33, seeing two or more has a probability of about 4%.

Be honest about what that is. Thirty leads is not a study, 4% is not significance after you have run twelve of these, and you should not put it in a board deck. It is a fire alarm. A fire alarm is exactly what you needed and exactly what no amount of dashboard could have given you, and the price was fifteen leads a quarter of rep attention.

Two of those rows are worth arguing about. Use a calibration curve rather than AUC: AUC tells you the model ranks well, which you already believed, while calibration tells you whether a 70 means seventy percent of anything. And that last row is the cheapest alarm in the building. Count how many leads from each segment reached a human this month versus last month. If any segment's human-touch count has fallen for three consecutive months, you have a loop. It is one query. Almost nobody runs it, and it would have caught the Q2 to Q4 table above by the end of Q2.

The legal layer that arrives sooner than you think

Most teams conclude that lead scoring sits comfortably outside GDPR Article 22, which restricts decisions "based solely on automated processing" that produce legal effects or similarly significantly affect someone. Being ranked 200th in a sales queue is not a mortgage refusal, and that reasoning is probably right for most B2B qualification today.

Probably. Read the SCHUFA judgment before you rely on it. In Case C-634/21, decided 7 December 2023, the Court of Justice of the EU held that the automated establishment of a probability value by a credit agency is itself automated individual decision-making under Article 22(1), where a third party draws strongly on that value to establish or terminate a contractual relationship. The agency never decided anything. A human at the bank did. That did not matter, because the score played a determining role.

The reasoning is about the score's function, not the scorer's industry. Now put it next to the automation bias research from two sections ago. If your qualifier produces a number, and a human downstream approves it 99% of the time because they are reviewing 400 an hour, then who made the decision? The SCHUFA logic and the automation bias literature are pointing at the same fact from opposite ends: nominal human involvement is not meaningful human involvement. Whether that becomes a legal problem for sales qualification specifically is unsettled, and this is not legal advice. But if your defence against Article 22 is "a human clicks approve," you should want that defence to be true, and the research says it usually is not.

The nearer deadline is not about scoring at all. EU AI Act Article 50 applies from 2 August 2026. It requires providers to ensure that AI systems intended to interact directly with natural persons inform those persons that they are interacting with an AI system, unless that is obvious to a reasonably well-informed, observant and circumspect person. If your qualification design has an agent holding a conversation to collect fields, that is a system interacting directly with a natural person, and the "obvious" carve-out is doing far less work than people hope: an agent good enough to qualify is by definition an agent people might not clock. Our post on outreach compliance in 2026 goes through the rest of the stack, including the layer that binds you regardless of what the law permits.

Disclosure is not the cost people fear. It is a filter. A person who knows they are talking to an agent and keeps typing has just given you an intent signal worth more than anything you extracted from the conversation.

What we built for this, and where we are the wrong tool

CRM Solid is built for teams whose leads arrive as messages, so the pieces map onto the architecture above fairly directly, and it is worth being specific about which pieces exist and which ones we chose not to build.

The fields layer is the contact record: tags, custom fields you define, a lead score, team assignment, and a timeline of every message across every channel. The behavioural half comes from Live Visitors, which gives cookieless page-by-page presence, UTM and ad-click attribution, and alerts to Telegram or email when a known contact is on your pricing page right now. That is the raw material for the intent table earlier in this post, and without something like it your intent model only knows what people typed.

Ranking rather than rejecting is what custom pipelines and channel-to-pipeline routing are for: an inbound Telegram lead can land on a different board than an Instagram one, and every lead lands somewhere. For the bottom of the queue, AI Agents are real and shipped: personas, knowledge bases, a rules engine, rate limits, per-contact pause, and human handoff, working across Telegram, X, email, and the social inbox. That is the "automated first touch that is not disqualification" from earlier, and the handoff and pause controls exist precisely because an agent holding a conversation it should not be holding is the expensive failure. If you want the category distinctions straight before you deploy one, we wrote up the difference between a chatbot and an agent honestly.

Outcomes close the loop through Deals: a six-stage pipeline with deal value and win probability, and a won deal posts income into the ledger automatically, so the label you eventually train on is anchored to money rather than to a rep's mood.

Now the part that matters more. We do not ship a learned propensity model, on purpose. There is no opaque number deciding your leads. The scoring is fields you define and rules you write, which is exactly the legibility argument made earlier, and it comes with a real cost: a well-built gradient boosted model trained on your own warehouse will out-predict a rules table, sometimes by a lot. If you have 50,000 leads a month and a data team, build that model. Then push the score in through the public REST API or the MCP server and let it drive routing here. That is a supported design and it is the right one at that scale. We are the wrong choice for a team that wants a black box to be smart on their behalf, and a reasonable choice for a team that needs to explain a number to a rep who disagrees with it.

Three more honest limits, since this post spent two long sections on an extraction layer. We do not ship that extractor. The custom fields are yours to define, and populating them from a messy thread is a human's job or your own model's job pushed in over the API. If you came here wanting to point our product at your inbox and receive Marta's JSON, that is not a thing you can buy from us today. Second, the in-chat feedback on AI Agents, the thumbs up and thumbs down that teaches the agent, adjusts how the agent writes. It is not a trained scoring model and it does not qualify anyone. And third, on the email inbox: connecting, syncing, reading, composing, replying, linking a thread to a contact and setting per-thread status all work, but AI analysis and AI drafting on email are not built yet. If your qualification design depends on a model reading your email and scoring it, we do not do that today, and we would rather you knew now. There is a free plan if you want to test the parts that do exist against your own inbox before deciding: see plans.

Common questions

What is AI lead qualification?

It is the use of a model to turn raw inbound conversations and behaviour into structured fields, and then into a priority. The useful version has two halves: a language model extracting facts from messy text with the evidence attached, and a formula you can read turning those facts into an ordering. The version where a model outputs a single opaque number is the version that fails quietly.

Is BANT still useful in 2026?

As a rep's mental checklist on a live discovery call, sometimes. As an automation target, no. BANT assumes a single Authority who can say yes, and Gartner's 2025 survey puts B2B buying groups at five to 16 people across as many as four functions, with 74% of them showing unhealthy conflict. It also assumes you can ask four direct questions, which in a DM at message three is how you get blocked rather than qualified.

Can an LLM score leads accurately?

It can extract and classify accurately. It cannot judge reliably. The largest evaluation of LLM-as-a-Judge to date, covering 21 judges and roughly 541,000 judgments, found that chance-corrected agreement was 33 to 41 points lower than the raw agreement everyone validates on, and that two production judges combined test-retest reliability above 0.95 with serious position bias. A consistent score is not a correct one.

Should AI ever disqualify a lead automatically?

Only on facts, never on forecasts. Automate exclusions you can verify by looking: an opt-out request, a spam broadcast, a competitor's employee. Do not automate exclusions that are predictions, because every lead your model removes is an observation you will never get, and after four quarters of that your evaluation data is a sample your own model selected. Rank instead. Position, not membership.

How often should a lead scoring model be retrained?

Retraining cadence is the wrong question, and it is the question the reject inference research says will mislead you: a natural retraining cycle is precisely how accuracy climbs while recall collapses. Fix the exploration budget first. Hold 5% of leads out of the model's control permanently, so that every retrain has fresh observations from the population your model dislikes. Then retrain quarterly.

Does GDPR apply to automated lead scoring?

Article 22 covers decisions based solely on automated processing with legal or similarly significant effects, and most B2B queue ranking probably falls short of that bar. But the CJEU's SCHUFA ruling in December 2023 held that a score is itself an automated decision when a third party draws strongly on it, even though a human formally decided. If your human approves 99% of what the model says, ask honestly who decided. This is not legal advice.

Where to start on Monday

Not with the model. Run one query first: for each segment you can name, count how many leads reached a human in each of the last six months. If any segment's count has fallen every month, you already have a loop, and you have it right now, before any AI touched anything. It cost you nothing to find and it will cost you a quarter to fix.

Then pick your fifteen. Choose the segment your team quietly believes is not worth the time, route 5% of it to a human regardless of any score, answer them in eight minutes, and write down what happens. That is the entire method. Everything else in this post is an elaboration of the habit of occasionally disobeying your own model.

When you do want the mechanics of scoring, fields, and routing built on conversations rather than form fills, our guide to contact lead scoring walks through the setup.