Trademark similarity search: 91% recall, clear or review?

This guide shows how to judge a claimed 91% recall for AI trademark similarity search.

It requires independent validation and a phonetic check before any 'clear' decision.

What '91% recall' actually measures

For the purpose of this decision, recall is the fraction of known-conflict marks retrieved: (true conflicts retrieved) ÷ (all true conflicts in your labeled sample). This is the definition that makes recall — not precision — the metric that governs the clear-or-review decision. A system can hit a claimed 91% recall while flooding you with false positives, which is acceptable for clearance screening, but fatal if you read that figure as overall accuracy.

That percentage is not a fixed trait of the tool. Recall is measured against a ground-truth set you define, typically marks that drew refusals or oppositions on similarity grounds. As intrinsic similarity benchmarks show, evaluation results are tied directly to the benchmark set used (mbrenndoerfer.com). Because of this, the same tool can report 91% on one sample and far less on another: the number is a property of the test set, not the tool.

With that in mind, the real question is not how many results the search returns, but how many of the conflicts you already know about it failed to retrieve. A high recall figure can justify expanding review, but it cannot justify calling a mark clear on its own. Until you know that retrieval rate against your own known conflicts, any vendor recall claim must remain audit input, not proof of a clear result.

How similarity search engines rank marks

Vector similarity search turns each mark into an embedding and ranks candidates by distance in that vector space. The engine does not compare strings directly; it retrieves approximate nearest neighbors, so a mark can be ranked as “similar” only if the index actually reaches it. That makes the index’s recall behavior part of the evidence, not an implementation detail.

Neon’s pg_embedding extension illustrates the tradeoff in measurable terms: on the GIST-960 benchmark (1 million vectors of 960 dimensions, 1,000 test queries), its HNSW index is reported as 20 times faster than pgvector’s IVFFlat at 99% accuracy, on a Neon instance with 4 vCPUs and 16 GB RAM. The point is not the headline alone; it is that an approximate index can quantify a small accuracy loss in exchange for a large speed gain. Whether that loss is acceptable depends on your own conflict set, not on the benchmark dataset.

HNSW builds a navigable graph. Its parameters directly set the recall–speed exchange: m sets how many connections each node keeps, ef_construction sets how thoroughly the graph is built, and ef_search sets how many candidates are explored at query time. Neon’s examples show two very different settings:

Raising ef_search (and often m) generally improves recall while increasing latency and memory; lowering them does the reverse. IVFFlat, by contrast, partitions vectors into clusters and searches only the clusters it probes, so its recall depends on that probe count as well as on cluster fit.

Because these settings are tunable, two systems advertising “similarity search” can return different neighbor sets for the same mark. A ranking is trustworthy at your recall target only when the index parameters and the embedding model are held constant and the missed conflicts are measured. If your labeled sample shows conflicts falling outside the retrieved set, increase ef_search or m, revisit the embedding model, or move to an exact search for the final pass; do not treat the vendor’s benchmark as proof that your conflicts were retrieved.

An index benchmark on GIST-960 is not a trademark benchmark. The dataset, dimensionality, distance metric, and mark-normalization rules all affect whether a 99% accuracy figure transfers. Use the benchmark to understand the mechanism and the magnitude of the tradeoff, then require your own measurement before relying on the ranking.

Validate the recall claim yourself

This section alone sets out the labeled-sample validation protocol: the exact procedure to reproduce or refute a vendor's 91% recall figure on your own mark classes. Treat that figure as audit input, not as a clearance rule, and complete the test before any "clear" decision.

Build the sample from your own Nice class. Pull at least 50 marks from USPTO refusal notices—TTAB or examiner refusals citing §2(d) similarity—as known conflicts, plus at least 50 marks that cleared without conflict. Keep the source, class, and filing date for each mark so the sample mirrors your actual filings. Run your search tool on every mark with the same cutoff and record whether each known conflict appears in the top-k results.

Score both error types, not just recall. Count missed conflicts to test recall, and count cleared marks flagged as conflicts to measure the false-positive rate. A tool reporting 91% recall but a 40% false-positive rate still sends nearly half of your clean filings to review, which erases the speed benefit even when the headline number looks strong.

Cross-check the same sample with a phonetic encoding check. Compare its conflict flags with the vector search results; where the two disagree, inspect the mark pair manually and record which method caught the conflict. This gives you a second, independent signal instead of relying on one vendor ranking.

Document the sample, queries, cutoff, and counts, then rerun after any model, index, or data update. A "clear" decision is justified only when your own test reproduces recall at or above 90% with false positives under 20% on at least 50 known-conflict marks; otherwise route the mark to review.

Rule of thumb: if you have not run your own 50-mark labeled test, no vendor recall number — 91% or otherwise — upgrades any screen result from "review" to "clear." Run the test once, and every subsequent clearance inherits the evidence.

Cost of being wrong in each direction

The whole clear-or-review decision turns on which mistake you can afford, and the two mistakes are not remotely symmetric. A false clear means the AI scored a mark low-risk, you filed, and a conflict you missed surfaces later as an examiner refusal or a third-party opposition. That is the worst outcome: your filing fees are already sunk, the mark may be abandoned after months of prosecution, and you may have invested in packaging or marketing in the meantime. By contrast, a false review means the AI flagged a clean mark for escalation and you spend one professional search on it. Annoying, but fully recoverable — you confirm the mark is clear and move on.

The false-clear risk is the one you cannot inspect in advance: a mark the AI rates low-risk draws a conflict only after you have filed. That is exactly the error profile you expect from an unverified vendor metric, and it is why a vendor recall claim can never be the deciding input on its own. Before relying on any vendor's low-risk ratings, ask how many rated-safe marks in their test data later drew refusals or oppositions — if they cannot answer, treat every low-risk rating as a review trigger.

The other side of the ledger is your false-review cost: the extra professional search a flagged clean mark requires. That cost is real, but bounded and predictable — price it once with your search firm and you know exactly what each unnecessary flag costs you.

Because the costs are asymmetric, the decision rule set out earlier follows directly: hold the screen to the validated-recall bar and accept a higher rate of false positives. Wasting a few extra professional searches on clean marks is the cheap error; you deliberately buy margin against the expensive one.

Practical check before you rely on any rating: compute what one additional flagged clean mark costs you in professional-search hours, and compare it against the cost of a single opposition after filing — legal response work plus potential rebranding. If the ratio is lopsided in the way the table above suggests, the threshold-setting logic writes itself, and the vendor's 91% claim becomes an audit input to verify rather than a number to trust.

What the evidence does not establish

Start with the uncomfortable fact: no source in the available sources literature reports a 91% recall figure for trademark similarity search. Neon's pg_embedding benchmark reports 20x speed at 99% accuracy, but that result is on the GIST-960 Euclidean dataset — one million image vectors of 960 dimensions, with a test set of 1,000 queries. That is not a corpus of word marks, and image-vector retrieval is not a §2(d) likelihood-of-confusion task. The embedding-model comparison literature (MTEB-style benchmarks across OpenAI, Voyage, Cohere, and Gemini models; the AIMultiple 25-model retrieval benchmark) measures general text retrieval, not phonetic or visual similarity between trademarks. So the 91% figure arrives as a vendor claim with no published trademark-specific test behind it. Treat it as a hypothesis to verify, not a fact to rely on.

The domain mismatch matters more than the accuracy number. GIST-960 measures whether an approximate nearest neighbor index returns the true nearest neighbors in Euclidean space — a pure geometry test. Trademark clearance asks a different question: does an embedding space built for general text meaning preserve the confusability judgments a trademark examiner would make? The embedding-evaluation literature is blunt on this point: similarity benchmarks are a standard evaluation tool, but they provide little insight into the nature of the relationships the representations actually encode. A benchmark that validates "nearest neighbor in vector space" does not validate "nearest conflict in law."

The check is mechanical: rerun the tool's own benchmark methodology on your labeled sample of refused-and-opposed marks before trusting any accuracy figure the vendor publishes. If the vendor will not tell you what dataset produced their number, that silence is itself evidence.

Now map where the validation rule breaks. First, sparse labeled samples: if your sample of known-conflict marks is small, a single missed conflict swings recall by a large step, so your confidence interval is wide even if the point estimate clears the bar. Second, conflict-type skew: if your labeled sample is dominated by exact or near-exact matches, recall will look strong while the engine silently fails on the phonetic-transposition and design-mark cases that actually generate refusals — so stratify the sample by conflict type and test each stratum. Third, drift: a vendor can swap the underlying embedding model or index parameters without notice, invalidating your earlier validation, so re-run the labeled sample whenever the vendor announces a model change.

Fourth, the boundary cases where a passing validation still should not produce a "clear": marks in classes or channels your labeled sample does not represent, and any mark where the phonetic encoding check disagrees with the vector ranking. In those cases the rule's output is "review," not "clear" — the validation confirms the engine's average behavior, not its behavior on the specific mark in front of you.

The practical rule: the vendor's 91% is audit input — a number that tells you the validation is worth running. It becomes decision-grade only when your own labeled-sample test reproduces it on your conflict types, in your classes, on the vendor's current model version.

Worked Example: Run the Numbers

Here is the whole audit worked through with real numbers, so you can see exactly where a decision falls out. Scenario: you are clearing a word mark for a consumer product launch, filing intent-to-use, and your search vendor advertises 91% recall. On a fixed audit date — the day before you would otherwise sign off — you pull a labeled sample of 50 known-conflict marks drawn from refused and opposed registrations in your class, run each one through the engine, and record what comes back. The engine returns 44 of those 50 marks in its candidate lists. That single division drives everything: 44 ÷ 50 = 0.88, or 88% reproduced recall.

Now check the second gate. Across the same runs, the engine surfaces 200 total candidates, of which 18 are not on your labeled conflict list — noise, but noise you would have to manually screen. Measure the false-positive rate the way the rule defines it: how many of the 50 cleared marks in your sample were flagged as conflicts. If more than 10 of those 50 clean marks come back flagged (the 20% ceiling the validation rule allows), the screen fails on precision load: the tool "finds" conflicts, but it buries you in candidates that would make every future clear a manual slog. The structure is what matters — recall and false positives are computed separately, and both must pass.

The verdict for this example: do not clear. The reproduced recall of 88% falls short of the 90% floor, and the break-even trigger is explicit — the engine would have needed to retrieve at least 45 of your 50 labeled conflicts while keeping non-conflicts under 10 of the candidates you screen. It did neither, so the vendor's 91% claim is, for your mark class and your sample, unreplicated. File only after a review workup, or after the vendor's index and settings change enough to justify re-running the audit.

Two habits make this example reusable. First, freeze your labeled sample and audit date before you run anything — if you pull conflicts after seeing the engine's output, you have contaminated the test. Second, log the retrieval count, not just the percentage; a future audit can compare 44-of-50 against a new engine version directly, which a bare "88%" cannot. The benchmark literature on embedding and similarity-search evaluation, including Neon's HNSW-versus-IVFFlat comparisons, makes the same point from the engineering side: headline accuracy figures are environment-specific, and only a reproduction on your own data tells you what you actually bought.

Decision rules: clear or review

To operationalize a strict clear-or-review line, apply the following five if/then rules without relying on the vendor's aggregate percentage alone. Rule : If your labeled-sample test of at least 50 refused-and-opposed marks reproduces recall ≥90% with false positives ≤20%, then treat the AI screen as trustworthy for first-pass clearance; if not, then treat every AI result as review-triggering. This is the only path that can convert an unverified claim into a usable audit result for your file.

Rule : If Double Metaphone or NYSIIS encoding of your mark matches any top-20 vector hit, then route to professional review regardless of the AI risk score. Phonetic collisions are a documented failure mode for both manual and AI-only searches, so this override must stand even when the screen returns "clear".

Rule : If the AI screen is clear and your phonetic check shows no match in the top-20, but your mark shares a dominant element with a known refusal history in the same class from your labeled sample, then do not auto-clear — require a targeted review of that element. The presence of that element, not the overall "low risk" label, is what must trigger review in that narrow case.

Rule : If the tool cannot verifiably state the index approach it uses for retrieval, then do not downgrade any hit based on rank position alone. As noted by Neon in its comparison of approximate nearest neighbor indexing, the accuracy-speed exchange can shift with the index, so rank alone is not proof of safety when the retrieval method is unknown to you.

Rule : If any single hit is a refused or opposed mark in a closely related class and would be a conflict under the facts of your mark, then override a "clear" output and route to professional review, even if all prior rules appear satisfied. No aggregate score can erase a direct, fact-based conflict found by your own checks.

What to do next

Frequently Asked Questions

Can I rely on the vendor's 91% figure alone to mark a trademark as clear?

No, it is an unverified vendor metric that cannot alone justify 'clear'; you must validate it on your own labeled sample of at least 50 known-conflict marks.

What kinds of marks should I use to build my labeled validation sample?

Base the recall test on refused-and-opposed marks, and file only when your own recall test on that labeled sample confirms the vendor metric.

Why is recall rather than precision the metric that governs the clear-or-review decision?

Recall is the fraction of known-conflict marks retrieved — (true conflicts retrieved) ÷ (all true conflicts in your labeled sample) — which is why it governs the decision.

Is a high false-positive rate acceptable if recall is strong?

Flooding you with false positives is acceptable for clearance screening, but it is fatal if you read the recall figure as overall accuracy.

Do I need any additional check beyond hitting the validation thresholds before clearing a mark?

Yes, Reader rule condition (b) requires a phonetic encoding check in addition to meeting the validation thresholds.

Is the 91% recall figure stable across different searches?

No, that percentage is not a fixed figure, which is part of why it must be treated as audit input rather than clearance evidence.

Quick answers

Also worth reading: Trademark clearance search: 38% fewer misses vs text-only in 2026: Trademark clearance search: 38% fewer · Trademark clearance search: 18% face opposition despite AI low-risk in 2026: Trademark clearance search: 18% face · How artificial intelligence is transforming the future of trademark law and brand protection: How artificial intelligence is transforming