Index Publication Policy: Independence, Corrections, Re-checks

This page is the reply to every dispute about the AI Shortlist Index. It says who pays for what, what a published edition is, what we will change after publication and what we will not, and how a vendor gets a re-check. The methodology page covers how the numbers are made. This page covers how they are governed.

Independence

WebPossible publishes the Index. WebPossible also runs Shortlist, a practice that sells work to property-ops software vendors, and WebPossible Media, which operates disclosed comparison sites. The same person runs all three. That is a conflict of interest on its face, so the rule that governs it is written down and repeated on every Index page:

“Paying WebPossible never affects Index placement. The Index measures what AI engines say. Shortlist clients buy the work that earns a position, not the position.”

In practice this means four things. No vendor can pay to be added, removed, moved, or re-described in a ranking table. Shortlist clients are measured by the same prompts, the same runs and the same extraction code as everyone else, and their rows carry no marker other than the data. A vendor that becomes a client after an edition is not re-run, re-scored or annotated in that edition. And the person selling the practice does not hand-edit any figure; every published number traces to a raw run file that is written once and never modified.

Where a Shortlist client moves in a later edition, that movement is reported the same way any vendor’s movement is reported. We may use it in a case study, with the client’s consent, and the case study will link to the edition so the reader can check it.

How an edition is published and frozen

An edition is a month. Runs are captured on scheduled days, extraction is run against the raw files, unknown vendor names go through a human review queue, and the accuracy gate on the methodology page must pass before any metric is exported. Then the edition is published at its own address, listed in the archive, and frozen.

Frozen means the ranking tables, source maps and vendor rows for that month do not change afterward. We do not re-run a prompt because a vendor asked, we do not re-score an answer because the extraction looked ungenerous, and we do not quietly fix a number and hope no one kept a screenshot. If something in a frozen edition is wrong, it is handled as a correction, below, with the original preserved.

The prompt set is frozen per edition too. A prompt that changes wording gets a new id and a new prompt set version; the old one stays in the library marked retired. Every change that can move a number is dated in the Index changelog.

What counts as a correction

A correction is a change to something we published that was wrong at the time we published it. Three kinds qualify.

Entity errors. A vendor’s canonical name, an alias, its website, or its category membership was wrong, so the extraction matched or missed mentions it should not have. These are the most common and the easiest to verify against a source URL.

Extraction errors. The code counted a common-word name (“Latch”, “Meld”, “Level”) where the answer used the word in its ordinary sense, or it missed a nickname that should have been an alias. The accuracy gate in the methodology is meant to catch most of these before publication; the ones that get through are corrected.

Process errors. A run that should have been marked degraded was counted, a seeded mention was not excluded, or a source was classified under the wrong page type.

A correction is not an engine giving a different answer this week than it gave on the run date, a vendor believing it deserves a higher position, or a vendor disliking the framing an engine used. Those are the measurement, and the measurement stands.

How a correction is issued

Anyone can report one through the contact page. Send the edition, the category, the row or source in question, and a URL that shows the correct fact. We verify against that URL, not against the report.

When a correction is accepted, the frozen edition is amended. The original figure stays visible with a dated note explaining what changed and why, the corrected figure appears next to it, and the change is logged in the changelog. Extraction corrections that affect metrics are re-run from the raw files with the fixed rule, and the amended metrics are labeled as amended. We do not overwrite. When a correction is declined, we say so on the record, with the data, in the same place.

Re-checks

Any vendor in a category may request one re-run of that category’s full prompt set per edition, using the “Request a re-check” link on its row. The re-check follows the same protocol as the edition: the same 25 prompts, three runs each, spread over separate days, on every engine that is not degraded. Nothing about the prompts or the extraction is adjusted for the requester.

The result is published regardless of outcome, alongside the original, labeled as a re-check with its own run dates. If the vendor appears more often on the re-check, the page shows it. If it appears less often, the page shows that too. The original edition figures are not replaced; both stand, because the difference between them is itself information about how stable the engines are on those prompts.

A re-check is not a dispute channel for the extraction rules or the prompt set. Those go through corrections and the changelog. One re-check per vendor per edition, and a vendor can request it whether or not it was named.

Claiming a row

Every vendor row carries a “Claim your row” button. Claiming requires a name, a title, and a work email address on the vendor’s own domain; we verify the domain by email before anything changes. Personal addresses and agency addresses do not qualify.

A claimed vendor may add a section to its profile labeled “vendor-supplied,” covering its own description, integrations and pricing model. That section is marked as the vendor’s words rather than ours, and it sits apart from the measured data. Claiming a row never changes a mention rate, a position score, a consistency figure or a rank, and it never moves a row in a table. A claimed row is measured exactly like an unclaimed one.

Badges

A vendor named in an edition may use the matching badge at no cost, in the Named, Top 3 or #1 variant it qualifies for. The badge must link to the vendor’s row, must show only the current or most recent edition, and may not be altered. A badge used to imply an endorsement the data does not support can be revoked. The full terms and the files are at Index badges.

Disclosure of WebPossible properties

WebPossible operates webpossible.com and, through WebPossible Media, a set of comparison sites in property-ops software categories. Any of these can turn up among the sources an engine draws on, and when one does, it appears in the category Source Map like any other domain, flagged “own network” and disclosed inline on the page. We do not remove our own properties from a Source Map and we do not promote them within it. Each network site carries an operated-by line and a sponsorship policy that cross-references this page.

Data reuse and attribution

You may quote, cite and link to any Index page. Attribute it as “AI Shortlist Index, WebPossible, [month] edition” with a link to the page you drew from. The prompt library is cited as “AI Shortlist Index Prompt Set v1.x, WebPossible.” You may not republish full ranking tables, present Index figures as your own research, or strip the disclosure from a Source Map when you reproduce part of it. Vendor names are used nominatively; logos appear only on claimed rows with permission.

Contact

Corrections, re-check questions, claim problems and disputes all go through the contact page. We answer on the record, with data, and the answer is added to the relevant page when it is of general interest.