Transparency Index
Vendor claims, checked against sources the vendors don’t control
Every directory in this market repeats what vendors say about themselves. This index does the other thing: it takes individual claims — pricing, revenue, founding, workforce — and checks each one against a primary source: a regulator filing, a public registry, the vendor’s own published terms, or named reporting. Each claim carries a dated stamp. A stamp the vendor cannot edit, pulled from a source they don’t control, refreshed on a published cadence.
How to read the stamps
- Verified A primary source the vendor does not control confirms the claim as stated. The source is linked and the check is dated.
- Unverified The claim exists — or the fact is absent — and no primary source confirms or refutes it. The absence we found is documented next to the stamp. Unverified is not an accusation; it is a statement about the evidence.
- Contradicts source A primary source states materially different values. Both texts are shown side by side, each linked. Nothing else is added — no adjectives, no motive.
Stamps are per claim, never per vendor, and the roll-ups below are arithmetic, not grades. Stamps expire: pricing and certification checks are re-checked within 90 days, all other checks within 180 days; past its review date a stamp reads “re-check due”, never silently green. The full rules are the methodology.
The ranking — share of checked claims that verify
| # | Vendor | Claims checked | Verified | Unverified | Contradicts source | Share verified | Last checked |
|---|---|---|---|---|---|---|---|
| 1 | Appen | 5 | 4 | 0 | 1 | 80% | Nov 8, 2026 |
| 2 | Prolific | 4 | 3 | 1 | 0 | 75% | Nov 8, 2026 |
| 3 | Scale AI | 4 | 2 | 2 | 0 | 50% | Nov 8, 2026 |
| 4 | Labelbox | 3 | 1 | 2 | 0 | 33% | Nov 8, 2026 |
| 5 | Surge AI | 3 | 1 | 2 | 0 | 33% | Nov 8, 2026 |
A vendor with fewer checked claims is not ranked below one with more; the ranking orders only the vendors checked so far, by the share of their checked claims that verify. The 26 vendors not yet checked are listed below as exactly that — not yet checked, with the wave order published in advance.
Full claim detail — wave 1
Appen — claims & checks
Each claim below was checked against a primary source the vendor does not control — a regulator filing, a public registry, the vendor’s own published terms, or named reporting. Stamps are per claim, never per vendor, and every stamp is dated. Roll-up: 4 of 5 checked claims Verified · 1 Contradicts source · last checked Nov 8, 2026. How stamps work: methodology · Transparency Index.
| Claim | Stamp | Source |
|---|---|---|
| Revenue Reported roughly $231 million in revenue for the twelve months to December 2025. Source records“Full-year operating revenue for 2025 was $230.8 million.” The Appendix 4E reports “revenue and other income” of $233.447 million for the same period; the directory’s figure matches the operating revenue line. | Verified | Appen 2025 Annual Report (ASX, 25 Feb 2026) |
| Founded 1996 Source records“Founded in 1996, Appen has customers and offices globally.” | Verified | Appen 2025 Annual Report (ASX, 25 Feb 2026) |
| Headquarters Sydney, Australia Source recordsRegistered office on the report cover: “Appen Limited, 9 Help Street, Chatswood, NSW 2067, Australia — ACN 138 878 298.” Chatswood is a Sydney suburb. | Verified | Appen 2025 Annual Report (ASX, 25 Feb 2026) |
| Crowd size Vendor statesA crowd of more than one million contributors across 170+ countries and 235+ languages (Appen website, as recorded in this directory). Source recordsThe 2025 Annual Report describes “a global crowd of more than 1 million skilled contractors who speak over 500 languages, in over 200 countries”, footnoted “Self-reported” and “Self-reported, includes territories”. | Contradicts source | Appen 2025 Annual Report (ASX, 25 Feb 2026) Both figures are published by Appen. The website figures recorded in this directory (170+ countries, 235+ languages) and the filed annual report figures (over 200 countries, over 500 languages) do not match. Crowd counts are company-reported in both places; no external registry counts them. |
| Pricing Not published — project-based quotes. Source recordsAppen’s sitemap lists no pricing page and the site publishes no rates; pricing is quoted per project. | Verified | Appen — appen.com sitemap Absence check: full sitemap and site navigation read on the check date. |
Right of reply: Appen — and any reader — can dispute a check with a source at [email protected]. Corrections are fixed, dated and logged publicly in the correction log.
Prolific — claims & checks
Each claim below was checked against a primary source the vendor does not control — a regulator filing, a public registry, the vendor’s own published terms, or named reporting. Stamps are per claim, never per vendor, and every stamp is dated. Roll-up: 3 of 4 checked claims Verified · 1 Unverified · last checked Nov 8, 2026. How stamps work: methodology · Transparency Index.
| Claim | Stamp | Source |
|---|---|---|
| Pricing Published — you set participant pay (min $8/hr, $12/hr recommended) + 42.8% corporate platform fee. Source records“Typically, we recommend you pay participants at least £9.00 / $12.00 per hour, while the minimum pay allowed is £6.00 / $8.00 per hour… our platform fee is usually 42.8% for corporate customers, and a discounted 33.3% for academic or non-profit customers.” | Verified | Prolific — pricing page The vendor’s page hedges the fee with “usually” and quotes pay floors in GBP and USD; the directory’s figures match the published rates. |
| Founded 2014 Source recordsPROLIFIC ACADEMIC LTD, company number 08991598, incorporated 10 April 2014. | Verified | UK Companies House — PROLIFIC ACADEMIC LTD (08991598) |
| Headquarters United Kingdom (Oxford origins) Source recordsRegistered office on the Companies House record: 483 Green Lanes, London, N13 4BS, England. | Verified | UK Companies House — PROLIFIC ACADEMIC LTD (08991598) The registry record is a London registered office; the company was founded at Oxford University, as the directory records. |
| Participant pool 300,000+ active verified participants; 1M+ on the waitlist. | Unverified | Prolific — pricing page The counts are located on Prolific’s own pricing page (“300,000+ active participants”; “over a million on the waitlist”). No external registry counts a platform’s participant pool; pool figures are company-reported by construction. |
Right of reply: Prolific — and any reader — can dispute a check with a source at [email protected]. Corrections are fixed, dated and logged publicly in the correction log.
Scale AI — claims & checks
Each claim below was checked against a primary source the vendor does not control — a regulator filing, a public registry, the vendor’s own published terms, or named reporting. Stamps are per claim, never per vendor, and every stamp is dated. Roll-up: 2 of 4 checked claims Verified · 2 Unverified · last checked Nov 8, 2026. How stamps work: methodology · Transparency Index.
| Claim | Stamp | Source |
|---|---|---|
| Pricing Not published — enterprise contracts, sales-led. Source recordsThe /pricing URL listed in Scale's sitemap resolves to a page titled “Meet with our team”. It publishes no rates, tiers or unit prices. | Verified | Scale AI — pricing page Checked scale.com, its sitemap and its pricing URL on the check date: no published pricing surface with numbers exists. |
| Ownership and valuation Meta took a 49% minority stake in June 2025 at a valuation above $29 billion. Source recordsMeta invested $14.3 billion for a 49% non-voting stake at a $29 billion post-money valuation (Reuters and Scale AI). | Verified | Reuters, June 2025 (deal summary via UpMarket) Private-company transaction: the figure is reported, not filed. Scale announced the investment itself; the $14.3B price was first reported by Reuters. |
| Certifications Compliance posture reported as SOC 2, ISO 27001 and FedRAMP. | Unverified | FedRAMP Marketplace Claim located on vendor materials. SOC 2 reports are not public and no public registry carries them, so a SOC 2 claim can never be stamped above “claim located”. The FedRAMP Marketplace is a script-rendered registry and its listing text could not be read on the check date. Neither confirmed nor refuted. |
| Founded 2016 | Unverified | Scale AI — about page Company-stated. No state registry record was read in this pass — registry search portals are script-rendered — so the year is carried as company-stated. |
Right of reply: Scale AI — and any reader — can dispute a check with a source at [email protected]. Corrections are fixed, dated and logged publicly in the correction log.
Labelbox — claims & checks
Each claim below was checked against a primary source the vendor does not control — a regulator filing, a public registry, the vendor’s own published terms, or named reporting. Stamps are per claim, never per vendor, and every stamp is dated. Roll-up: 1 of 3 checked claims Verified · 2 Unverified · last checked Nov 8, 2026. How stamps work: methodology · Transparency Index.
| Claim | Stamp | Source |
|---|---|---|
| Pricing Published — Free tier (500 LBU/mo); Starter $0.10 per LBU; Enterprise custom. Source recordsLabelbox units (LBUs), per the documentation’s Limits table: Free — “500 LBU free each month”; Starter — “Fixed rate: $0.10 per LBU”; Enterprise — sales-quoted. Documentation page updated October 5, 2026. | Verified | Labelbox documentation — Limits Rates are published in Labelbox’s developer documentation. The marketing pricing page was retired; the documentation is the rate surface of record. |
| Founded 2018 | Unverified | Labelbox — labelbox.com Company-stated. No registry record was read in this pass. |
| Funding and profitability The company has raised $189 million and reports being profitable. | Unverified | Labelbox — labelbox.com Labelbox is private; no filing publishes a funding total or a profitability figure. Both numbers are company-reported. |
Right of reply: Labelbox — and any reader — can dispute a check with a source at [email protected]. Corrections are fixed, dated and logged publicly in the correction log.
Surge AI — claims & checks
Each claim below was checked against a primary source the vendor does not control — a regulator filing, a public registry, the vendor’s own published terms, or named reporting. Stamps are per claim, never per vendor, and every stamp is dated. Roll-up: 1 of 3 checked claims Verified · 2 Unverified · last checked Nov 8, 2026. How stamps work: methodology · Transparency Index.
| Claim | Stamp | Source |
|---|---|---|
| Revenue Reuters reported in July 2025 that Surge passed $1 billion in revenue in 2024 — more than Scale AI over the same period — while remaining bootstrapped and profitable. Source recordsSurge “took in more than $1 billion in revenue last year” (2024), per Reuters reporting carried by PYMNTS on July 1, 2025; the same reporting put Scale AI’s 2024 revenue at roughly $870 million and described the fundraise as Surge’s first. | Verified | Reuters, July 2025 (carried by PYMNTS) Surge is private and publishes no audited financials. The claim as recorded in this directory is a claim about the reporting, and the reporting exists as described; the underlying figure is reported, not filed. |
| Pricing Not published — enterprise, priced per task. | Unverified | Surge AI — surge.ai No pricing page was located in directory research or third-party listings. surge.ai is script-rendered and returned no readable text on the check date, so the absence is documented rather than confirmed. |
| Founded 2020 | Unverified | Surge AI — surge.ai Company-stated. No registry record for Surge Labs, Inc. was read in this pass. |
Right of reply: Surge AI — and any reader — can dispute a check with a source at [email protected]. Corrections are fixed, dated and logged publicly in the correction log.
Not yet checked
These vendors carry no stamps and no implied scores. Checks run in published waves: wave 2 (Mercor, Snorkel AI, iMerit, Sama, Micro1), wave 3 (the platform vendors), wave 4 (the remaining managed services, data brokers and — last, as the least-documented vendor in the directory — Zeton).
| Vendor | Model | Wave | Status |
|---|---|---|---|
| Mercor | Expert network | Wave 2 | Not yet checked |
| Micro1 | Expert network | Wave 2 | Not yet checked |
| Snorkel AI | Data lab | Wave 2 | Not yet checked |
| iMerit | Managed service | Wave 2 | Not yet checked |
| Sama | Managed service | Wave 2 | Not yet checked |
| Kili Technology | Platform | Wave 3 | Not yet checked |
| SuperAnnotate | Platform | Wave 3 | Not yet checked |
| Encord | Platform | Wave 3 | Not yet checked |
| CVAT | Platform | Wave 3 | Not yet checked |
| Label Studio (HumanSignal) | Platform | Wave 3 | Not yet checked |
| Lightly AI | Platform | Wave 3 | Not yet checked |
| BasicAI | Platform | Wave 3 | Not yet checked |
| V7 | Platform | Wave 3 | Not yet checked |
| Dataloop | Platform | Wave 3 | Not yet checked |
| CleverX | Expert network | Wave 4 | Not yet checked |
| CloudFactory | Managed service | Wave 4 | Not yet checked |
| Cogito Tech | Managed service | Wave 4 | Not yet checked |
| Label Your Data | Managed service | Wave 4 | Not yet checked |
| Zilo AI | Managed service | Wave 4 | Not yet checked |
| TELUS Digital | Managed service | Wave 4 | Not yet checked |
| FileYield | Data broker / marketplace | Wave 4 | Not yet checked |
| DataVendor (HUD) | Data broker / marketplace | Wave 4 | Not yet checked |
| Troveo | Data broker / marketplace | Wave 4 | Not yet checked |
| Nyne AI | Data broker / marketplace | Wave 4 | Not yet checked |
| Versos AI | Data broker / marketplace | Wave 4 | Not yet checked |
| Zeton | Data lab | Wave 4 | Not yet checked |
The other directories, audited the same way
A ranking is a claim too. This standing audit applies the same questions to the other lists in this niche: who published the list, who ranks first, and is the publisher a vendor?
- Vendor-published “best data annotation” lists. Read page one of the search results for this category: nearly every list is published by a vendor that ranks its own product first. Publisher: a vendor. Ranked first: the publisher. This publication sells no annotation services and ranks no house product — the methodology is the receipt.
- Metaclade Research, “Best LLM Training Data Providers” (October 2026). Ranked first: Zeton. Distribution: a press release. In this index, that ranking is recorded as what it is: a press-released ranking — a source class, stated next to the claim wherever it appears.
Machine-readable — take the data
The index is published as open data, generated from the same records as this page:
- /api/v1/transparency-index.json — the ranking above, as JSON.
- /api/v1/vendors.json — all 31 vendors, with full claim-level stamps for checked vendors.
- /api/v1/claims.csv — the full claims table as a downloadable CSV dataset.
- Per-vendor JSON:
/api/v1/vendors/{slug}.json— e.g. /api/v1/vendors/appen.json.
Reuse terms: cite with a link to this page or to the vendor profile. The stamps are dated — quote the date. A stamp quoted without its date is a claim we did not make.
Right of reply
Any vendor can dispute any check with a source. Email [email protected]; verified corrections are fixed, dated and entered in the public correction log. Vendors get the same process as readers — accuracy is not a partnership perk, and no vendor can buy, edit or expedite a stamp.