How expert data vendors check credentials

A buyer paying for “expert” data is paying for one thing above all: that the person who wrote or graded each item really holds the expertise claimed. It is also the claim that is hardest to see from outside. This piece sets out what vendors publish about it, what a check can and cannot prove, and how to build verification into a contract.

What do vendors actually publish?

Very little. On 7 November 2026 we read the pages of fifteen vendors that sell expert data, from managed services such as Surge AI, Turing and Toloka to marketplaces such as Prolific. None of those pages sets out a credential process step by step: which registries are checked, what test an applicant sits, how many fail, how often licenses are re-checked.

What vendors do publish is adjacent. Some publish pay: Labelbox’s expert network, Alignerr, advertises as much as $150 an hour, and Handshake AI says fellows see the hourly rate before joining a project. One publishes security certifications: Kili Technology’s pricing page carries SOC 2 Type II and ISO 27001 badges. Neither tells you who the experts are.

That gap does not mean vendors skip vetting. It means the evidence lives in sales calls, security questionnaires and contracts, which is where a buyer has to go and get it.

Which credentials can be checked publicly?

Licensed professions in the United States are the easy case, because the license is a public record.

Two limits apply to all of these. A registry proves the license, not that the person typing is the license holder, so identity checks matter as much as the lookup. And a license proves a minimum, not skill at the task: a lawyer admitted in 1998 who has done only real estate closings may grade a securities question badly.

What about fields with no license?

Software engineering, data science, most finance roles and most academic science have no public license. Here vendors rely on tests, and the buyer’s best check is the same: a graded task.

The usual pattern is a paid pilot. The vendor’s experts complete a few hundred items, your team grades them blind against the rubric, and the acceptance rate becomes the measure of the bench. A PhD can be confirmed through the university or a clearinghouse, and publications through an ORCID record, but neither says the person can write a hard, checkable problem in your subfield.

Where do credential checks fail?

Three failure modes come up in practice.

Identity. The license is real, but the account belongs to someone else. Ask how the vendor confirms identity at onboarding and again during the project.

Drift. The expert was licensed at onboarding and lapsed, or was disciplined, months later. Ask how often the vendor re-checks.

Credential inflation. The brief asked for licensed attorneys; the bench includes law graduates who have not been admitted. Both are useful, at different prices. Ask for each expert’s credential as a field in the delivered data, so you can see the mix.

How do you put verification into the contract?

A short clause does most of the work:

1. The vendor warrants that each expert holds the credential specified for the task, verified against the relevant registry before work starts.
2. Licenses are re-checked at a stated interval for the life of the project.
3. The buyer may audit the evidence for a random sample of experts.
4. Work by an expert who did not hold the credential is redone at the vendor’s cost.

Add a line on conflicts for law and finance work, where a practicing professional may have a conflict with your company or its customers.

What questions separate vendors?

When we compare expert data companies for a buyer, these five questions do more than any website:

• Which registries do you check, for which credentials, and do you keep the evidence?
• What test does an expert pass before working on a task like ours, and who wrote it?
• What share of applicants in our field pass that test?
• How many qualified experts in our field are active now, not merely registered?
• Can we audit a sample of credential records?

A vendor that answers all five in writing, with numbers, is showing you its process. One that answers in adjectives is showing you its marketing.

What changes for medical work?

Medical expert work on this site means clinicians answering and grading clinical questions from their own training, never patient records. The check is the same as any licensed field: state license and board actions through the state medical board or DocInfo, specialty through the ABMS lookup. The extra question is specialty depth, because a vendor with many physicians may have only two or three in the specialty you need, and two graders cannot settle a disagreement.

Verification is the one part of an expert data purchase where a buyer can replace trust with a lookup, so it is worth the hour it takes.