Hire the vendor that puts credentialed engineers, attorneys, scientists and physicians on your AI training and evaluation data. Tell us the field and the task; we name a vendor to you before anyone is contacted.
Free for buyers. A vendor may pay us once an introduction becomes a contract.
Figures as each company publishes them. Pay rates are what experts earn, not what a buyer pays. Not offers.
One task, one qualified person, one rubric. The credential behind each answer is the product, so it should be something you can look up.
RubricAB
Clinically accurate52
Says when to see a doctor41
States what is uncertain43
Plain language54
No patient records. Invented example, not a real project.
Attorneys, CPAs, physicians and nurses can be looked up in about a minute. Ask your vendor which registries it uses and how often it re-checks good standing.
Of the 15 vendors we track, none publishes a step-by-step credential process. Kili Technology shows SOC 2 Type II and ISO 27001, which cover data handling, not who the experts are.
TELUS Digital cites 500+ languages and Surge AI more than 70. A wide bench may still hold only a few people who can grade legal or medical text in your language, so ask for that count.
Vendors rarely list the professions they cover. What you can control is the evidence of expertise you require.
Written demonstrations, code review and tasks scored by automated verifiers.
Ask forA paid calibration task graded on your rubric. No license exists for engineers.
Contract redlines, case-law reasoning, statutory questions, rubric grading.
Ask forBar admission and good standing on the licensing state’s attorney search.
Financial modeling, filings analysis, accounting treatment, audit-style review.
Ask forCPAverify for CPAs, FINRA BrokerCheck for registered reps. Modeling needs no license.
Graduate-level physics, chemistry, biology and math problems with worked solutions.
Ask forDegree verification through a clearinghouse; publications on ORCID.
Clinical questions written and graded by physicians and nurses. Expertise only, no patient records.
Ask forDocInfo for physicians, ABMS for board status, Nursys for nurses.
Multilingual writing and grading, often inside one of the fields above.
Ask forATA certification for the pair, or a sample graded by a second native speaker.
14 of 15
A managed service, sometimes sold on top of the vendor's own labeling software. It recruits, screens and reviews, and you buy finished, graded items.
1 of 15
A marketplace such as Prolific recruits people and you manage the study. The researcher pays each participant and Prolific adds its fee.
Seven vendors whose published services include code. Listed in no ranked order; each quotes per project. The full software page.
Sells labs preference data, written demonstrations, rubrics, verifiers and human evals, plus ready-made expert datasets; also builds RL environments.
Works in more than 70 languages, by its own count
Supplies labs with post-training and evaluation data. Its RL environments include working replicas of the Jira, Salesforce and Zendesk interfaces, and MCP back ends.
Listed on the ASX. Offers labeling and LLM post-training data, and a catalog of RL tasks scored by verifiers.
500 coding tasks on offer, every one with its own automatic checker
Managed data company headquartered in Amsterdam. Its menu: SFT demonstration data, RLHF/DPO preference comparisons, machine-checkable eval and RL tasks, and red-team work.
Finds, screens and manages specialists whose output is post-training and reinforcement learning data for model builders.
Pitches an expert data service to labs and large companies (custom datasets, benchmarks, evals) and builds bespoke agent systems too.
Hires out its network of experts for human evals and expert-written training sets; also publishes the APEX benchmarks and builds RL environments.
Disclosure: any vendor on this list could pay us a fee should one of our introductions turn into a contract. Inclusion, order and wording here are not for sale.
Most comparisons between vendors should happen in the pilot, not on a sales call.
You define the task, the output format and a rubric. The vendor proposes a credential bar and a bench size.
A small round, often a few hundred items, graded by your own team.
Experts and reviewers align on the hard cases. Agreement between graders is measured and the rubric tightened.
Work arrives in rounds with a second-pass review and an acceptance rate you can audit.
A vendor that recruits people with professional credentials (engineers, attorneys, accountants, scientists, physicians, translators) to write demonstrations, rank model answers, build rubrics and grade outputs for AI training and evaluation. Most sell a managed service; a few run marketplaces where you manage the experts yourself.
None of the vendors on this site publishes a per-item or per-hour price for a managed expert project; each quotes per project. What some do publish is pay: Prolific sets an $8/hr floor for study participants and recommends $12/hr, and Alignerr, Labelbox's expert network, tells experts they can earn as much as $150/hr. Platform fees, management and review sit on top, so treat pay as a floor for your budget. Price a project turns hours into a rough range.
Through the vendors that recruit clinicians to write and grade answers to clinical questions, with no patient records involved. Handshake AI's Fellowship pays specialists by project and shows the hourly rate before you sign up; Alignerr tells experts they can earn up to $150/hr; Scale AI takes domain experts through its Outlier platform. Expect a vendor to look up your license on a public registry such as DocInfo or Nursys. This site introduces buyers to vendors and does not place experts. Science and medicine vendors.
Yes. The work is contract redlines, case-law reasoning, statutory questions and grading model answers against a rubric. Expert networks such as micro1 and Handshake AI describe recruiting specialists in general terms, and pay differs by project; none of the pages we read names law as a field, explains bar checks or publishes a rate for attorneys. A careful vendor confirms bar admission and good standing on the licensing state's public attorney search. Law and finance vendors.
Most vendors say they vet experts but do not publish the steps. Licenses for attorneys, CPAs, physicians and nurses can be looked up on public registries, so ask the vendor which ones it uses, how often it re-checks good standing and what share of applicants it turns away. Our credential checks page lists the registries by field.
For code, it depends on whether you need demonstrations, code review or tasks with automated verifiers. Appen lists 500 coding tasks paired with automated verifiers; Surge AI, Toloka and Turing describe verifiers or RL environments more broadly. For medicine, no vendor we track publishes the size of its clinical bench, so ask for the number of physicians or nurses in your specialty. Either way, a short pilot graded against your rubric settles it faster than any ranking.
More in the blog.
About two minutes, in three parts: company ZIP code, your work email, then some taps describing the task. A person reads every brief.
Describe the project in words only. No datasets, prompts, samples or files, here or by email.
Plain terms: we run an introduction service, and a vendor we introduce could pay us for it.
No newsletter and no mailing list. When nothing fits, our email says that instead.