Software and coding experts
Seven expert data vendors list code among the work they deliver, from written solutions to tasks graded by test suites. None publishes a buyer price, and none of the pages we read says how it tests an engineer's skill, so a paid pilot is the real comparison.
Last checked against vendor pages: 7 November 2026. Figures change; each row names the page it came from.
Vendors whose published services include code
| Vendor | What its own pages describe | Published detail | How you buy |
|---|---|---|---|
| Surge AI | Sells labs preference data, written demonstrations, rubrics, verifiers and human evals, plus ready-made expert datasets; also builds RL environments. | Works in more than 70 languages, by its own count | Managed service |
| Turing | Supplies labs with post-training and evaluation data. Its RL environments include working replicas of the Jira, Salesforce and Zendesk interfaces, and MCP back ends. | Not published | Managed service |
| Appen | Listed on the ASX. Offers labeling and LLM post-training data, and a catalog of RL tasks scored by verifiers. | 500 coding tasks on offer, every one with its own automatic checker | Managed service |
| Toloka | Managed data company headquartered in Amsterdam. Its menu: SFT demonstration data, RLHF/DPO preference comparisons, machine-checkable eval and RL tasks, and red-team work. | Not published | Managed service |
| micro1 | Finds, screens and manages specialists whose output is post-training and reinforcement learning data for model builders. | Not published | Managed service |
| Snorkel AI | Pitches an expert data service to labs and large companies (custom datasets, benchmarks, evals) and builds bespoke agent systems too. | Not published | Platform plus services |
| Mercor | Hires out its network of experts for human evals and expert-written training sets; also publishes the APEX benchmarks and builds RL environments. | Not published | Managed service |
Disclosure: a vendor in this table could pay us a fee should one of our introductions turn into a contract. Inclusion, order and wording here are not for sale.
What coding work do expert data vendors sell?
Four kinds of work show up again and again in vendor descriptions. Demonstrations (supervised fine-tuning data): an engineer writes the ideal solution and explains it. Preferences: an engineer compares two model answers and says which is better and why. Verifiable tasks: an engineer writes a problem plus the tests that grade it, so a model can be trained or scored without a human in the loop. Environments: a working replica of a tool, such as an issue tracker or a CRM, where an agent is set loose and its actions are checked.
The vendors' own wording shows where each leans. Appen lists 500 coding tasks, each paired with an automated verifier. Turing describes RL environments built as UI clones of Jira, Salesforce and Zendesk, plus MCP backend environments. Toloka names auto-verifiable tasks for evaluation and RL. Surge AI lists rubrics and verifiers alongside RLHF and demonstrations.
How do you check a software expert without a license?
There is no bar exam for engineers, so vendors rely on tests, and none of the seven pages we read says what those tests are. The practical answer is to grade the vendor's engineers yourself on a small paid trial.
| Credential asked for | How to check it yourself |
|---|---|
| Working engineer | No license exists. Ask for a paid calibration task graded against your rubric, plus a public code history where there is one. |
| Specialist (security, compilers, ML systems) | Ask the vendor how it tested the specialism: a domain test written by its own reviewers, not a self-reported skill list. |
| Computer science PhD | Degree verification through the university or a clearinghouse; publications through an ORCID record. Lookup: National Student Clearinghouse. |
What does coding data cost?
No vendor in the table publishes a per-task or per-hour buyer price. The nearest public figures come from Epoch AI, which interviewed people working on RL environments: asked what one task and its grader cost, they answered "$200 to $2000 mostly" and called a $20,000 task an outlier. Exclusivity, they said, multiplies the bill four- or fivefold. Epoch is clear that these are interview estimates, not contract terms.
For simpler work (demonstrations and preferences) the driver is expert hours. Our project pricer turns items and minutes per item into hours, using published pay figures as reference points.
Questions to put to each vendor
- How many engineers qualified in our language and framework are available this quarter?
- What test did they pass, who wrote it, and what share of applicants fail it?
- Who writes the verifiers, and who checks that a verifier cannot be gamed?
- Is any task or environment resold to other clients? What does exclusivity add?
- Where does our code or repository context sit during the project, and who can see it?
Ask every vendor on the short-list the same questions in writing, so the answers can be laid side by side.
Where these figures come from
- Surge AI (surgehq.ai) (page checked November 7, 2026)
- Turing (turing.com) (page checked November 7, 2026)
- Appen (appen.com) (page checked November 7, 2026)
- Toloka (toloka.ai) (page checked November 7, 2026)
- micro1 (micro1.ai) (page checked November 7, 2026)
- Snorkel AI (snorkel.ai) (page checked November 7, 2026)
- Mercor (mercor.com) (page checked November 7, 2026)
- "An FAQ on reinforcement learning environments", Epoch AI Gradient Updates, January 12, 2026 (page checked November 7, 2026)
Questions buyers ask
Do I need software engineers or will general annotators do?
For code review, debugging traces and anything graded on correctness beyond "the tests pass", you need working engineers. For tasks scored only by automated tests, the expert writes the task and the tests; the grading itself is mechanical.
Which programming languages do vendors cover?
None of the vendor pages we read lists the programming languages its experts cover. Name yours in the brief and ask for the number of qualified engineers per language.
What is a verifier?
A program that checks a model’s answer automatically, such as a unit test suite for a coding task. Appen’s catalog of 500 coding tasks is built this way, one checker per task.
Can I buy coding data off the shelf?
Yes, from some. Surge AI’s product page includes ready-made expert datasets next to its custom work. Ask whether a dataset is exclusive to you; Epoch AI's interviews suggest exclusive work costs several times more.
Get a quote
About two minutes, in three parts: company ZIP code, your work email, then some taps describing the task. A person reads every brief.
Describe the project in words only. No datasets, prompts, samples or files, here or by email.
Plain terms: we run an introduction service, and a vendor we introduce could pay us for it.
Brief received.
- We set your brief beside what each expert data vendor publishes about fields, credentials and minimums.
- Our email back names one vendor and says why. At that stage your brief has gone nowhere.
- Reply yes and we make the introduction by email; from there, scope, price and contract are settled directly by you and the vendor.
No newsletter and no mailing list. When nothing fits, our email says that instead.