DatumForms.
Join the panel Sign in Create a survey

DATUMFORMS DOCUMENTATION

Run an AI feedback study

Automate repeated human evaluation rounds and feed versioned results into your AI workflow.

For developers and research teams

Updated 15 November 2026

On this page

Before you start

Define the evaluation objective, eligibility, consent and quality rubric. Prepare task content appropriate for participants and a secure destination for results.

Open: API integration → survey authoring, recruitment and response reads.

Steps

  1. Create a concise evaluation questionnaire through groups.save, surveys.save and groups.assign_form. Ask participants to compare outputs, assign labels, rate a response or explain a preference.
  2. Review and publish the exact task version through surveys.review and surveys.publish. Store the group, form, question and version identifiers with the evaluation batch.
  3. Read audience options, quote the target and create the study through the recruitment API. Confirm the approved budget and funding, then launch recruitment.
  4. Monitor submitted-response and quota counts. Read feedback with responses.list and collection:read when context is needed. Preserve answer coordinates and task version in your evaluation dataset.
  5. Read the completed feedback after DatumForms applies its automatic panel checks. Panel responses are accepted by DatumForms; your pipeline does not manually accept or reject them.
  6. Stop recruitment when the round is complete. For changed prompts, outputs or rubric, publish a new task round and retain its identifiers for comparison.

Check the result

Your pipeline receives traceable human feedback grouped by evaluation batch and original task definition. Automation handles authoring, recruitment monitoring and response retrieval without requiring manual authoring steps.

Troubleshooting

If a round’s publication changes after quoting, create a fresh study against the intended version. Use current audience availability. Contact DatumForms support with the study and response IDs if a panel response needs investigation. Avoid combining materially different task versions without labelling them.

Next steps

Create/publish API recipe, recruitment API and response export.

Need help with this task? Contact DatumForms. Include the article URL and the step you reached.

Tracking preferences

Essential functions stay on. Choose whether to allow optional tracking on this website. You can change your choices in the footer at any time.

Read our privacy policy