Before you start
Define the evaluation objective, eligibility, consent and quality rubric. Prepare task content appropriate for participants and a secure destination for results.
Open: API integration → survey authoring, recruitment and response reads.
Steps
- Create a concise evaluation questionnaire through groups.save, surveys.save and groups.assign_form. Ask participants to compare outputs, assign labels, rate a response or explain a preference.
- Review and publish the exact task version through surveys.review and surveys.publish. Store the group, form, question and version identifiers with the evaluation batch.
- Read audience options, quote the target and create the study through the recruitment API. Confirm the approved budget and funding, then launch recruitment.
- Monitor submitted-response and quota counts. Read feedback with responses.list and collection:read when context is needed. Preserve answer coordinates and task version in your evaluation dataset.
- Read the completed feedback after DatumForms applies its automatic panel checks. Panel responses are accepted by DatumForms; your pipeline does not manually accept or reject them.
- Stop recruitment when the round is complete. For changed prompts, outputs or rubric, publish a new task round and retain its identifiers for comparison.
Check the result
Your pipeline receives traceable human feedback grouped by evaluation batch and original task definition. Automation handles authoring, recruitment monitoring and response retrieval without requiring manual authoring steps.
Troubleshooting
If a round’s publication changes after quoting, create a fresh study against the intended version. Use current audience availability. Contact DatumForms support with the study and response IDs if a panel response needs investigation. Avoid combining materially different task versions without labelling them.
Next steps
Create/publish API recipe, recruitment API and response export.