Software engineers design debugging and integration tasks, with rubrics, that test AI agents on enterprise code. Suits working engineers who can write and verify fixes and define checkable correctness criteria.
About the Role
Alignerr is building a dataset of expert tasks that train and evaluate advanced AI agents on real enterprise work. As a Task Author for the Software Engineer role, you will design and calibrate real engineering tasks — diagnosing failing integrations from logs, tracing bugs through a codebase, and writing and verifying fixes — that genuinely challenge a capable AI agent. This role requires hands-on engineering skill; generalist profiles cannot do this work.
Key Responsibilities
- Author realistic engineering task prompts: integration failures, log-based debugging, bug traces through source code, and configuration or integration fixes.
- Author realistic engineering task prompts: integration failures, log-based debugging, bug traces through source code, and configuration or integration fixes.
- Write scoring rubrics that define exactly what a correct fix or diagnosis looks like and how it is verified — including live environment validation where applicable.
- Set up task environments with realistic codebases, logs, alerts, and system states that drop an AI agent into a plausible engineering situation.
- Solve each task yourself — write the actual fix and confirm it works — to validate the task is sound and the rubric is accurate.
- Calibrate task difficulty — adjust code complexity, log volume, failure modes, or ambiguity until the task reliably challenges the model to the intended degree.
- Review and correct AI-drafted task prompts or rubrics when provided.
- Working software engineer with hands-on experience in industry — this is not a theoretical or instructional role.
- Working software engineer with hands-on experience in industry — this is not a theoretical or instructional role.
- Fluency in Git, version control workflows, and code review practices.
- Strong debugging skills: comfortable reading logs, using observability tools, and tracing failures through a codebase.
- Experience with infrastructure, integrations, and configuration code (APIs, services, config files, CI/CD).
- Ability to write and verify a code fix in a live or simulated environment.
- Ability to define precise, checkable correctness criteria for engineering outcomes.
- Experience in manufacturing, industrial IoT, or enterprise SaaS engineering contexts.
- Background with monitoring/observability platforms (e.g., Datadog, PagerDuty, Grafana).
Listing from Alignerr's job board. Details can change; the platform's page is the final word.
