Remote Data Engineering Expert

As a Data Engineering Expert, you will play a crucial role in advancing frontier agent evaluations in data engineering by building long-horizon pipeline tasks. These tasks will be designed to mirror your existing work and will be evaluated against a deterministic rubric that measures agent performance against verifiable ground truth. Your focus will be on creating scenarios that yield checkable answers, avoiding open-ended essays or subjective judgments.

Key Responsibilities

• Pipelines: Develop ETL/ELT and dbt models that produce specified output tables, incorporating incremental logic with defined watermark behavior.
• Orchestration and Quality: Create Airflow/Dagster DAGs that successfully pass a test suite, and implement data quality tests with known pass/fail cases.
• Warehouse Design: Design schemas that match defined contracts and meet performance targets tied to a measured query-time budget.

These scenarios will be challenging and require long sessions of focused work.

Qualifications

• BS or MS in Computer Science or a related field, with 3+ years of experience in data engineering or analytics engineering.
• Expertise in one or more of the following areas: dbt model development, pipeline orchestration (Airflow, Dagster, Prefect), warehouse design (Snowflake, BigQuery, Redshift, Databricks), and data quality and testing.
• Ability to read and produce data engineering artifacts, including dbt models, DAGs, schema documents, data contracts, and test suites.
• Strong written communication skills, with the ability to articulate reasoning step by step and encode it into deterministic rubrics.
• Must be located in the United States or Canada.

Work Terms

This is a remote position with an hourly employment type.

Compensation

The compensation ranges from $90 to $125 per hour, depending on domain depth and prior experience. Strong contributors will have opportunities for promotion based on task quality and throughput.