Description
Data Platform Engineer
This is a vacancy for the 26-27 school year with a target start date of July 1, 2026
Mission:
The Data Platform Engineer builds and operates IDEA's data infrastructure on Snowflake, enabling reliable, scalable access to data that supports analytics, reporting, and research across multiple states. This role designs automated ingestion pipelines, optimizes platform performance and cost, and ensures the data platform functions as a production-grade system for downstream teams.
Reporting to the Manager of Data Platform Engineering, this engineer works hands-on with ELT pipelines, infrastructure-as-code, and Snowflake administration while contributing to IDEA's transition from legacy ETL systems to a modern lakehouse architecture.
Supervisory Responsibilities:
Individual contributor role with no direct reports. Senior engineers may mentor peers and lead technical initiatives.
Location:
This is a full-time remote position based in Texas, with preference given to candidates who live in Austin, El Paso, Houston, Permian Basin (Midland/Odessa), Rio Grande Valley, San Antonio, and Tarrant County (Fort Worth), or who are willing to relocate.
Travel Expectations:
Minimal travel (5-10% annually) for collaboration, training, or critical implementation milestones.
What You'll Do - Accountabilities
Essential Duties:
- Design, build, and maintain automated ELT pipelines ingesting data from diverse source systems into Snowflake.
- Configure and manage cloud-native ingestion tools and custom Python-based pipelines when needed.
- Build and maintain Bronze-layer tables with schema evolution handling, audit metadata, and lineage.
- Implement ingestion-level validation and monitoring to catch issues early.
- Document source configurations, refresh schedules, and troubleshooting procedures.
- Partner with Analytics Engineering to ensure ingestion patterns support downstream transformation needs.
- Administer Snowflake environments, including databases, schemas, warehouses, access controls, and security settings.
Additional Duties and Responsibilities:
- Optimize performance and cost through warehouse sizing, clustering, query analysis, and resource monitoring.
- Manage Snowflake objects using infrastructure-as-code patterns.
- Implement security best practices including RBAC, encryption, auditing, and network policies.
- Evaluate and adopt new Snowflake capabilities as appropriate.
- Own Terraform-based infrastructure definitions for Snowflake and related platform components.
- Automate recurring operational tasks such as provisioning, access grants, and environment setup.
- Build CI/CD pipelines for infrastructure changes with testing and safe deployment practices.
- Develop reusable templates and modules to accelerate onboarding of new sources and domains.
- Maintain clear documentation and runbooks for platform operations.
- Implement monitoring and alerting for pipelines, platform health, and performance.
- Troubleshoot pipeline failures and platform issues using systematic root-cause analysis.
- Embed observability (logging, metrics, alerts) into all production pipelines.
- Collaborate closely with Analytics Engineering, DataOps , and Data Governance partners.
- Participate in code reviews and design discussions.
- Share platform knowledge through documentation, mentoring, and team forums.
- Contribute to retrospectives and continuous improvement efforts.
Knowledge and Skills - Competencies
- Make Strategic Decisions: This team member uses data, feedback, and insights to inform thoughtful decision-making, while considering the impact on their direct reports and team. They communicate decisions with clear rationale and begin to connect their choices to broader team objectives.
- Manage Work and Teams: This team member sets clear, measurable goals and regularly reflects on progress, adjusting actions as needed. They prioritize work aligned with their goals using a task management system and consistently meet deadlines through effective time management.
- Grow Self and Others: This team member regularly offers affirming and adjusting feedback, maintaining a positive balance that reinforces growth and motivation. They provide transparent, candid performance insights and offer consistent coaching and development aligned with individual goals, supporting both direct reports and cross-functional partners.
- Build a Culture of Trust: This team member proactively builds strong personal and professional relationships with individual stakeholders and regularly seeks feedback to improve their work experience. They create a supportive environment where others feel safe to take risks and learn from mistakes without fear of retribution.
- Communicate Deliberately: This team member communicates thoughtfully by anticipating potential misunderstandings and providing necessary context to ensure clarity. They leverage structured communication channels to address challenges, ask meaningful questio