Part-Time Data Ingest Engineer (contractor)

Digital Public Library of America
Boston, US
RemoteCareer-pivot friendly

Why this role

Pace
Steady
The job involves running monthly ingest cycles and providing regular status updates, indicating a steady but consistent pace of work.
Collaboration
Low
The position requires coordination with DPLA staff on metadata mapping and delivery, as well as troubleshooting ingestion errors with staff support, highlighting the need for teamwork.
Autonomy
High
The role demands self-directed work, with the contractor managing their own schedule and tasks independently.
Decision Impact
Individual
Decisions made in this role can impact the reliability and efficiency of the data ingestion process, affecting the overall performance of the library's data systems.
Role Level
Individual Contributor
The complexity arises from managing various technical components such as Scala pipelines, AWS services, and CI/CD processes, requiring a high level of technical expertise.
Career Pivot Friendly
Welcomes transferable skills
Individuals with experience in data management and pipeline operations in other industries, such as finance or healthcare, can easily transition into this role due to the transferable nature of technical skills.

Derived from job-description analysis by Serendipath's career intelligence engine.

What success looks like

  • maintained metadata ingestion process
  • resolved ingestion errors
Typical background
data engineering or related field

Transferable backgrounds

  • Coming from Data Engineer at a technology firm
    Data Pipeline Management · AWS Services
    Experience in managing data pipelines and working with AWS services directly applies to the role's requirements.
  • Coming from Software Developer in a startup environment
    Self-Directed Work · CI/CD Pipeline Maintenance
    The ability to work independently and maintain CI/CD pipelines is highly relevant to this position.

Skills & requirements

Required

Metadata IngestionPipeline MaintenanceTroubleshooting

Preferred

Cultural Heritage Metadata Standards

Stack & domain

ScalaApache SparkAWSJson-ldDAMSGitHubCollaborationSelf-directionProblem-solvingMetadata IngestionCultural HeritagePipeline Operations

About the role

This role involves overseeing the ingestion of metadata from various partners into the Digital Public Library of America's system, requiring a candidate who can manage data pipelines and troubleshoot issues efficiently.

Original posting from Digital Public Library of America via Indeed

Part-Time Data Ingest Engineer – Contract Digital Public Library of America (DPLA)

Remote | ~10–20 hrs/week | Fixed-term contract

DPLA is looking for a part-time contractor to coordinate and maintain metadata ingest operations. This position is directly involved in maintaining DPLA's ingestion process of harvesting, mapping, enriching, and indexing metadata from contributing partners.

What you'll be doing

  • Running monthly ingest cycles across active partner contributions (harvesting, mapping, enrichment, indexing)
  • Coordinating with DPLA staff on metadata mapping and delivery
  • Monitoring pipeline reliability and addressing bottlenecks or single points of failure
  • Troubleshooting ingestion errors and coordinating resolution with DPLA staff
  • Supporting deployments and maintaining CI/CD pipeline health
  • Providing regular status updates to DPLA staff

Technical environment

  • Pipeline: Scala, Apache Spark, Amazon EC2 and EMR, AWS S3, Apache Avro, Python scripts
  • Metadata: JSON-LD via DPLA MAP
  • APIs: Scala-based RESTful API on Elasticsearch 7, PostgreSQL auth backend
  • CI/CD: GitHub Actions, Docker, Terraform, AWS CodePipeline

What we're looking for

  • Hands-on experience with Spark/Scala pipelines and AWS (EC2, EMR, S3)
  • Familiarity with cultural heritage metadata standards (RDF, JSON-LD, Dublin Core, MODS, or similar) and DAMS (CONTENTdm, etc.)
  • Experience working across metadata quality, pipeline ops, and infrastructure
  • Familiarity with GitHub-based collaborative workflows
  • Self-directed

Details

  • 10–20 hours/week, flexible scheduling
  • $75 - $150 hourly rate (commensurate with experience)
  • An initial 3–6 month fixed-term contract, commencing April 1, with the possibility of extension.
  • Independent contractor arrangement (W-9/1099)
  • Must be legally authorized to work in the United States without company sponsorship

About the Digital Public Library of America

The Digital Public Library of America amplifies the value of libraries and cultural organizations as Americans' most trusted sources of shared knowledge. We do this by collaborating with partners to accelerate innovative tools and ideas that empower and equip libraries to make information more accessible. DPLA is a 501(c)(3) non-profit. Visit https://dp.la (https://dp.la/).

Source: Digital Public Library of America careers (Indeed)

Similar roles