Role
We are seeking a skilled and detail-oriented engineer to join our dynamic team. In this role, you will be responsible for the development, maintenance, and optimization of real-time systems that enable data collection, processing, analysis, visualization, and alerting. You will work closely with data engineers, software developers, and data analysts to ensure the smooth operation of our data acquisition infrastructure. The ideal candidate has a strong background in web scraping, data extraction, and system monitoring, along with a passion for solving complex technical challenges.
Responsibilities
- Design, develop, and maintain scalable data extraction systems.
- Parse and clean data from various sources (HTML, XML, JSON, APIs, JavaScript-rendered pages).
- Implement efficient request patterns, caching strategies, and data validation.
- Store and manage large-scale datasets in databases (SQL/NoSQL).
- Monitor system health, handle failures gracefully, and ensure data quality.
- Design and manage databases and data warehouses.
- Document extraction logic, data schemas, and operational procedures.
- Build and maintain ETL/ELT pipelines.
- Collaborate with data engineers, analysts, and product teams to understand requirements.
Qualifications
- Bachelor’s degree in computer science, Information Technology, or related field.
- Strong programming skills in Python.
- Proven experience in web scraping, data extraction, and crawler development using tools such as Scrapy, Beautiful Soup.
- Web Technologies: Deep understanding of HTML/CSS parsing, JavaScript execution, REST/GraphQL APIs.
- Data Handling: Experience with SQL/NoSQL databases, data validation, and ETL processes.
- Infrastructure: Familiarity with containerization (Docker/Kubernetes).
- Problem-Solving: Ability to adapt to site structure changes and design maintainable solutions.
Preferred Qualifications
- Experience with Playwright, Selenium, or other headless browser tools.
- Knowledge of message queues (Kafka) and distributed processing.
- Familiarity with data quality frameworks or observability tools.
- Experience with data visualization or analytic platforms.
Location of work
Hong Kong Science Park
Benefits
- Competitive salary.
- Annual leave and group medical insurance.
- Good team culture.
- Assistance to apply for an IANG VISA.
Other: Fresh graduates are welcome.