Candidates with experience in Big Data infrastructure and a background in national security or defense with leadership capabilities will find this senior-level role meaningful.
Derived from job-description analysis by Serendipath's career intelligence engine.
Original posting from Csit via Lever
CSIT develops products to advance the national security interests of Singapore. We use our products in a wide range of operations, including but not limited to Counter-terrorism and Computer Network Defence. We are looking for talented Senior Big Data Platform Staff Engineer. Join the awesome CSIT family and use cutting-edge technologies to protect the nation.
Responsibilities:
We are seeking an experience Big Data Platform Staff Engineer to join our team, responsible for managing and optimizing our Big Data Infrastructure platform. The successful candidate will have a deep understanding of Big Data technologies, architectures, and systems, as well as experience with managing large-scale data processing and analytics environments. The role will involve designing, implementing, and maintaining the Big Data Infrastructure platform, ensuring high availability, scalability, and performance.
- Design, implement and manage the Big Data Infrastructure platform, including Hadoop, Spark, NoSQL databases, and data lakes.
- Manage Big Data clusters, including monitoring, troubleshooting, and optimizing cluster performance, as well as managing job scheduling and resource allocation.
- Design and implement storage solutions for Big Data workloads, including HDFS, S3, and other object storage systems.
- Ensure the security and governance of the Big Data Infrastructure platform, including managing access controls, implementing data encryption, and conducting regular security audits.
- Collaborate with data engineers and data scientists to optimize Big Data application performance and provide technical support and guidance on using the Big Data Infrastructure platform.
- Plan and manage Big Data resource capacity, including forecasting, procurement, and deployment of new hardware and software.
Requirements (Minimum Qualifications):
- Bachelor’s degree in Computer Science, Engineering, or a related field
- At least 8 years of experience in managing Big Data Platforms. Proven experience with technologies like Spark, Iceberg, Trino, Elastic Stack, NiFi, Kafka, Airflow.
- Expertise in Big Data Engineering, especially in building streaming/batch pipelines, architecting data warehouses, and deploying AI/ML infrastructure.
- Capable of managing and utilizing diverse storage protocols, including S3, NFS, SMB, and HDFS.
- Strong knowledge in securing applications through SSL/TLS encryption, data at rest encryption, and robust authentication/authorization (AuthN/AuthZ) using OIDC/OAuth2.
- Comprehensive understanding of core networking principles, including firewall configuration, DNS management, routing and subnetting.
- Experience with scripting languages, such as Bash, Python, Scala and Java.
- Proficient in Kubernetes orchestration, leveraging its ecosystem for automated scaling, self-healing, and seamless CI/CD integration.
- Experience in supporting search workloads in multi-terabytes and billions of records
- Experience with agile development methodologies and version control systems, such as Git.
- Experience in leading engineering teams.
Nice-to-haves:
- Certifications in Big Data, such as Hadoop Certified Developer or Spark Certified Developer.
- Experience in managing physical and virtual infrastructures.
- knowledgeable in deploying and scaling LLM-based and agentic AI/ML applications.
Why join us?:
- The work is purposeful and meaningful
- You will work with the best engineers
- We work with modern technologies and tech stacks
- We have excellent engineering culture and work-life balance
- We aspire to engineering and operational excellence
- We empower to innovate
- We grow together as a family
As CSIT is an agency under the Ministry of Defence (Singapore), only Singapore Citizens will be considered.
Source: Csit careers (Lever)