Job Description
Work mode: Hybrid (with 4 days a week at London, Canary Wharf)
Contract duration: Long term
Location: London
Must have skills:
-Python, Postgres SQL, Azure Databricks, AWS(S3), Git, Azure DevOps CICD, Apache Airflow
-Preferred Domain: Energy trading
Minimum number of relevant years of experience: 8+
Detailed Job Description:
JD:
- 8+ years of experience in python scripting.
- Exposed to python packages such as NumPy, pandas, polars beautiful soup, Selenium, Requests etc.
- Experience in building native python ETL pipelines for scraping/processing.
- Good to have experience in processing big data.
- Proficient in SQL programming, Postgres SQL. Spark knowledge is a good to have
- Knowledge on DevOps CI/CD like Azure Pipelines, Jenkins, Git.
- Experience working with AWS services(S3, lambda,etc) and Azure Databricks.
- Have experience in delivering project with Agile and Scrum methodology.
- Able to co-ordinate with Teams across multiple locations and time zones
- Strong interpersonal and communication skills with an ability to lead a team and keep them motivated.
- Python, Postgres SQL, Azure Databricks, AWS(S3), Git, Azure DevOps CICD, Apache Airflow
Key Responsibilities:
- Lead the design, development, and optimization of scalable data pipelines, data processing frameworks, and enterprise grade Python applications.
- Provide technical leadership in Python development, including best practices for automation, data processing, and integration workflows.
- Work extensively with Python and data centric libraries such as NumPy, Pandas, BeautifulSoup, Selenium, pdfplumber, and Requests.
- Architect and optimize SQL queries and database solutions, with strong proficiency in PostgreSQL and other relational data stores.
- Integrate DevOps practices using CI/CD pipelines, Jenkins, Git, and ensure smooth deployment of data solutions.
- Develop and manage cloud based data workflows on platforms such as AWS (S3) and Azure Databricks.
- Drive agile delivery by collaborating with cross functional teams under Agile/Scrum methodologies.