Data Engineer - Python

Position Overview

We are seeking a Specialist with 5 to 7 years of experience in designing and implementing data engineering solutions. This role focuses on leveraging Python and Azure Data Factory to build scalable ETL processes and data pipelines within an Azure environment.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes using Python and Azure Data Factory.
• Collaborate with data application teams and product owners to understand requirements and deliver analytics solutions.
• Utilize Azure Data Factory and Azure Data Lake for data ingestion, storage, and processing.
• Implement data migration and transformation workflows leveraging Azure cloud services.
• Ensure data quality, performance tuning, and optimization of data pipelines.
• Support continuous integration and continuous deployment (CICD) pipelines for data engineering projects.
• Apply best practices for data governance, privacy, and security within Azure environments.
• Lead the design and implementation of both batch and streaming data pipelines.
• Provide technical leadership and mentorship to team members.
• Collaborate with DevOps teams to manage deployment, monitoring, and support of solutions.
• Troubleshoot and resolve issues to ensure high availability and reliability.
• Participate in architectural discussions and develop coding standards and security guidelines.
• Drive automation of data workflows and improve operational efficiency.
• Support production environments with timely incident response and root cause analysis.

Required Qualifications

• 5 to 7 years of relevant experience in data engineering or related fields.
• Expertise in Python programming.
• Proficiency in Azure Data Factory and its ecosystem.

Preferred Qualifications

• Familiarity with Azure Data Lake and other Azure cloud services.