Data Engineer (Azure)

Job Title: Data Engineer (Azure)

Location: Halifax, ON (Hybrid – 3 days/week onsite at client location)

Employment Type: expected to run until October with chance of renewal

Overview

We are looking for a skilled Data Engineer to design, build, and optimize scalable data platforms within the Azure ecosystem. This role will focus on developing robust ETL pipelines, enabling advanced analytics and machine learning use cases, and contributing to the architecture of modern cloud-based data solutions.

You will work closely with analytics, data science, and engineering teams to ensure data is reliable, secure, and performance-optimized across the enterprise.

Key Responsibilities

• Design, develop, and optimize ETL/ELT pipelines using Azure Data Factory, Databricks, and PySpark
• Build scalable, high-performance data processing frameworks within Azure Data Lake environments
• Ingest, transform, and model structured and semi-structured data to support analytics and ML initiatives
• Collaborate with cross-functional stakeholders to translate business requirements into technical data solutions
• Implement best practices for data governance, security, performance tuning, and cost optimization in Azure
• Automate and orchestrate workflows using Databricks Jobs and Azure Data Factory pipelines
• Monitor pipeline reliability and performance using Azure Monitor and logging frameworks
• Contribute to cloud data architecture design, documentation, and continuous improvement initiatives

Technical Environment

• Azure Data Factory
• Azure Data Lake Storage (ADLS)
• Azure Databricks
• PySpark / Spark
• Azure Monitor
• Cloud-based analytics and machine learning platforms

Required Skills & Experience

• Strong hands-on experience with Azure Data Factory and Azure Databricks
• Proficiency in Python and PySpark for distributed data processing
• Experience building and optimizing data pipelines in Azure Data Lake environments
• Solid understanding of data modeling, transformation techniques, and performance tuning
• Experience implementing data governance and security best practices in cloud platforms
• Familiarity with monitoring, logging, and production support processes
• Strong documentation and communication skills

Nice to Have

• Experience supporting machine learning workflows
• Knowledge of CI/CD practices for data engineering
• Experience working in agile delivery environments

Ideal Candidate

• Strong problem-solver with attention to data quality and performance
• Comfortable working in a fast-paced, cloud-first environment
• Able to operate independently while collaborating effectively across teams
• Focused on building scalable, reliable, and maintainable data solutions