Data Quality Engineer-Azure Data Platform Testing

Job Description: Data Quality Engineer – Azure Data Platform Testing

Overview

Seeking a seasoned Azure Data Engineer Tester to drive test strategy and ensure data quality across a large-scale Azure data platform modernization initiative. Data is a critical enterprise asset, and this role ensures accurate ingestion, transformation, storage, and reporting across cloud-based data solutions. The ideal candidate has test lead experience, strong automation skills, Agile exposure, and familiarity with P&C insurance domains.

Key Responsibilities

Test Strategy & Leadership

• Drive overall test strategy and create effective test cases and scenarios based on business and user requirements
• Define and implement test automation strategy, ensuring the right balance of manual and automated testing
• Participate in architecture, design, requirements, and code reviews; provide feedback to stakeholders

Azure Data Pipeline Testing

• Test Azure data pipelines built using services such as Microsoft Azure, Azure Data Factory, and Azure Synapse Analytics
• Validate end-to-end data movement from source to destination across batch and streaming workflows

Ensure Data Quality & Accuracy

• Validate ETL/ELT transformations across Medallion architecture layers
• Identify missing, duplicate, or incorrect data and ensure business rules are applied accurately
• Validate analytical outputs such as dashboards, reports, and data visualizations

Database & Storage Validation

• Test data stored in Azure SQL Database and Azure Data Lake Storage for integrity, reliability, and performance
• Validate schema, source-to-target mappings, and data models

Automation & Performance Testing

• Develop automated test scripts using SQL, Python, and PySpark
• Perform performance, scalability, and load testing of pipelines and data platforms
• Track defects and test cycles using Jira

Desired Skills & Competencies

Data & Technical Acumen

• Strong understanding of data warehousing concepts, star schemas, data marts, and Medallion architecture
• Ability to read data models and update source-to-target mapping documents
• Hands-on experience with Snowflake, Azure, DBT, Python, and PySpark

Industry & Methodology

• Experience working in Agile environments
• Knowledge of Property & Casualty (P&C) and Specialty Insurance is preferred

Responsibilities: Key Responsibilities

Test Strategy & Leadership

• Drive overall test strategy and create effective test cases and scenarios based on business and user requirements
• Define and implement test automation strategy, ensuring the right balance of manual and automated testing
• Participate in architecture, design, requirements, and code reviews; provide feedback to stakeholders

Azure Data Pipeline Testing

• Test Azure data pipelines built using services such as Microsoft Azure, Azure Data Factory, and Azure Synapse Analytics
• Validate end-to-end data movement from source to destination across batch and streaming workflows

Ensure Data Quality & Accuracy

• Validate ETL/ELT transformations across Medallion architecture layers
• Identify missing, duplicate, or incorrect data and ensure business rules are applied accurately
• Validate analytical outputs such as dashboards, reports, and data visualizations

Database & Storage Validation

• Test data stored in Azure SQL Database and Azure Data Lake Storage for integrity, reliability, and performance
• Validate schema, source-to-target mappings, and data models

Automation & Performance Testing

• Develop automated test scripts using SQL, Python, and PySpark
• Perform performance, scalability, and load testing of pipelines and data platforms
• Track defects and test cycles using Jira

Qualifications: Bachelors Degree