Mid-Level Data Engineer

Role Overview The Mid-level Data Engineer will support the design, build, and optimize scalable data pipelines and services that power advanced analytics and machine learning solutions. This role emphasizes data quality, performance, and interoperability in modern cloud environments, enabling CPSC’s strategic acceleration toward Sentinel-driven product safety analytics. The engineer will ensure secure, efficient, and reproducible data workflows that support predictive modeling, real-time monitoring, and actionable insights. Key Responsibilities

• Data Pipeline Engineering o Develop production-grade ETL workflows using Python and Microsoft-based frameworks to ingest, transform, and validate large-scale structured and unstructured data. o Implement schema enforcement, data validation, and quality checks to maintain integrity across diverse sources. o Optimize pipelines for performance, scalability, and fault tolerance using open-source and cloud-native patterns.
• Cloud Integration & Orchestration o Architect and manage Azure-based data solutions, including Data Lake Storage, Azure SQL, and cloud storage access from Python services. o Design and deploy workflow orchestration using Azure Data Factory or Foundry for scheduling, monitoring, and automation. o Ensure secure integration of APIs and services within the Microsoft ecosystem for seamless data exchange.
• Advanced Technical Development o Build Python-based data services leveraging libraries such as Pandas, Pytorch, and other open-source frameworks for high-performance processing. o Implement logging, monitoring, and performance tuning for robust operational reliability. o Develop API endpoints and microservices to enable interoperability with analytics and ML platforms.
• Collaboration & Governance o Work closely with data scientists, analysts, and cloud architects to deliver clean, reliable data for predictive modeling and real-time dashboards. o Apply data governance best practices, ensuring compliance, reproducibility, and auditability across workflows. o Contribute to Agile team processes, driving iterative improvements and shared problem-solving. Desired Experience
• 3+ years developing and deploying advanced statistical and machine learning models or supporting data pipelines for such models.
• Proficiency in Python (Pandas required; scikit-learn, NumPy, and related libraries preferred).
• Strong SQL skills and experience integrating data from relational databases.
• Hands-on experience in cloud environments (Azure); Microsoft Data Engineer certification advantageous. Additional Technical Expertise
• Open-source frameworks for production-grade data pipelines.
• ETL development using Python and Microsoft technologies.
• Data validation, schema enforcement, and quality assurance.
• API development within Microsoft ecosystem.
• Performance optimization, logging, and monitoring for large-scale systems.
• Azure Data Lake Storage integration and Azure SQL connectivity.
• Workflow orchestration with Azure Data Factory.
• Deployment and operation of Python-based data services in Azure.
• Familiarity with open-source data processing libraries (Pandas, PyTorch, Tensorflow etc.). Preferred Qualifications
• Bachelor’s, Ph.D., or equivalent professional experience in Data Science, Computer Science, Statistics, Engineering, or related field.
• Knowledge of data engineering patterns for scalable, secure systems in regulated environments.

Pay: $140,000.00 - $160,000.00 per year

Work Location: In person