Lead Data Engineer-Microsoft Fabric

About Boston Insights

At Boston Insights, our mission is to make clinical supply chains reliable and resilient, ensuring uninterrupted drug delivery to patients. We empower pharmaceutical companies to avoid delays, reduce waste, and accelerate delivery with AXIOM, our SaaS-based clinical supply chain control tower.

AXIOM provides real-time visibility into supply networks, enabling efficient planning, early risk detection, and reduced inventory waste. Our commitment lies in tackling challenges such as delays, shortages, and rising costs in clinical trials.

Role Overview

Boston Insights is seeking a Lead Data Engineer specializing in Microsoft Fabric to join our dynamic team. The Lead Data Engineer will be instrumental in designing, implementing, and managing scalable data solutions. This is hands-on technical leadership role where you will architect scalable data pipelines, build Lakehouse and Warehouse structures, and drive data quality and governance across our multi-tenant SaaS platform.

You will work closely with the product and architecture teams to deliver analytics-ready data that powers real-time clinical supply chain intelligence for global pharmaceutical companies.

Key Responsibilities

• Design and build end-to-end data ingestion and transformation pipelines using Microsoft Fabric Data

Factory, Dataflows Gen2, Spark, and PySpark notebooks

• Architect and maintain Lakehouse and Warehouse structures within Microsoft Fabric, implementing

medallion architecture (Bronze, Silver, Gold layers)

• Manage OneLake file structures, Delta tables, partitioning strategies, and storage governance for a multi-

tenant SaaS environment

• Develop incremental loading frameworks, data quality validation, and error handling mechanisms
• Optimize query performance, schema design, and data models to support Power BI semantic models and

real-time analytics

• Implement CI/CD pipelines for data engineering artifacts using Azure DevOps and Git integration
• Collaborate with Power BI developers, Power Apps developers, and data analysts to ensure seamless data

delivery for reporting and application layers

• Support multi-tenant workspace isolation (workspace-per-customer model on shared Fabric capacity)

and enforce tenant-level data security

• Build and maintain integrations with external systems including ERP, MES, WMS, CTMS, RTSM/IRT, and

3PL platforms

• Produce and maintain comprehensive technical documentation covering data models, pipeline logic,

workspace structure, and governance policies

• Mentor junior data engineers, establish coding standards, and conduct code reviews
• Stay current with Microsoft Fabric updates, new features, and best practices to continuously improve the

platform

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a

related field

• 4–5+ years of hands-on experience in data engineering, with at least 1–2 years working with Microsoft

Fabric

• Strong proficiency in SQL, PySpark, Python, and Delta Lake
• Deep experience with Microsoft Fabric components: Lakehouse, Warehouse, Data Factory, Dataflows

Gen2, Notebooks, and OneLake

• Solid understanding of ETL/ELT patterns, data modeling (star schema, snowflake schema), and data

warehousing concepts

• Experience with Azure services: Azure Data Lake, Azure Key Vault, Azure DevOps, Microsoft Entra ID
• Proficiency in Power BI integration — semantic models, datasets, and DirectLake mode
• Experience with CI/CD workflows for data engineering (Git integration, deployment pipelines)
• Strong understanding of data governance, security, and compliance in multi-tenant environments
• Excellent written and verbal communication skills; ability to produce clear technical documentation

Preferred Qualifications

• Microsoft Certified: Fabric Data Engineer Associate (DP-700) certification
• Experience in pharmaceutical, life sciences, or healthcare supply chain domains
• Familiarity with clinical trial data systems (CTMS, RTSM/IRT, CDISC standards)
• Experience with Power Apps, Power Automate, or Power Pages development
• Exposure to AI/ML integration within Microsoft Fabric (Synapse Data Science, notebooks)
• Experience architecting multi-tenant SaaS data platforms
• Knowledge of Kusto Query Language (KQL) and Real-Time Analytics in Fabric

What We Offer

• Opportunity to build a category-defining product in clinical supply chain technology from the ground up
• Direct impact on global pharmaceutical supply chains and patient outcomes
• Work with cutting-edge Microsoft Fabric and Azure technologies
• Collaborative, startup-paced environment with significant ownership and growth potential
• Competitive compensation and benefits
• Remote-first work culture with flexibility