The Cloud Data Platform Engineer plays a critical role in supporting the development and operation of cloud infrastructure for the UIA platform, enabling real-time analytics, streaming, lakehouse architectures, and ML/AI-driven workloads. This role is responsible for the end-to-end lifecycle of the platform, including development, stability, scalability, and operations. It emphasizes automation, cost efficiency, and reliability to support enterprise-grade data workloads across the organization.

On UIA, the Cloud Data Platform Engineer is responsible for the end-to-end lifecycle of platform components.

The platform has a strong focus on automation and the provisioning of self-service real-time data analytics environments on Microsoft Azure. A key responsibility is enabling scalable and standardized Azure-based infrastructure, with Azure Databricks as a central component, along with the necessary surrounding services.

Engineers are expected to design, implement, and operate Azure-native and Databricks cloud modules to enable real-time and batch data analytics workloads, integrating these services for the purposes of a Customer Data Platform (CDP).

Responsibilities

• Implement and manage cloud infrastructure on Microsoft Azure with automated deployments and updates using Infrastructure as Code (IaC), primarily with Terraform.
• Enable self-service capabilities for provisioning and managing platform resources, allowing users to efficiently consume data platform services without deep infrastructure knowledge.
• Design, provision, and manage Azure Databricks workspaces, ensuring alignment with best practices, standardization, and seamless integration into the broader Azure environment.
• Implement and integrate supporting Azure services required for a scalable data platform, including storage, networking, security, and data-related services.
• Integrate and manage streaming infrastructure components such as Kafka or
• Azure Event Hubs within the platform, ensuring they are accessible and usable for platform consumers.
• Ensure security, compliance, and governance across all platform components, including adherence to tagging policies for security, budgeting, and reporting.
• Implement observability and integrate platform components into central monitoring solutions.
• Ensure the stability and reliability of the platform by participating in on-call rotations for critical incidents.
• Conduct root cause analysis (RCA) for incidents, performance issues, and outages within platform workloads, documenting findings, recommending preventive measures, and implementing corrective actions to enhance system reliability.
• Design and own platform components within a feature team, collaborating with other engineers to deliver scalable and reliable data platform capabilities.

Requirements

• Proficient and clear English language skills.
• Very good conversational and collaboration skills: listening to nuances, responding concisely, and expressing opinions effectively.
• Strong mindset for building secure, reliable, cost-efficient, and scalable services across the full platform lifecycle (design, build, operate).
• Ability to quickly explore and work with new technologies, even with limited documentation.
• At least 2 years of experience as a Cloud Engineer or Platform Engineer in Azure or GCP (preferably Azure).
• Strong expertise in Terraform for Infrastructure-as-Code and cloud resource management.
• Experience with Azure DevOps or GitHub Enterprise and Git for source control, pull requests, and collaboration.
• Strong problem-solving skills and ability to operate in complex environments.
• Solid understanding of Microsoft Azure, including the ability to design and manage cloud infrastructure and services.
• Hands-on experience with Azure Databricks, including provisioning, configuration, and applying best practices for workspace setup and integration.
• Ability to design, implement, and maintain CI/CD pipelines, with strong scripting skills (e. g., Bash, Python) to automate build, deployment, and operational workflows across cloud environments.
• Knowledge of at least one programming language for automation - Python, Golang.
• Experience with Azure data services, specifically: Azure Data Lake Storage (ADLS), Azure Event Hubs
• Understanding of how to integrate streaming platforms such as Kafka or Azure
• Event Hubs into a broader data platform and enable their usage by other teams.
• Experience with Kubernetes.
• Familiarity with GitOps principles and workflows.
• Knowledge of analytics workloads, both Batch and Real-time.
• Deeper experience with Azure Databricks, such as performance optimization, advanced configurations, and operational best practices.
• Experience working with Apache Kafka beyond managed services.
• Understanding of Customer Data Platforms (CDPs) and their role in enabling personalization and marketing use cases.
• Experience with Google Cloud Platform (GCP), enabling potential cross-cloud contributions in the future.

Technical Environment:

• Cloud Provider: Microsoft Azure.
• Data Platform and Processing: Azure Databricks.
• Data Storage: Azure Data Lake Storage (ADLS).
• Streaming and Messaging: Azure Event Hubs, Apache Kafka.
• Networking and Security: Azure Virtual Network, NSGs, Azure Firewall, Entra ID, RBAC, Key Vault.
• Identity and Access Management: Entra ID.
• Infrastructure as Code: Terraform.
• Monitoring and Observability: Elastic Cloud, OpenTelemetry.
• Cost Management / FinOps: Azure Cost Management and amp; Billing, tagging policies, budgeting, and optimization.
• Governance and amp Compliance: Azure Policy, Security Center.