Senior Data Engineer (Databricks / AWS)

Senior Data Engineer (Databricks / AWS)

Position Type: Full-time, permanent

Location: Downtown Toronto, ON (Hybrid, 3 days/week)

Compensation: CAD 120,000 - 150,000

AI use during the recruitment process: Yes

We are seeking a highly skilled Senior Data Engineer to join our client's growing data engineering team. In this role, you will contribute to an enterprise-wide Centralized Data Platform (CDP) built on Databricks, helping to design, build, and optimize scalable data infrastructure within an AWS cloud environment.

You will collaborate closely with data scientists, analysts, and business stakeholders to transform raw financial data into high-quality, secure, and actionable insights. This is an opportunity to work with cutting-edge technologies while solving complex data challenges in a global financial services environment.

Key Responsibilities:

• Design and develop scalable Databricks solutions leveraging Lakehouse architecture
• Build, optimize, and maintain ETL/ELT pipelines
• Develop structured streaming pipelines for real-time data processing
• Configure and optimize Databricks clusters and Spark jobs for performance and cost efficiency
• Utilize Delta Live Tables for data ingestion and transformation workflows
• Implement Unity Catalog and IAM best practices for governance and access control
• Support infrastructure provisioning and management using Terraform
• Participate in Agile/Scrum ceremonies and collaborate across cross-functional teams
• Implement monitoring solutions to ensure pipeline performance and data quality
• Contribute to code reviews, documentation, and knowledge-sharing initiatives

Required Qualifications:

• 6+ years of experience in data engineering
• 2+ years of hands-on experience with Databricks
• Strong proficiency in Python and Spark
• Demonstrated experience leveraging AI in development workflows
• Proven experience working with AWS (S3, Glue, Lambda) or similar cloud platforms
• Deep understanding of data modeling and advanced SQL
• Experience with Delta Lake and Lakehouse architecture
• Strong knowledge of ETL/ELT principles and best practices
• Experience with version control systems (Git)
• Proven ability to optimize and scale data pipelines
• Excellent analytical, problem-solving, communication, and collaboration skills

Nice to Have

• Financial services industry experience
• Experience working across multiple cloud providers
• Knowledge of AI/ML implementation patterns
• API development experience
• Real-time data processing experience
• Familiarity with data governance frameworks

What Sets You Apart

• Passion for emerging technologies and continuous learning
• Experience implementing complex enterprise data solutions
• Strong technical documentation skills
• Experience using AI tools to enhance productivity
• Ability to mentor and support junior team members

Technical Environment

• Primary Platform: Databricks
• Cloud Platform: AWS (S3, Glue, Lambda)
• Languages: Python, SQL
• Tools: Delta Lake, Unity Catalog, Git
• Additional Focus Areas: Real-time processing, API integrations

The indicated pay range for this position is a good-faith estimate based on the qualifications necessary for the position, including experience, training, and other considerations permitted by law. Additionally, it is emphasized that the pay band mentioned herein is the one established by the client company. Factors that may be used when making an offer may include a candidate’s skills, experience and geographic location, the expected quality and quantity of work. Most candidates will start at the bottom half of the pay range, with the upper end reserved for candidates with extensive experience and skills and who live in geographic markets commanding a higher starting pay. An employee’s pay history will not be a contributing factor where prohibited by local law.

This information is subject to change and serves as a general guideline for compensation discussions. Actual offers may vary based on specific circumstances and company policies.

#cws

Job Types: Full-time, Permanent

Pay: $120,000.00-$150,000.00 per year

Benefits

• Dental care
• Paid time off
• RRSP match

Work Location: Hybrid remote in Toronto, ON M5C 2B8