Data Engineer

Key Responsibilities: Design, develop, and maintain robust ETL pipelines using PySpark, Python, and SQL. Implement and manage data solutions on AWS (S3, Glue, EMR, Redshift, Lambda, etc.). Collaborate with data scientists, analysts, and business stakeholders to deliver reliable datasets for reporting and advanced analytics. Optimize data workflows for performance, scalability, and cost efficiency. Ensure data quality, governance, and compliance across all systems. Monitor, troubleshoot, and improve existing data pipelines and infrastructure. Document technical processes and contribute to best practices in data engineering. Requirements: Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field. Proven experience as a Data Engineer (5+ years preferred). Strong proficiency in Python and PySpark for data processing. Advanced knowledge of SQL for querying, optimization, and data modeling. Hands-on experience with AWS services (S3, Glue, EMR, Redshift, Lambda, CloudWatch). Experience with workflow orchestration tools (Airflow, Step Functions, etc.) is a plus. Solid understanding of distributed computing, big data frameworks, and data lake/warehouse concepts. Strong problem-solving skills and ability to work in cross-functional teams. Good To Have Skills: Experience with Apache Kafka, Kinesis, or other streaming data platforms. Familiarity with Docker and container orchestration using Kubernetes/EKS. Knowledge of Delta Lake, Iceberg, or Hudi for modern data lake architectures. Experience with CI/CD pipelines (GitHub Actions, CodePipeline, Jenkins, etc.). Hands-on experience with Terraform or CloudFormation for IaC (Infrastructure as Code). Understanding of data security, encryption, IAM policies, and access control best practices