Manager | GCP Data Engineer | Ahmedabad | Engineering | Data Modernization & Migration
Deloitte
· ✓ Verified company
📍 Ahmedabad, India · On-site · Full-time
About the role
Role Summary We are looking for a GCP Data Engineering Manager / AD with 8 to 15+ years of experience to lead the delivery and optimization of cloud-based data platforms. The role combines strong expertise in GCP data services with hands-on performance engineering for large-scale Spark and Dataproc workloads. Key Responsibilities GCP Data Engineering and Platform Delivery - Design and deliver data solutions using Cloud Run, Cloud Run Functions with Python, Cloud SQL PostgreSQL, Pub/Sub, GCS and BigQuery. - Build and manage orchestration workflows using Cloud Composer / Managed Airflow. - Drive Airflow and Composer optimization, including FinOps, capacity planning and cost governance. - Support Databricks-based data processing and integration. - Design and integrate REST APIs across cloud applications and data platforms. - Provide technical leadership, architecture guidance and delivery oversight to engineering teams. Performance Engineering and Dataproc Optimization - Execute workload test waves and enhance Python-based execution templates. - Troubleshoot Spark, YARN and Dataproc failures using Spark UI and platform diagnostics. - Analyze approximately 60 workload profiles and capture cost-performance benchmarks. - Define N1/N2 mapping strategies and translate custom machine requirements into standard GCP machine types. - Recommend VM sizing, schema design and performance improvements for Big Data workloads. Required Skills - 8 to 15 years in Data Engineering, with strong experience on GCP. - Python, Spark, Dataproc, GCS, BigQuery and Cloud Data Architecture. - Cloud Run, Cloud Functions, Cloud SQL PostgreSQL and Pub/Sub. - Cloud Composer / Airflow administration, optimization and FinOps. - Spark UI debugging, YARN diagnostics and workload benchmarking. - Strong stakeholder management, team leadership and solution design capabilities. B.E/B.tech