T&T- Engineering - Senior Consultant - AWS Data engineer - Bengaluru

T&T- Engineering - Senior Consultant - AWS Data engineer DBT- Bengaluru
• Job requisition ID : 106097 
• Location: Bengaluru
• Entity: Deloitte Touche Tohmatsu India LLP 

The Team

Deloitte’s Technology & Transformation practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.            

Your work profile

We are looking for Data Engineering professionals with strong expertise. This role requires 5-10 years' experience in AWS ecosystems, strong dimensional modelling skills, and expertise in implementing data pipeline, data quality, and governance using modern data stack tools. 

Key Skills & Competencies

• Design, develop, and maintain AWS Glue ETL jobs using PySpark / Python.
• Build and manage AWS Glue Crawlers, Data Catalogs, and metadata structures.
• Implement batch and near-real-time ingestion pipelines into Amazon S3–based data lakes.
• Design efficient partitioning, schema management, and job performance tuning strategies.
• Handle error handling, retry logic, logging, and operational robustness of Glue jobs.
• Develop Glue Workflows and Triggers to orchestrate multi-step pipelines.

•  Build data pipelines using:
• Amazon S3 (raw, curated, consumption layers)
• AWS Glue
• Athena and/or Redshift for query and analytics access
• Apply data modeling practices aligned to analytics use cases (curated / consumption-ready data).
• Support structured, semi-structured, and JSON-based data sources.

• Implement data quality checks and validation within Glue pipelines.
• Ensure schema evolution, backward compatibility, and controlled data changes.
• Apply IAM roles, permissions, and encryption standards for data access.
• Work with governance teams to support lineage, cataloging, and audit requirements.

• Monitor Glue jobs and data pipelines using CloudWatch logs and metrics.
• Perform troubleshooting, root cause analysis, and pipeline optimization.
• Ensure SLAs for critical data pipelines and analytics feeds.

• Partner closely with data product owners, BI teams, and downstream consumers.
• Translate business and analytics requirements into scalable technical solutions.
• Produce clear documentation for pipelines, datasets, and operational runbooks.

 Preferred Qualifications

• Education - B. Tech / BE in Computer Science or Information Technology.

Success Metric:

• 5+ years of experience in Data Engineering roles.
• Strong hands-on experience with AWS Glue (Jobs, Crawlers, Catalog, Workflows).
• Proficiency in Python and PySpark for ETL development.
• Solid experience with Amazon S3–based data lake architectures.
• Experience using Athena and/or Redshift for analytics consumption.
• Core Tools: Expert-level AWS Databricks, PySpark, Spark SQL, Airflow and dbt.
• Cloud & ETL: Extensive experience with AWS and Databricks or similar cloud environments in medallion architecture
• Languages: Strong Python and SQL skills.
• Engineering Practices: Git/GitHub, CI/CD, and data modeling (Kimball methodology).
• Data Governance: Experience with Unity Catalog or similar data cataloging/governance tools.

Location and way of working:

• Base location: Bengaluru
• This profile involves travelling to client locations