Requisition ID: 257434
Join a purpose driven winning team, committed to results, in an inclusive and high-performing culture.
We are looking for a hands‑on Data Engineer with deep expertise in Apache Spark and strong programming skills in Python, Scala, and Java. This role is centered on building high performance, scalable data pipelines and processing large datasets in a distributed environment.
You will work primarily on Spark based data processing running on Azure Databricks, developing production grade code that supports enterprise analytics, reporting, and data products. This is an engineering heavy role for someone who enjoys writing clean, efficient code and optimizing distributed workloads.
Is this role right for you? In this role, you will:
- Design, develop, and maintain large‑scale Spark applications using Python, Scala, and/or Java
- Build and optimize batch and streaming data pipelines in distributed environments
- Write production‑quality Spark code with strong focus on performance, scalability, and reliability
- Optimize Spark jobs (partitioning, caching, shuffles, memory tuning, execution plans)
- Develop reusable Spark frameworks, libraries, and utilities
- Work with structured and semi‑structured data (Parquet, Delta, CSV, JSON)
- Collaborate with platform, analytics, and data science teams to support downstream use cases
- Debug and troubleshoot Spark job failures and performance issues in production
- Follow best practices for code quality, testing, logging, and documentation
Do you have the skills that will enable you to succeed in this role? We'd love to work with you if you have:
- Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience)
- 3+ years of experience as a Data Engineer or in a similar role
- Strong hands‑on experience with Apache Spark
- Proficiency in one or more of the following languages: (Python, Scala or Java)
- Strong understanding of distributed systems and data processing concepts
- Solid experience with SQL for data transformation and analysis
- Experience working with data lakes and columnar formats (e.g., Parquet, Delta Lake)
- Familiarity with Git and standard software engineering practices
- Strong problem‑solving skills and attention to detail
- Experience running Spark on Azure Databricks an asset.
- Experience migrating Spark jobs from on‑prem Hadoop/Cloudera to cloud platforms an asset.
- Familiarity with orchestration tools (e.g., Airflow, Azure Data Factory) an asset.
- Exposure to cloud storage (ADLS Gen2) and cloud security concepts an asset.
What's in it for you?
- Diversity, Equity, Inclusion & Allyship - We strive to create an inclusive culture where every employee is empowered to reach their fullest potential, respected for who they are, and are embraced through bias-free practices and inclusive values across Scotiabank. We embrace diversity and provide opportunities for all employee to learn, grow & participate through our various Employee Resource Groups (ERGs) that span across diverse gender identities, ethnicity, race, age, ability & veterans.
- Accessibility and Workplace Accommodations - We value the unique skills and experiences each individual brings to the Bank, and are committed to creating and maintaining an inclusive and accessible environment for everyone. Scotiabank continues to locate, remove and prevent barriers so that we can build a diverse and inclusive environment while meeting accessibility requirements.
- Upskilling through online courses, cross-functional development opportunities, and tuition assistance.
- Competitive Rewards program including bonus, flexible vacation, personal, sick days and benefits will start on day one.
- Dynamic Ecosystem - Free tea & coffee, universal washrooms, and lots of space for team collaboration.
- Community Engagement - No matter where you choose to work from; we offer opportunities for community engagement & belonging with our various programs.
Location(s): Canada : Ontario : Toronto
Scotiabank is a leading bank in the Americas. Guided by our purpose: "for every future", we help our customers, their families and their communities achieve success through a broad range of advice, products and services, including personal and commercial banking, wealth management and private banking, corporate and investment banking, and capital markets.
At Scotiabank, we value the unique skills and experiences each individual brings to the Bank, and are committed to creating and maintaining an inclusive and accessible environment for everyone. If you require accommodation (including, but not limited to, an accessible interview site, alternate format documents, ASL Interpreter, or Assistive Technology) during the recruitment and selection process, please let our Recruitment team know. If you require technical assistance, please click here. Candidates must apply directly online to be considered for this role. We thank all applicants for their interest in a career at Scotiabank; however, only those candidates who are selected for an interview will be contacted.