We are seeking a Senior DataOps/Cloud Data Engineer to design, build, and optimize high-performance data pipelines within a sophisticated cloud environment. This role is focused on the end-to-end automation of ETL/ELT processes, utilizing modern platforms like Databricks, Unity Catalog, and Azure Data Factory. You will be responsible for the migration and integration of very large datasets (VLD) across diverse platforms, ensuring data security, scalability, and optimal performance for both OLAP and OLTP environments.
...
Rate: $533.22-$569.16/hr
Advantages
Cutting-Edge Tech: Hands-on work with Databricks, Microsoft Fabric, and PySpark in an enterprise setting.
Short-Term High Impact: A focused 4-month contract with a strong potential for extension based on performance.
Architectural Influence: Lead the implementation of modern Data Lakehouse and Medallion architectures.
DataOps Leadership: Drive best practices in CI/CD and automation for Very Large Data (VLD) environments.
Toronto-Based: Consistent professional engagement at a central downtown office location.
Responsibilities
Pipeline Engineering: Design, develop, and automate complex data pipelines from source to end-user using Databricks and Delta Live Tables (DLT).
Platform Development: Build unified data platforms utilizing Unity Catalog and manage cloud data services including repositories, Data Lakes, and Lakehouses.
Data Migration: Lead data migration and integration efforts across disparate platforms, including Oracle to Azure Data Lake, Databricks, and Microsoft Fabric.
Model Design: Implement fact/dimension models and data mapping for enterprise-level data lakehouse solutions following the Medallion Architecture.
Automation & DevOps: Implement DataOps principles through CI/CD pipelines (Azure DevOps Git integration) and data provisioning automation.
Performance Tuning: Monitor and tune data pipeline performance, including query optimization, troubleshooting, failover, and recovery support.
Collaboration: Elicit requirements from stakeholders and work closely with digital product teams to ensure data exchange and security standards are met.
Qualifications
Core Experience: 8+ years of experience in data engineering, specifically designing and optimizing complex ETL/ELT processes.
Databricks Expertise: Proficient in Databricks, Delta Live Tables, and Unity Catalog.
Cloud & Tools: Extensive experience with Azure Data Factory, Azure Data Lake, and Azure DevOps for CI/CD.
Database Mastery: Expert knowledge of relational databases (Oracle, MySQL, SQL Server) and advanced SQL (PL/SQL, T-SQL).
Big Data Frameworks: Advanced skills in PySpark and scripting languages like Python, Scala, or Unix Shell.
Modeling: Proven ability in relational and dimensional data modelling for enterprise data warehouses.
Communication: Strong analytical and interpersonal skills to present technical solutions and manage stakeholder expectations.
Summary
If you're interested in the Senior DataOps/Cloud Data Engineer role based in Toronto, we encourage you to apply online at www.randstad.ca. Only qualified candidates will be contacted for the next steps. We look forward to hearing from you!
Randstad Canada is committed to fostering a workforce reflective of all peoples of Canada. As a result, we are committed to developing and implementing strategies to increase the equity, diversity and inclusion within the workplace by examining our internal policies, practices, and systems throughout the entire lifecycle of our workforce, including its recruitment, retention and advancement for all employees. In addition to our deep commitment to respecting human rights, we are dedicated to positive actions to affect change to ensure everyone has full participation in the workforce free from any barriers, systemic or otherwise, especially equity-seeking groups who are usually underrepresented in Canada's workforce, including those who identify as women or non-binary/gender non-conforming; Indigenous or Aboriginal Peoples; persons with disabilities (visible or invisible) and; members of visible minorities, racialized groups and the LGBTQ2+ community.
Randstad Canada is committed to creating and maintaining an inclusive and accessible workplace for all its candidates and employees by supporting their accessibility and accommodation needs throughout the employment lifecycle. We ask that all job applications please identify any accommodation requirements by sending an email to accessibility@randstad.ca to ensure their ability to fully participate in the interview process.
This posting is for existing and upcoming vacancies.
show more
We are seeking a Senior DataOps/Cloud Data Engineer to design, build, and optimize high-performance data pipelines within a sophisticated cloud environment. This role is focused on the end-to-end automation of ETL/ELT processes, utilizing modern platforms like Databricks, Unity Catalog, and Azure Data Factory. You will be responsible for the migration and integration of very large datasets (VLD) across diverse platforms, ensuring data security, scalability, and optimal performance for both OLAP and OLTP environments.
Rate: $533.22-$569.16/hr
Advantages
Cutting-Edge Tech: Hands-on work with Databricks, Microsoft Fabric, and PySpark in an enterprise setting.
Short-Term High Impact: A focused 4-month contract with a strong potential for extension based on performance.
Architectural Influence: Lead the implementation of modern Data Lakehouse and Medallion architectures.
DataOps Leadership: Drive best practices in CI/CD and automation for Very Large Data (VLD) environments.
Toronto-Based: Consistent professional engagement at a central downtown office location.
Responsibilities
...
Pipeline Engineering: Design, develop, and automate complex data pipelines from source to end-user using Databricks and Delta Live Tables (DLT).
Platform Development: Build unified data platforms utilizing Unity Catalog and manage cloud data services including repositories, Data Lakes, and Lakehouses.
Data Migration: Lead data migration and integration efforts across disparate platforms, including Oracle to Azure Data Lake, Databricks, and Microsoft Fabric.
Model Design: Implement fact/dimension models and data mapping for enterprise-level data lakehouse solutions following the Medallion Architecture.
Automation & DevOps: Implement DataOps principles through CI/CD pipelines (Azure DevOps Git integration) and data provisioning automation.
Performance Tuning: Monitor and tune data pipeline performance, including query optimization, troubleshooting, failover, and recovery support.
Collaboration: Elicit requirements from stakeholders and work closely with digital product teams to ensure data exchange and security standards are met.
Qualifications
Core Experience: 8+ years of experience in data engineering, specifically designing and optimizing complex ETL/ELT processes.
Databricks Expertise: Proficient in Databricks, Delta Live Tables, and Unity Catalog.
Cloud & Tools: Extensive experience with Azure Data Factory, Azure Data Lake, and Azure DevOps for CI/CD.
Database Mastery: Expert knowledge of relational databases (Oracle, MySQL, SQL Server) and advanced SQL (PL/SQL, T-SQL).
Big Data Frameworks: Advanced skills in PySpark and scripting languages like Python, Scala, or Unix Shell.
Modeling: Proven ability in relational and dimensional data modelling for enterprise data warehouses.
Communication: Strong analytical and interpersonal skills to present technical solutions and manage stakeholder expectations.
Summary
If you're interested in the Senior DataOps/Cloud Data Engineer role based in Toronto, we encourage you to apply online at www.randstad.ca. Only qualified candidates will be contacted for the next steps. We look forward to hearing from you!
Randstad Canada is committed to fostering a workforce reflective of all peoples of Canada. As a result, we are committed to developing and implementing strategies to increase the equity, diversity and inclusion within the workplace by examining our internal policies, practices, and systems throughout the entire lifecycle of our workforce, including its recruitment, retention and advancement for all employees. In addition to our deep commitment to respecting human rights, we are dedicated to positive actions to affect change to ensure everyone has full participation in the workforce free from any barriers, systemic or otherwise, especially equity-seeking groups who are usually underrepresented in Canada's workforce, including those who identify as women or non-binary/gender non-conforming; Indigenous or Aboriginal Peoples; persons with disabilities (visible or invisible) and; members of visible minorities, racialized groups and the LGBTQ2+ community.
Randstad Canada is committed to creating and maintaining an inclusive and accessible workplace for all its candidates and employees by supporting their accessibility and accommodation needs throughout the employment lifecycle. We ask that all job applications please identify any accommodation requirements by sending an email to accessibility@randstad.ca to ensure their ability to fully participate in the interview process.
This posting is for existing and upcoming vacancies.
show more
