Design and implement scalable cloud-native data architectures, build and optimise ETL and big-data pipelines using Databricks/Spark/Delta Lake, ensure data quality and governance, automate workflows with Python/Scala/SQL, and collaborate with data scientists and analysts to enable AI-driven insights.
Our client is looking for a Sr. Data Engineer who thrives on solving complex data challenges, designing robust architectures, and optimising big data pipelines. You’ll work with cutting-edge cloud and AI solutions, ensuring that data is accessible, scalable, and analytics-ready.
This role blends deep technical expertise, strategic problem-solving, and hands-on implementation. You’ll build and optimise ETL pipelines, data lakes, and data platforms while collaborating with cross-functional teams to drive AI-powered insights.
- Build scalable data architectures – Design and implement high-performance data pipelines and cloud-native solutions.
- Optimise big data processing – Develop efficient and cost-effective data solutions using Databricks, Apache Spark™, and Delta Lake.
- Ensure data quality and governance – Implement best practices for data validation, lineage, and security.
- Drive automation and efficiency – Use Python, Scala, or SQL to automate data workflows and streamline processes.
- Collaborate with cross-functional teams – Work with data scientists, analysts, and engineers to deliver data-driven solutions.
- Stay ahead of the curve – Continuously explore new data engineering trends, AI advancements, and cloud technologies.
Qualifications
- 3+ years of experience in data engineering, big data, or cloud data solutions.
- Proficiency in Python, Scala, or SQL for building scalable data pipelines.
- Experience with Databricks, Apache Spark™, and Delta Lake.
- Familiarity with cloud platforms (AWS, Azure) and modern data architectures.
- Understanding of ETL processes, data lakes, and warehouse solutions.
- Excellent problem-solving skills – Able to tackle complex data challenges with innovative solutions.
- A passion for automation, AI, and optimising data workflows.
Preferred Qualifications
- Databricks Certification (e.g., Databricks Certified Data Engineer or ML Associate).
- Experience with real-time streaming architectures (Kafka, Kinesis, or similar).
- Knowledge of AI/ML pipelines and MLOps best practices.
- Prior experience in a consulting or client-facing role.
Benefits
- Be part of an AI-native company – We live and breathe cutting-edge AI and data solutions.
- Work on high-impact projects – We partner with industry leaders to solve some of the toughest data challenges.
- Shape the future of data engineering – Be at the forefront of AI-driven transformation.
- Competitive compensation – We offer a generous salary and performance-based bonuses.
- Flexible work culture – Hybrid and remote work options to fit your lifestyle.
- Grow with us – Access top-tier learning resources, training, and leadership development programs.
The base pay range for this role is A$125,000 – A$175,000 per year.
Similar Jobs
AdTech • Beauty • Marketing Tech • Retail • Pharmaceutical
Lead the design and development of cloud-based data platforms, pipelines, ETL/ELT processes, data models, warehouses, and lakes using Azure and Databricks. Build integrations, optimize performance, implement governance, monitor pipelines, resolve issues, and create Power BI reports. Collaborate with product managers and technical teams to deliver scalable, reusable data solutions and document processes.
Top Skills:
SparkAzure Data FactoryAzure Data Lake StorageAzure DatabricksAzure DevopsAzure Sql DatabaseAzure Synapse AnalyticsChefCi/CdDatabricks Delta LakeGitAzureMicrosoft Power BiPuppetPysparkPythonScalaSQLSQL Server
AdTech • Beauty • Marketing Tech • Retail • Pharmaceutical
Lead the design and development of cloud-based data platforms using Azure and Databricks. Build and optimize ETL/ELT pipelines, data models, warehouses, lakes, and integrations; implement governance, monitoring, and troubleshooting; and create Power BI reports. Collaborate with product managers and cross-functional teams to deliver scalable, high-quality data solutions, while documenting processes and applying coding and DevOps best practices.
Top Skills:
SparkAzure Data FactoryAzure Data Lake StorageAzure DatabricksAzure DevopsAzure Sql DatabaseAzure Synapse AnalyticsChefCi/CdDatabricks Delta LakeEltETLGitAzureMicrosoft Power BiPuppetPysparkPythonScalaSQLSQL Server
Artificial Intelligence • Fintech • Machine Learning • Software • App development • Conversational AI • Generative AI
Migrate SQL Server data products and analytical workloads to a modern cloud platform. Redesign legacy SQL, ETL processes, and business logic; build scalable pipelines and analytical models; validate data quality; collaborate with stakeholders; and improve platform reliability, performance, cost efficiency, and AI-assisted data engineering automation.
Top Skills:
BigQueryCi/CdCloud ComposerETLGCPGoogle Cloud StorageIamMs Sql ServerPythonSQL
What you need to know about the Sydney Tech Scene
From opera to comedy shows, the Sydney Opera House hosts more than 1,600 performances a year, yet its entertainment sector isn't the only one taking center stage. The city's tech sector has earned a reputation as one of the fastest-growing in the region. More specifically, its IT sector stands out as the country's third-largest, growing at twice the rate of overall employment in the past decade as businesses continue to digitize their operations to stay competitive.
