Design and implement scalable cloud-native data architectures, build and optimise ETL and big-data pipelines using Databricks/Spark/Delta Lake, ensure data quality and governance, automate workflows with Python/Scala/SQL, and collaborate with data scientists and analysts to enable AI-driven insights.
Our client is looking for a Sr. Data Engineer who thrives on solving complex data challenges, designing robust architectures, and optimising big data pipelines. You’ll work with cutting-edge cloud and AI solutions, ensuring that data is accessible, scalable, and analytics-ready.
This role blends deep technical expertise, strategic problem-solving, and hands-on implementation. You’ll build and optimise ETL pipelines, data lakes, and data platforms while collaborating with cross-functional teams to drive AI-powered insights.
- Build scalable data architectures – Design and implement high-performance data pipelines and cloud-native solutions.
- Optimise big data processing – Develop efficient and cost-effective data solutions using Databricks, Apache Spark™, and Delta Lake.
- Ensure data quality and governance – Implement best practices for data validation, lineage, and security.
- Drive automation and efficiency – Use Python, Scala, or SQL to automate data workflows and streamline processes.
- Collaborate with cross-functional teams – Work with data scientists, analysts, and engineers to deliver data-driven solutions.
- Stay ahead of the curve – Continuously explore new data engineering trends, AI advancements, and cloud technologies.
Qualifications
- 3+ years of experience in data engineering, big data, or cloud data solutions.
- Proficiency in Python, Scala, or SQL for building scalable data pipelines.
- Experience with Databricks, Apache Spark™, and Delta Lake.
- Familiarity with cloud platforms (AWS, Azure) and modern data architectures.
- Understanding of ETL processes, data lakes, and warehouse solutions.
- Excellent problem-solving skills – Able to tackle complex data challenges with innovative solutions.
- A passion for automation, AI, and optimising data workflows.
Preferred Qualifications
- Databricks Certification (e.g., Databricks Certified Data Engineer or ML Associate).
- Experience with real-time streaming architectures (Kafka, Kinesis, or similar).
- Knowledge of AI/ML pipelines and MLOps best practices.
- Prior experience in a consulting or client-facing role.
Benefits
- Be part of an AI-native company – We live and breathe cutting-edge AI and data solutions.
- Work on high-impact projects – We partner with industry leaders to solve some of the toughest data challenges.
- Shape the future of data engineering – Be at the forefront of AI-driven transformation.
- Competitive compensation – We offer a generous salary and performance-based bonuses.
- Flexible work culture – Hybrid and remote work options to fit your lifestyle.
- Grow with us – Access top-tier learning resources, training, and leadership development programs.
The base pay range for this role is A$125,000 – A$175,000 per year.
Similar Jobs
AdTech • Beauty • Marketing Tech • Retail • Pharmaceutical
Lead design, development, deployment, and continuous improvement of cloud-native data and analytics solutions on Azure. Act as a hands-on technical lead building scalable, secure end-to-end data platforms using Spark/Databricks, Python/PySpark, and orchestration tools. Ensure CI/CD, DevOps, automation, code quality, mentor team members, resolve complex technical issues, and collaborate with stakeholders and vendors to drive platform evolution.
Top Skills:
AgileAirflowApi IntegrationsAzureAzure Data FactoryCi/CdDatabricksDevOpsPysparkPythonSparkSQLUnit Testing
Agency • Information Technology • Professional Services • Manufacturing
The Sr Data Engineer will optimize and refactor data pipelines, manage data ingestion from APIs, and support product engineering teams. Responsibilities include design best practices, enhance the modern data stack, and troubleshoot data issues, ensuring resilience and operational excellence.
Top Skills:
DockerEltGitPythonSQL
Artificial Intelligence • Fintech • Machine Learning • Software • App development • Conversational AI • Generative AI
The Senior Data Engineer will design and maintain data pipelines, implement Medallion architecture, develop ETL/ELT processes, and improve data quality while collaborating with analysts to derive insights from data.
Top Skills:
AirbyteAirflowDbtPythonSnowflakeSQL
What you need to know about the Sydney Tech Scene
From opera to comedy shows, the Sydney Opera House hosts more than 1,600 performances a year, yet its entertainment sector isn't the only one taking center stage. The city's tech sector has earned a reputation as one of the fastest-growing in the region. More specifically, its IT sector stands out as the country's third-largest, growing at twice the rate of overall employment in the past decade as businesses continue to digitize their operations to stay competitive.
