Shift (shift.com.au) Logo

Shift (shift.com.au)

Senior Data Engineer

Posted Yesterday
Be an Early Applicant
Hybrid
Sydney, New South Wales, AUS
Senior level
Hybrid
Sydney, New South Wales, AUS
Senior level
Own and evolve Shift’s Azure Databricks data platform, including engineering standards, CI/CD, testing, monitoring, governance, scalable ELT pipelines, and cost optimization. Lead the data workstream for a core lending system migration, build trusted dimensional models, improve data quality and freshness, and mentor junior engineers. Partner with analysts, data scientists, product managers, and engineering teams to deliver reliable self-service data products.
The summary above was generated by AI

Who is Shift?

Shift provides business finance to small, established and corporate sized businesses in Australia, including Business Overdraft, Equipment Line, Asset Finance, Term Loan and Trade Account. Founded in 2014, Shift has funded more than 30,000 Australian businesses.

At Shift, our unique approach to product innovation combined with our collaborative culture means you can build your career in a supportive environment. You’ll be joining a diverse team of over 300 people who are always looking to deliver better outcomes for Australian businesses.

About the role:

Want to put your stamp on a data platform? You’ll own our Azure Databricks platform, set its standards and roadmap, and lead the data side of a core lending system migration. You’ll be the senior engineer in a small team, with two junior engineers to develop and plenty of room to shape how data works at Shift.

What you’ll do:

  • Own the data platform roadmap and engineering standards, including CI/CD, separate environments, automated testing, monitoring and access governance through Unity Catalog

  • Design and lead the build of robust, scalable ELT pipelines on Azure Databricks

  • Lead the data workstream for our core lending platform migration: remap pipelines and models, and reconcile outputs before and after cutover alongside Engineering

  • Build and extend a curated, dimensional data model that becomes the default place the business goes for trusted, self-serve data

  • Lift data quality, freshness and alerting so issues are caught before the business notices them

  • Keep the platform cost-efficient through cluster optimisation, job orchestration and smart use of compute

  • Mentor and develop our junior data engineers through code review, pairing and clear standards

  • Partner with analysts, data scientists, product managers and Engineering to turn business needs into data products people trust

What you’ll bring:

  • 5+ years in data engineering, with deep hands-on experience on Azure Databricks (Spark, Delta Lake, Unity Catalog)

  • Strong Python and SQL, plus experience with CI/CD and deploying pipelines as code (Azure DevOps or GitHub, Databricks Asset Bundles)

  • A track record of lifting standards on an existing platform, not just building something new on a clean slate

  • Solid dimensional modelling experience (star schemas, conformed dimensions, medallion architecture)

  • Experience mentoring engineers and leading technical decisions through design and code review

  • The ability to explain technical direction clearly to non-technical stakeholders, and an instinct to work across teams rather than in a silo

  • Experience leading the data side of a core system migration is a strong plus

Key benefits:

  • Hybrid working: we’re headquartered in North Sydney with state-based workplaces and offer a hybrid work policy. Our offices have spaces for collaboration, socialising and focused work.

  • Parental leave: up to 26 weeks paid.

  • Strategy Week: twice a year the whole team comes together to hear the business strategy and product and technology roadmap first-hand, with fortnightly all-company business updates in between.

  • Flat structure: everyone can learn from each other and senior leaders across the business.

  • Wellbeing: benefits supporting your physical, psychological and financial wellbeing, from a day off on your birthday to end of trip facilities.

Additional information:
If you don’t meet all the qualifications or skills but think you’d be a great fit, please apply anyway.

Learn more about working at Shift at shift.com.au/careers.

#LI-Hybrid

Similar Jobs

4 Days Ago
In-Office
Sydney, New South Wales, AUS
Senior level
Senior level
Information Technology • Marketing Tech • Transportation • Travel • Financial Services
Develop and operate scalable, reliable data platforms and pipelines across their full lifecycle. The role involves Snowflake data engineering, cloud architecture, dbt transformations, Airflow orchestration, SQL and Python development, infrastructure as code, data quality, security, governance, and cost optimization. The engineer will translate business requirements into technical solutions, provide design and code leadership, mentor engineers, collaborate with architects and product teams, and support enterprise data products across multiple business domains.
Top Skills: AlationApache AirflowApache KafkaAPIsAWSAws Managed Workflows For Apache AirflowAzureCloudabilityDatadogDbt CoreGCPPower BIPythonSaas PlatformsSnowflakeSnowflake DcmSnowflake OpenflowSQLTerraform
12 Days Ago
In-Office
North Sydney, Sydney, New South Wales, AUS
Senior level
Senior level
Digital Media • News + Entertainment
Designs and maintains scalable data pipelines and models, centralizes enterprise data in a warehouse, and processes hundreds of millions of daily events. Leads technical solution design, promotes coding, DevOps, metadata, and testing standards, and integrates AI tools into engineering workflows. Collaborates with Product, Cyber Security, Platform Engineering, and stakeholders to deliver data publishing solutions, resolve incidents, document systems, and shape requirements.
Top Skills: Ai ToolsApache AirflowAWSAzureCi/CdCloud BuildConcourseData PipelinesData WarehousingDbtDevOpsGCPInfrastructure As CodeOlapOltpPythonSQLTerraform
13 Days Ago
Hybrid
North Sydney, New South Wales, AUS
Senior level
Senior level
Healthtech • Professional Services • Social Impact
Lead the design, development, optimization, and support of enterprise data platforms using Microsoft Fabric. Build scalable batch and near-real-time pipelines, lakehouses, warehouses, semantic models, and data transformations. Drive governance, security, performance, disaster recovery, and platform modernization while supporting production environments. Provide technical leadership, mentoring, stakeholder engagement, troubleshooting, and continuous improvement across enterprise data and analytics initiatives.
Top Skills: SparkAzureAzure Data FactoryAzure SqlData WarehouseDataflows Gen2Infrastructure As CodeIso 27001ItilLakehouseMedallion ArchitectureMicrosoft FabricMicrosoft Sql ServerOnelakePower BIPowershellPysparkPythonSemantic ModelsSpark NotebooksSQL

What you need to know about the Sydney Tech Scene

From opera to comedy shows, the Sydney Opera House hosts more than 1,600 performances a year, yet its entertainment sector isn't the only one taking center stage. The city's tech sector has earned a reputation as one of the fastest-growing in the region. More specifically, its IT sector stands out as the country's third-largest, growing at twice the rate of overall employment in the past decade as businesses continue to digitize their operations to stay competitive.

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account