Lead Data Engineer

Company Description

Technology is our how. And people are our why. For over two decades, we have been harnessing technology to drive meaningful change.
 
By combining world-class engineering, industry expertise and a people-centric mindset, we consult and partner with leading brands from various industries to create dynamic platforms and intelligent digital experiences that drive innovation and transform businesses.
 
From prototype to real-world impact - be part of a global shift by doing work that matters.

Job Description

Role Overview

At Endava, we believe that harnessing Data & AI is essential for navigating today’s rapidly evolving technology landscape. With a global team of over 500 data and AI professionals, our Data & AI Discipline focuses on real-world impact, delivering sustainable solutions underpinned by proven frameworks and methodologies.

As we expand our capability in Australia, we’re looking for a Lead Data Engineer to help build and scale our local offering. This role is key to orchestrating modern data pipelines, architectures, and governance frameworks that enable our clients to unlock actionable insights, improve decision-making, and drive continuous innovation.

The Lead Data Engineer designs, builds, and optimises robust and scalable data infrastructure. This role ensures that data is not only collected, but also strategically transformed, modelled, governed, and surfaced to enable impactful business decisions. Collaborating with analysts, data scientists, and technical business analysts is central to delivering high-quality solutions aligned with our clients’ goals.

Key Responsibilities

Data Pipeline Development

• Design, build, and maintain real-time and batch data pipelines that enable high-performance analytics.
• Develop ETL and ELT processes to transform, cleanse, and enrich data from diverse sources.
• Optimise pipelines for scalability, reusability, and cost efficiency using modern frameworks and cloud-native tools.

Data Infrastructure & Architecture

• Design and implement data architectures that support secure, reliable, and high-availability storage and access.
• Ensure data infrastructure supports seamless integration across reporting, analytics, and ML workloads.

Data Integration, Transformation & Modelling

• Collaborate with analysts and data scientists to understand source-to-target mappings and design appropriate models.
• Perform data wrangling, modelling, and transformation to enable accurate and timely insights.
• Maintain high standards of data quality, lineage, and documentation.

Collaboration

• Work closely with Data Scientists, Data Analysts, and Technical Business Analysts to ensure business requirements are met.
• Support junior engineers and foster a culture of technical excellence and collaboration.

Governance, Security & Compliance

• Implement and uphold enterprise-grade security and privacy practices, including RBAC, encryption, and compliance (e.g., APP).
• Promote best practices in data stewardship, documentation, and ownership.

Performance & Innovation

• Continuously refine data platforms for reliability, performance, and cost optimisation.
• Stay current with technology trends, recommending new tools or approaches where beneficial.

Qualifications

Key Skills & Competencies

Technical Skills

• Programming: Python, SQL
• Data Orchestration & Transformation: DBT, Airflow, Dagster
• Big Data Processing: Apache Spark, Databricks, Dask
• Cloud Platforms: AWS (Glue, Redshift), Azure (Synapse, Fabric, Data Factory), GCP (BigQuery, Dataflow)
• Storage & Warehousing: Relational (PostgreSQL, SQL Server, Snowflake), NoSQL (MongoDB, Cassandra), Graph (Neo4j, Amazon Neptune)
• DevOps & Automation: Docker, Kubernetes, Terraform, CI/CD for data workflows
• Monitoring & Maintenance: Tools for data observability, lineage tracking, and pipeline health

Architectural Competencies

• Data Modelling: Dimensional, relational, and hierarchical design
• Scalability & Performance: Building high-availability data architectures
• Governance & Compliance: Security models, audit trails, access control

Soft Skills

• Problem Solving: Ability to debug complex data challenges quickly and effectively
• Collaboration: Strong cross-functional communicator and team player
• Leadership: Supports and mentors engineers, promotes engineering best practices

Preferred Industry Experience

• Experience working in regulated industries such as banking, financial services, insurance, or healthcare is highly valued.

Additional Information

Discover some of the global benefits that empower our people to become the best version of themselves:

• Finance: Competitive salary package, share plan, company performance bonuses, value-based recognition awards, referral bonus;  
• Career Development: Career coaching, global career opportunities, non-linear career paths, internal development programmes for management and technical leadership;
• Learning Opportunities: Complex projects, rotations, internal tech communities, training, certifications, coaching, online learning platforms subscriptions, pass-it-on sessions, workshops, conferences;
• Work-Life Balance: Hybrid work and flexible working hours, employee assistance programme;
• Health: Global internal wellbeing programme, access to wellbeing apps;
• Community: Global internal tech communities, hobby clubs and interest groups, inclusion and diversity programmes, events and celebrations.

At Endava, we’re committed to creating an open, inclusive, and respectful environment where everyone feels safe, valued, and empowered to be their best. We welcome applications from people of all backgrounds, experiences, and perspectives—because we know that inclusive teams help us deliver smarter, more innovative solutions for our customers. Hiring decisions are based on merit, skills, qualifications, and potential. If you need adjustments or support during the recruitment process, please let us know.