XPT Software Australia Pty Ltd Jobs

Data Engineer with GCP

XPT Software Australia Pty Ltd

Data Engineer with GCP

Posted 2 Days Ago

Be an Early Applicant

In-Office

Sydney, New South Wales, AUS

Mid level

In-Office

Sydney, New South Wales, AUS

Mid level

Design, develop, and maintain scalable batch and real-time data pipelines on Google Cloud. Build ETL/ELT workflows, data lakes, and warehouses using BigQuery, Dataflow, Composer, Pub/Sub, Python, PySpark, and Apache Beam. Perform data modeling, quality validation, monitoring, performance and cost optimization, while supporting security, governance, and collaboration with analytics and business teams.

The summary above was generated by AI

Key Responsibilities

Design, develop, and maintain scalable data pipelines using GCP services.
Build and optimise ETL/ELT workflows for batch and real-time data processing.
Work with Google BigQuery for data warehousing, SQL development, and query optimisation.
Develop data processing solutions using Python, PySpark, and Apache Beam.
Implement data integration using Cloud Dataflow, Cloud Composer, and Pub/Sub.
Build and manage data lakes and data warehouses on Google Cloud.
Implement data quality checks, data validation, and monitoring.
Optimise pipeline performance, scalability, and cost efficiency.
Collaborate with data analysts, data scientists, architects, and business teams.
Ensure data security, governance, and compliance best practices.

Mandatory Technical Skills

Strong experience in Google Cloud Platform (GCP).
Hands-on experience with BigQuery.
Strong programming skills in Python and SQL.
Experience developing ETL/ELT data pipelines.
Experience with Cloud Dataflow and/or Apache Beam.
Knowledge of Cloud Storage (GCS).
Experience with data modelling and data warehousing.
Experience with batch and real-time data processing.

Similar Jobs

NCS Group Australia

Data Engineer

19 Days Ago

Hybrid

Sydney, New South Wales, AUS

Entry level

Artificial Intelligence • Cloud • Information Technology • Consulting • Cybersecurity • Big Data Analytics

Build and maintain secure, scalable real-time and batch data pipelines on Google Cloud. Develop transformation workflows, orchestrate cloud services, integrate upstream data sources, troubleshoot ingestion issues, and implement governance controls including data masking, redaction, row-level and column-level security, taxonomies, and policy tags.

Top Skills: BigQueryCloud RunDataflowDataformEventarcGoogle Cloud PlatformWorkflows

XPT Software Australia Pty Ltd

Data Engineer

One Month Ago

In-Office

Sydney, New South Wales, AUS

Entry level

Information Technology • Software • Consulting

Develop and maintain scalable batch and streaming data pipelines using native GCP services. Design data warehouses and models in BigQuery, write Python and SQL, and use Apache Beam or Spark for data processing. Preferred experience includes AWS or Azure integration, Kafka, containerization with Docker and Kubernetes, and GCP data governance, security, IAM, and compliance.

Top Skills: Apache BeamSparkAWSAzureBigQueryCloud StorageDataflowDockerGkeGoogle Cloud PlatformIamKafkaKubernetesPub/SubPythonSQL

NCS Group Australia

Data Engineer

3 Months Ago

Hybrid

Sydney, New South Wales, AUS

Mid level

Artificial Intelligence • Cloud • Information Technology • Consulting • Cybersecurity • Big Data Analytics

Design, build, and maintain petabyte-scale ETL/ELT data pipelines and BigQuery data models on GCP. Collaborate with data scientists and engineers to ensure data quality, implement CI/CD for pipelines, monitor production systems, and recommend new cloud data technologies. Support streaming data processing and infrastructure-as-code for secure, scalable solutions.

Top Skills: Apache AirflowApache BeamApp EngineBashBigQueryCi/CdCloud ComposerCloud StorageCompute EngineDataflowDataopsDnsDockerFirewallGCPGitKubernetesKubernetes EngineLoad BalancingPub/SubPythonTerraformVpc

What you need to know about the Sydney Tech Scene

From opera to comedy shows, the Sydney Opera House hosts more than 1,600 performances a year, yet its entertainment sector isn't the only one taking center stage. The city's tech sector has earned a reputation as one of the fastest-growing in the region. More specifically, its IT sector stands out as the country's third-largest, growing at twice the rate of overall employment in the past decade as businesses continue to digitize their operations to stay competitive.