XPT Software Australia Pty Ltd Jobs

Data Engineer-GCP

XPT Software Australia Pty Ltd

Data Engineer-GCP

Posted 24 Days Ago

Be an Early Applicant

In-Office

Sydney, New South Wales, AUS

Entry level

In-Office

Sydney, New South Wales, AUS

Entry level

Develop and maintain scalable batch and streaming data pipelines using native GCP services. Design data warehouses and models in BigQuery, write Python and SQL, and use Apache Beam or Spark for data processing. Preferred experience includes AWS or Azure integration, Kafka, containerization with Docker and Kubernetes, and GCP data governance, security, IAM, and compliance.

The summary above was generated by AI

Job Details:

Must Have Skills:

 Strong GCP Data Services Expertise - Hands-on experience with BigQuery, Dataflow, Pub/Sub, Cloud Storage, Ability to design scalable data solutions using native GCP tools.

 Experience in Data Pipeline Development (Batch & Streaming) - Building and maintaining ETL/ELT pipelines, Experience with real-time streaming and batch processing.

 Strong Programming & Query Skills - Proficiency in Python and SQL, Experience with data processing frameworks like Apache Beam or Spark

 Understanding of Data Modeling & Warehousing - Strong understanding of data modeling (star/snowflake schemas), Experience designing and optimizing data warehouses (BigQuery)

Nice to Have Skills:

 Experience in Multi-Cloud Exposure : Understanding of cross-cloud data integration and migration, Experience working with AWS or Azure alongside GCP

 Knowledge of Streaming & Messaging Frameworks : Experience with Kafka or advanced Pub/Sub patterns, Knowledge of real-time data architectures

 Basic Understanding of Containerization & Orchestration : Hands-on with Docker and Kubernetes (GKE), Ability to deploy and manage data workloads in containerized environments

 Data Governance & Security : Familiarity with data lineage, cataloging, and governance tools, Understanding of IAM roles, data security, and compliance in GCP.

Similar Jobs

NCS Group Australia

Data Engineer

11 Days Ago

Hybrid

Sydney, New South Wales, AUS

Entry level

Artificial Intelligence • Cloud • Information Technology • Consulting • Cybersecurity • Big Data Analytics

Build and manage scalable, secure, high-throughput data pipelines and infrastructure on Google Cloud. Design BigQuery data models, develop batch and real-time ETL/ELT systems using Dataflow, Apache Beam, Composer, Airflow, and Pub/Sub, and support data quality, governance, CI/CD, monitoring, and troubleshooting. Collaborate with data scientists, analysts, and engineers while implementing Terraform-based infrastructure, secure network architectures, and enterprise cloud solutions.

Top Skills: Apache AirflowApache BeamApp EngineBashBigQueryCi/CdCloud ComposerCloud StorageCompute EngineDataflowDataopsDnsDockerFirewallsGitGoogle Cloud PlatformGoogle Kubernetes EngineKubernetesLoad BalancingPub/SubPythonSubnetsTerraformVpc

NCS Group Australia

Data Engineer

2 Months Ago

Hybrid

Sydney, New South Wales, AUS

Mid level

Artificial Intelligence • Cloud • Information Technology • Consulting • Cybersecurity • Big Data Analytics

Design, build, and maintain petabyte-scale ETL/ELT data pipelines and BigQuery data models on GCP. Collaborate with data scientists and engineers to ensure data quality, implement CI/CD for pipelines, monitor production systems, and recommend new cloud data technologies. Support streaming data processing and infrastructure-as-code for secure, scalable solutions.

Top Skills: Apache AirflowApache BeamApp EngineBashBigQueryCi/CdCloud ComposerCloud StorageCompute EngineDataflowDataopsDnsDockerFirewallGCPGitKubernetesKubernetes EngineLoad BalancingPub/SubPythonTerraformVpc

Stake

Data Engineer

2 Months Ago

In-Office

Sydney, New South Wales, AUS

Mid level

Financial Services

Build, maintain and scale production data pipelines and platforms on GCP (BigQuery, Dataflow, Cloud Storage). Improve data quality, observability, governance and self-service capabilities; partner with engineers, analysts and product teams to design data models and datasets that inform business decisions.

Top Skills: BigQueryCloud StorageDataflowDockerGCPKubernetesMetabasePythonSQL

What you need to know about the Sydney Tech Scene

From opera to comedy shows, the Sydney Opera House hosts more than 1,600 performances a year, yet its entertainment sector isn't the only one taking center stage. The city's tech sector has earned a reputation as one of the fastest-growing in the region. More specifically, its IT sector stands out as the country's third-largest, growing at twice the rate of overall employment in the past decade as businesses continue to digitize their operations to stay competitive.