Technical leader responsible for designing and operating scalable, fault-tolerant ETL/ELT pipelines and real-time stream processing using Kafka and RabbitMQ. Optimize ClickHouse schemas and queries for large financial datasets, deploy secure cloud data services, implement monitoring and automated recovery, and mentor data engineers while aligning data strategy with business objectives.
About The Company
TMGM is one of the fastest-growing CFD and Forex brokers in the Asia-Pacific region, trusted by over 150,000 clients globally. With a strong focus on innovation, client satisfaction, and regulatory compliance, we continue to expand our presence across international markets.
About The Role
As a Senior Data Engineer, you will be a technical leader responsible for the architecture, scalability, and reliability of our high-throughput, real-time data ecosystem. You will oversee the evolution of our data infrastructure, leveraging Kafka, RabbitMQ, Airflow, and ClickHouse to power mission-critical financial analytics. Your role is to bridge the gap between complex business requirements and high-performance engineering, ensuring our data pipelines can handle the rigours of real-time financial data processing.
Requirements
Key Responsibilities:
- Lead the design and evolution of highly scalable, fault-tolerant ETL/ELT pipelines.
- Dive the strategy for real-time messaging and stream processing using Kafka and RabbitMQ to ensure sub-second data availability.
- Act as the subject matter expert for ClickHouse, optimising complex schema designs, indexing strategies, and query performance for large-scale financial datasets.
- Oversee the deployment of data services within cloud environments, implementing advanced security protocols and data governance standards essential for the finance industry.
- Collaborate with senior leadership to align data strategy with business objectives. Mentor data engineers through code reviews and technical guidance.
- Implement advanced monitoring and automated recovery systems to ensure the integrity and quality of high-stakes financial data.
Requirements:
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- Proven experience in data engineering, with a strong background in designing and implementing ETL processes within cloud environments.
- Experience within the Finance or Trading technology sector, with a proven track record of handling real-time market or transactional data.
- Strong programming skills in Python, with experience in developing robust, maintainable, and scalable data processing pipelines.
- Extensive SQL knowledge and experience.
- Excellent problem-solving skills and the ability to work collaboratively in a team environment.
- Strong communication skills, with the ability to convey complex technical concepts to non-technical stakeholders.
Benefits
- Hybrid working arrangement - 2 Days of remote work per week
- Opportunities for enriching career growth, including exposure to regional contexts
- Complimentary snacks and beverages available in the office pantry
- Healthcare coverage (medical, dental, optical), gym benefits
- Flexibility in smart casual dress code
- Young, vibrant and open work culture
TMGM Sydney, New South Wales, AUS Office
Sydney, New South Wales, Australia
Similar Jobs
Artificial Intelligence • Cloud • Information Technology • Consulting
Leads consulting, architecture, and delivery for large-scale service provider and telecommunications network transformation programs. Designs Juniper-based BNG, CGNAT, ISP, IP core, routing, switching, and broadband architectures; advises executives; governs technical decisions; mentors teams; and leads customer workshops. Supports presales through solution strategy, statements of work, cost modeling, proposals, and opportunity development while ensuring delivery quality, customer outcomes, and revenue growth.
Top Skills:
Broadband Network Gateway (Bng)Carrier Grade Nat (Cgnat)Ip CoreIpv4Ipv6JuniperJuniper AcxJuniper MxMplsNetwork AutomationNetwork OrchestrationRouting And SwitchingSegment Routing
Artificial Intelligence • Cloud • Information Technology • Consulting
Provides remote technical support for customer infrastructure, resolving hardware and software incidents through phone and e-support. Diagnoses routers, switches, firewalls, operating systems, VMware, servers, and storage; records incidents and resolutions in ServiceNow and knowledge systems; escalates issues when needed; manages customer communication and service-level requirements while maintaining ITIL practices.
Top Skills:
FirewallsHp-UxHpe Monitoring SystemsIncident Tracking SystemsItilKnowledge Management SystemsMicrosoft Windows ServerRoutersServicenowStorage SystemsSuse LinuxSwitchesVMware
Artificial Intelligence • Cloud • Information Technology • Consulting
Manages virtual sales for server and storage solutions across assigned Canberra accounts within federal government and defense agencies. Builds account plans, generates and qualifies leads, develops sales pipelines, forecasts activity, positions HPE products, and drives acquisitions, retention, and sales closure. Collaborates with field sales, partners, and customers, while potentially coaching team members.
Top Skills:
Customer Relationship Management (Crm)SalesforceServerStorage
What you need to know about the Sydney Tech Scene
From opera to comedy shows, the Sydney Opera House hosts more than 1,600 performances a year, yet its entertainment sector isn't the only one taking center stage. The city's tech sector has earned a reputation as one of the fastest-growing in the region. More specifically, its IT sector stands out as the country's third-largest, growing at twice the rate of overall employment in the past decade as businesses continue to digitize their operations to stay competitive.
