Hyphen Connect Limited
LLM Pre-training & Distributed Engineer (AI Infrastructure)
Be an Early Applicant
Lead orchestration and optimization of large-scale LLM pretraining across 1,000+ GPUs. Manage distributed training with PyTorch/DeepSpeed/Megatron-LM, tune networking and memory (InfiniBand/RDMA), and implement checkpointing and robust failure recovery for long-running jobs.
We are seeking a highly skilled LLM Pre-training & Distributed Systems Engineer. This role is essential for orchestrating large-scale machine learning training runs and optimizing distributed infrastructure. The ideal candidate will have a deep understanding of GPU clusters and extensive experience in system engineering to ensure efficient and reliable training processes.
Responsibilities:
- Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
- Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
- Automate checkpointing and failure recovery during month-long training runs.
Required Skills:
- Deep expertise in 3D parallelism (Data, Tensor, Pipeline).
- Experience managing SLURM or Kubernetes-based GPU clusters.
- Strong systems engineering background (C++, CUDA, Python).
Similar Jobs
Cloud • Computer Vision • Information Technology • Sales • Security • Cybersecurity
Lead complex cybersecurity investigations across endpoint, identity, email, network, and cloud environments. Analyze EDR telemetry, forensic artifacts, network data, malware, and Microsoft 365 attacks to determine root cause and scope. Develop remediation plans, mentor analysts, improve SOC processes, build automation, and advise customers during critical incidents. Contribute to threat intelligence, detection strategy, AI-enabled security operations, thought leadership, and industry engagement.
Top Skills:
Agentic Soc WorkflowsAi ModelsEdrMicrosoft 365Mitre Att&CkPrompt Engineering
Artificial Intelligence • Healthtech • Machine Learning • Natural Language Processing • Biotech • Pharmaceutical
Leads the pharmaceutical site PMO, governing a portfolio of strategic transformation programs including technology transfers, new product introductions, capital initiatives, digital roadmaps, and operational readiness. Oversees prioritization, resources, risks, budgets, benefits realization, executive reporting, and stage-gate governance. Partners with cross-functional teams across operations, quality, engineering, supply chain, regulatory, EHS, and finance while coaching project managers and improving PMO standards.
Top Skills:
GmpLeanMicrosoft ProjectPrimaveraSix SigmaSmartsheet
Automotive
Establish and lead Ford Pro’s Australian Project Management Office across multiple workstreams. Responsibilities include creating governance, standards, reporting cadence, integrated plans, RAID and decision logs, change control, health checks, steering committee updates, and executive communications. The role requires improving programme delivery maturity, coordinating teams across Australia and overseas, managing ambiguity, and using dashboards and AI or automation tools to improve project visibility.
Top Skills:
Ai Automation ToolsData Dashboards
What you need to know about the Sydney Tech Scene
From opera to comedy shows, the Sydney Opera House hosts more than 1,600 performances a year, yet its entertainment sector isn't the only one taking center stage. The city's tech sector has earned a reputation as one of the fastest-growing in the region. More specifically, its IT sector stands out as the country's third-largest, growing at twice the rate of overall employment in the past decade as businesses continue to digitize their operations to stay competitive.
