Postdoctoral Researcher (OCCL)

Project background

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the graph of thoughts. We are firmly in the Age of Computation, where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid growth in computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country's researchers to advance in this most important field.

The current era of artificial intelligence is predominantly driven by advances in computational power and infrastructure. As models scale to unprecedented sizes, their capabilities are enhanced through strategies such as reinforcement learning (RL) and innovative frameworks like the graph of thoughts. We are firmly in the Age of Computation, where breakthroughs in AI are synonymous with the ability to harness and optimize massive computational resources. However, this rapid grow thin computational demands comes with a critical challenge: energy limitations. Around the globe, new power plants are being constructed to meet the demands of AI workloads, underscoring the urgent need for efficiency to ensure these advancements benefit humanity sustainably. Especially in Singapore, such resources may be scarce and hinder the ability of the country's researchers to advance in this most important field.

Our collaborative efforts with the A.STAR Institute of Advanced Intelligence and Computing (ASTAR IAIC) and other partners in Singapore, such as NTU, could be pivotal in this field, driving the development and implementation of innovative AI methods and saving millions of dollars in compute time while accelerating scientific productivity. To enhance the efficacy of AI for Science models, it is imperative to integrate simulations that encode prior scientific knowledge within these AI frameworks. This integration is aimed at creating new models imbued with robust prior knowledge, such as simulated behaviors defined by fundamental equations and conservation laws. Our approach involves the close collaboration of experts from A.STAR IAIC, who bring extensive experience in high-performance computing and AI-driven scientific research. Together, we aim to push the boundaries of what AI can achieve in scientific domains, ensuring that simulations and AI methodologies are combined to produce highly accurate and reliable models.

Job description

The Postdoctoral Researcher will be co-leading the development of a novel open-source accelerator communication library (OCCL). This communication library should support point-to-point as well as collective communication calls. It should allow communication initiated by the host, as well as communication initiated within accelerator device kernels. It should support a wide range of devices and network substrates, such as NVLINK and next-generation ethernet networks.

Some key components in the work are:

• Co-lead the design and development of an open-source accelerator communication library (OCCL) supporting point-to-point and collective communication for distributed AI workloads.
• Design and implement communication primitives and algorithms optimised for machine learning training and inference workloads, supporting communication initiated by both host applications and accelerator device kernels.
• Investigate and optimise communication patterns for large-scale deep learning workloads, including techniques for improving efficiency through communication scheduling and overlap with computation.
• Ensure the communication library is configurable and portable across diverse hardware platforms and network substrates, including modern high-performance interconnect technologies used in HPC and ML environments such as UCX, NVLINK, InfiniBand, UltraEthernet and related networking frameworks.
• Evaluate and optimise the communication library and associated software components on GPU clusters and large-scale computing systems, ensuring performance scalability, robustness and usability across heterogeneous computing architectures.
• Collaborate with researchers and engineers at partner institutions in Singapore and Zurich to integrate OCCL with the broader FastTrackAI software ecosystem.
• Maintain effective communication and collaboration with supervisors, research staff and external partners distributed across Singapore and Zurich, ensuring alignment of research objectives and deliverables.
• Prepare and publish key scientific outputs, including papers documenting research outcomes and system developments.
• Contribute to the co-supervision of MSc and BSc students working on related research topics.

Selection Criteria

• PhD in Computer Science or any related field.
• The successful candidates should have a strong background in computer science, parallel computing, compilers, performance optimisation, and high-performance computer systems. Excellent knowledge of C and C++ is required.
• A good working knowledge of the design principles of existing high-level communication libraries, such as MPI and NCCL.
• Experience and/or knowledge of modern communication and network technologies used in HPC and ML environments, such as UCX, NVLINK, InfiniBand, UltraEthernet or related frameworks.
• Prior knowledge of communication optimisation for large-scale ML training and inference workloads is essential. A candidate with existing publications in distributed ML systems, HPC systems or related are highly desirable.
• Since the aim of this project is to build a software product, not only a prototype, prior experience in managing and carrying out larger software engineering projects is required.
• The team working on this subject is distributed across Singapore and Zurich, thus a successful candidate must both be able to work independently and communicate effectively with remote colleagues.
• Applicants need to be proficient in written and spoken English.

Why SEC is your employer of choice

• Accredited with 5 Tripartite Standards by Tripartite Alliance for Fair & Progressive Employment Practices (TAFEP) Singapore.
• A diverse workplace with 32 nationalities, offering ample opportunities for mutual learning.
• Positive and inclusive working environment.
• 25 days of annual leave for fixed-term contracts.
• 1 day of Birthday Leave.
• Annual dental benefits.
• Committed to being a supportive employer as you prioritize your physical and mental wellness.
• Comprehensive healthcare insurance coverage.
• Flexible hybrid work arrangement (up to 2 days per week from home).
• Abundant networking opportunities across various disciplines.
• Accredited with NS mark certification.

The Singapore-ETH Centre is an equal opportunity and family-friendly employer. All candidates will be evaluated on their merits and qualifications, without regards to gender, race, age or religion.

Curious So are we.

We look forward to receiving your online application with the following documents:

• A cover letter with a specific statement of your motivation for the project and how your background and experience align with the position. Please reference the requirements outlined above and relate your own background and past experience to those.
• Your CV including the name and contact information of 2 references.
• A copy of your university transcripts as PDFs.
• Reference or recommendation letters may be included.

Applications via email or postal services will not be considered.

Work location: 1 Create Way, CREATE Tower, Singapore 138602 (NUS University Town)
Contract Duration: 2 Years

Remuneration will be commensurate with the candidate's qualifications and relevant experience.

Further information about The Singapore-ETH Centre can be found on our website: https://sec.ethz.ch/
For further information, please contact: Mrs Iva Kabosch (ETH Zurich) at [Confidential Information]