Job Summary:

We are seeking an Embedded LLM Systems Engineer to design, develop and optimise LLM inference solutions for embedded, mobile and edge devices. The role covers inference-engine development, model compression, heterogeneous hardware optimisation and efficient model deployment.

Duties/ Responsibilities:

On-Device Inference Engine Development

Design, develop and optimise LLM inference engines for embedded, mobile and edge devices.
Work on operator development, graph optimisation, memory management and multi-backend adaptation.
Develop solutions using frameworks such as llama.cpp, TensorRT-LLM, MNN, ONNX Runtime or comparable technologies.
Research and apply quantisation techniques such as INT4, INT8 and FP16.
Work with relevant technologies such as NEON/SVE, Vulkan Compute, OpenCL or comparable platforms.
Conduct training-inference consistency validation and support efficient deployment across cloud and edge environments.
Evaluate practical ways to translate emerging AI capabilities into embedded product applications.

Basic Requirements:

At least three years of relevant experience in on-device inference, AI infrastructure, embedded systems engineering or a related area.
Bachelor's degree or above in Computer Science, Electrical/Electronic Engineering, Mathematics or a related discipline, or equivalent practical experience.
Proficiency in written and spoken English sufficient to read technical documentation and research papers, participate in technical discussions and prepare clear engineering documentation.
Strong proficiency in modern C++, with a good understanding of memory models, concurrency and low-level performance optimisation.
Proficiency in Python for model conversion, evaluation, automation scripts and training-related tooling.
Experience with CUDA, MediaPipe or related technologies is advantageous.

Preferred Qualifications

Any of the following would be advantageous:

Contributions to established open-source inference projects such as llama.cpp, vLLM, TensorRT-LLM, MLC-LLM or MNN.
Publications in recognised conferences or journals on efficient inference, model compression or on-device deployment.
Recognition in competitions such as ACM-ICPC, NOI, Kaggle or on-device AI challenges.
Experience with prompt engineering, Retrieval-Augmented Generation or AI agent frameworks such as LangChain or LlamaIndex.
Hands-on experience deploying and optimising inference frameworks such as vLLM, TGI, llama.cpp, TensorRT-LLM or MLC-LLM.
Knowledge of model alignment and fine-tuning techniques, including RLHF, SFT and DPO.
Practical experience fine-tuning or evaluating models using authorised, organisation-owned or appropriately licensed datasets.
Strong interest in emerging LLM technologies and the ability to translate new capabilities into practical product value.