ARTIFICIAL INTELLIGENCE (AI) RESEARCHER

Job Responsibilities:

1. Design and develop distributed video data acquisition systems to support video large model training requirements, covering approved, public, licensed or internally authorised video data sources, including short-form video, long-form video, media libraries, material libraries and social media content where permitted.

2. Analyse video streaming formats and media delivery workflows such as HLS/DASH, video playback interfaces, dynamic web content and metadata structures, and develop compliant solutions for video ingestion, media processing, file merging, subtitle extraction and metadata synchronisation.

3. Build high-concurrency data acquisition and processing clusters using Docker, Kubernetes and related infrastructure, including scheduling, system monitoring, request management, reliability controls and resource optimisation.

4. Keep track of technical changes in domestic and international video platforms, web technologies and media delivery mechanisms, and develop stable, reusable and compliant technical solutions for authorised data acquisition and processing.

5. Develop and optimise systems for dynamic source management, intelligent task scheduling, account and access lifecycle management where applicable, adaptive request rate control, bandwidth optimisation and server cost control.

6. Provide practical solutions for complex video data acquisition and processing scenarios involving approved international and regional platforms, licensed data sources, public datasets, internal datasets and partner-provided datasets.

7. Support video data pipeline development and data quality governance, including system operations, monitoring, documentation, data traceability, risk controls and compliance management.

8. Explore emerging technologies in large-scale video data processing, multimodal data acquisition, AI training dataset construction and responsible AI data governance.

Requirements:

1. Bachelor's degree or above in Computer Science, Computer Engineering, Network Engineering, Big Data, Information Security, Artificial Intelligence or a related field, or equivalent practical experience.
2. At least 5 years of experience in crawler systems, data acquisition systems, backend engineering, distributed systems or large-scale data pipeline development, including at least 3 years of practical experience in video, streaming media or multimedia data processing. Experience in AI multimodal large model data acquisition is preferred.
3. Proficient in Python and familiar with at least one other programming language such as Go, Java, C++ or Scala familiar with asynchronous processing, distributed crawler architecture, task scheduling and tools such as Scrapy, Celery, Playwright, Puppeteer or Mitmproxy for authorised testing and debugging.
4. Strong understanding of HTTP/HTTPS/TCP protocols, streaming media protocols such as HLS/DASH, web technologies, data parsing, encryption concepts, authentication mechanisms, Cookie/Token handling and media metadata processing.
5. Familiar with distributed systems, concurrency, proxy management, request management, access control, system reliability and large-scale overseas data acquisition scenarios, with awareness of platform terms, copyright, privacy and data compliance requirements.
6. Familiar with audio and video processing, including FFmpeg batch transcoding, key frame extraction, video hashing, subtitle parsing and large-scale multimedia object storage solutions such as MinIO, OSS or similar systems.
7. Familiar with Docker, Kubernetes, logging and monitoring tools such as Prometheus and Grafana, with the ability to build and maintain reliable data acquisition and processing clusters.
   
8. Proficiency in Mandarin is preferred, as the role requires frequent collaboration with Mandarin-speaking technical stakeholders.

Preferred Qualifications:

1. Experience working in text-to-video, multimodal large model, video generation or AI model training companies, and experience delivering large-scale video training datasets.

2. Experience with Android/iOS application analysis, authorised packet capture, dynamic debugging or mobile data integration for legitimate testing, troubleshooting or system integration purposes.

3. Familiar with big data tools such as Spark or Flink, with the ability to process, clean and filter TB-scale video metadata.

4. Experience developing browser compatibility testing, behaviour simulation, request scheduling, data validation or automated quality control systems.

5. Familiar with copyright-compliant media libraries, film and television content sources, gaming-related video sources, vertical industry video data sources or licensed data partnerships.

6. Experience in crawler or data engineering team management, technical solution review, system architecture design or leading complex technical projects.