Responsibilities
About the Team Flow is ByteDance's AI-native innovation business team, focusing on developing products in emerging fields such as LLM. Our product portfolio includes Doubao (China) and Cici (Global). The Model Evaluation Team plays a key role within Flow, providing accurate and timely assessment signals for product iterations. We are responsible for the entire evaluation process, from designing methodology to executing evaluations, identifying issues, and proposing feasible improvement suggestions. Join us to explore innovative opportunities in AI, grow in a dynamic team environment, and create real value for users! Responsibilities 1. Evaluation Project Management: Lead execution of evaluation projects on LLM and other GenAI domains with a highly hands-on approach. Apply deep evaluation know-hows and domain expertise to solve sophisticated challenges in LLM capability assessment, and ensure timely and quality delivery of results 2. Data Set Curation: Build human and/or automated evaluation data sets that represent authentic user scenarios across key segments and modalities. Ensure automated evaluation results align with expert judgment and reliably predict online user preferences 3. Evaluation Results Analysis: Deliver evaluation reports with clear insights into model and product performance, converting nuanced case analyses into generalizable patterns. Analyze model performance & user signals to identify gaps in existing evaluation framework and propose actionable optimization directions 4. Cross-Functional Collaboration: Coordinate effectively with internal evaluators and external vendors to ensure project goals are met on time. Communicate effectively across product, engineering and other functions to build consensus around project objectives, methodologies and results
Qualifications
Minimum Qualifications 1. Bachelor's degree or above in Science, Arts & Humanities, or related fields, demonstrated ability to learn and synthesize knowledge across diverse domains 2. Excellent English writing and critique skills OR domain expertise in a specific knowledge field (e.g., Science, Humanities) with adequate language critique ability 3. 2+ years of experience in product operations, project management or strategy roles in technology companies 4. Strong critical thinking, numerical, and analytical skills are required 5. Proven project management capabilities with both internal and external stakeholders comfortable handling ambiguous requirements and adapting to changes while maintaining project momentum 6. Comfortable with meeting deadlines and generating creative solutions, especially Prompt Engineering techniques, to enhance the quality of both individual and team outputs 7. Proficiency in Mandarin is required for communication with key stakeholders and the businesses with teams based in China. Preferred Qualifications 1. Hands-on experience with LLM, GenAI and search product performance evaluation is strongly preferred 2. A deep interest in LLMs is essential the ideal candidate is an enthusiastic learner who finds engagement with diverse case studies stimulating
