字节跳动Conversational AI Large Model Researcher | 智能对话大模型算法工程师-电商-筋斗云人才计划
任职要求
1. Got doctor degree, preferably with a background in computer science, software engineering, artificial intelligence, mathematics, or related fields. 2. Solid foundation in machine learning, with in-depth understanding of deep learning, multimodal models, generative models, and other technologies; strong mathematical and ability to self-learn are required. 3. Proficient in relevant machine learning frameworks (e.g., PyTorch, TensorFlow) and engineering frameworks, with strong coding skills. 4. Prior experience in the field of multimodal large models, especially in long-text processing or film/television drama domains, is preferred. 5. Candidates who have published papers in top-tier computer science conferences/journals (e.g., NeurIPS, ICML, CVPR, ICCV, ECCV, IJCAI, AAAI, KDD, SIGIR, WWW, ACL, PAMI, IJCV) or have competition experience are highly preferred. 1、获得博士学位，博士在读，计算机、软件、人工智能、数学等相关专业优先； 2、扎实的机器学习基础，深入理解深度学习、多模态模型及生成模型等技术，具备良好的数理基础和自学能力； 3、熟练掌握相关机器学习框架和工程框架，具备扎实的编码能力； 4、在多模态大模型领域有相关经验，尤其在长文本、影视剧领域有经验者优先； 5、在计算机科学高水平会议和期刊（如NIPS、ICML、CVPR、ICCV、ECCV、IJCAI、AAAI、KDD、SIGIR、WWW、ACL、PAMI、IJCV等）发表过论文或具备竞赛经验者优先。
工作职责
Team Introduction: Dedicated to building an industry-leading large-model dialogue system, the team serves hundreds of millions of daily active users, with application scenarios covering the entire Douyin e-commerce ecosystem. This includes core business scenarios such as platform customer service, platform merchant service, merchant customer service, influencer customer service, and innovative intelligent shopping guides. Through continuous technological innovation and optimization, the team has successfully established a complete intelligent dialogue solution, delivering significant efficiency improvements and user experience enhancements to e-commerce operations. Research Objectives: Develop an LLM-based customer service chatbot for TikTok and Douyin E-commerce, enabling intelligent customer service interactions. The LLM will handle the entire user inquiry process, including request clarification, solution negotiation, and execution. Necessity: LLM's strong conversational and reasoning abilities make it especially suitable for intelligent customer service, capable of potentially reaching the service standards of excellent human representatives. Research Content: Design a multi-agent framework based on LLM, integrating planning-agent, reply-agent, and tool-agent. Each agent will specialize in different functions, working collaboratively to manage the complete service process—from issue identification and solution negotiation to solution implementation and feedback. 1) Reply-agent ensures the proposed solutions comply with platform policies and service guidelines, avoids excessive improvisation or hallucinations, and maintains smooth communication and negotiation with the user. 2) Planning-agent identifies user demands and problem scenarios, sourcing relevant service guidelines and constraints as well as recognizing risk scenarios. 3) Tool-agent validates the legality of tool usage, accurately interprets the results from tool interactions, and manages execution dependencies of various actions. Research Challenges: Compliance with service guidelines: Ensuring the chatbot's solutions adhere to platform service guidelines (such as available refund within xx days of parcel arrival and coupon limits per user per week). Dynamic feedback adaptation: Static adherence to service rules and providing fixed solutions can limit the flexibility of reply-agents, preventing them from acting like excellent human customer service representatives. By enabling reply-agents to interact in real-time with their environment, considering user's behavioral trends, demands expressed during inquiries, and feedback on proposed solutions, personalized service can be provided. This approach fosters adaptive responses and progressive services and solutions, closely mirroring the flexibility and excellence of human customer service. Self-reflection: Employing LLM's capabilities to understand, analyze, and evaluate its own behavior, fostering self-supervision and decision refinement through reflection on outputs, particularly with complex and ambiguous tasks. Complex image processing: Handling scenarios involving numerous complex images (including shipping order photos, bank transaction screenshots, images of damaged goods received, and seller qualification certifications). These images contain key information crucial to enhancing the chatbot's problem resolution capabilities. 团队介绍： 智能对话团队，致力于打造业界领先的大模型对话系统。团队服务的日活用户超过数亿，应用场景覆盖抖音电商全链路，包括平台客服、平台商服、商家客服、达人客服，以及创新的智能导购等核心业务场景，通过持续的技术创新和优化，成功构建了一套完整的智能对话解决方案，为电商业务带来了显著的效率提升和用户体验改善。 课题目标： 构建基于LLM的电商客服机器人（Chatbot），服务TikTok和抖音电商智能客服场景，由LLM完成一次用户进线的完整接待过程，包括诉求澄清、方案协商、方案执行等阶段。 必要性： LLM具有强大的对话和推理能力，智能客服是LLM能够发挥价值的最典型场景，有机会能够达到匹配优秀人工客服的服务能力。 课题内容： 设计一个基于LLM 的 multi-agent framework，将 planning-agent、reply-agent、tool-agent 集成到一起，每个 agent 负责不同能力，互相协同，完成从问题定位、方案协商，到方案执行、结果反馈等服务全流程。reply agent 需要确保给用户提供的方案是符合平台的相关政策和service policy的，不自行过度发挥、不出现幻觉，顺滑的完成和用户的沟通协商过程；planning agent 完成定位用户诉求和问题场景，以便从外部获取该场景的服务准则和约束，如何识别风险场景；tool agent 需要确保工具调用的合法性、接收和解析工具调用的返回结果，另外一些动作的执行存在前后依赖的问题。 课题挑战： 1、遵循服务准则：如何确保方案Chatbot提供的方案是follow平台服务准则的，例如到货xx天之内可以申请退款、同一用户一星期内最多发送xx额度的优惠券； 2、感知环境反馈：reply agent如果只能死板的follow当前场景服务准则，提供一层不变的方案，是无法像优秀客服一样做到灵活变通的。让Agent能够实时的和环境打通，通过结合当前用户进线前的行为动线、进线后表达的诉求和用户对 agent 提供方案的反馈，为用户提供个性化的服务，对用户的实时反馈有响应，像优秀客服一样能随机应变，递进式的提供服务和解决方案； 3、进行自我反思：利用LLM理解、分析和评价其自身的行为，使LLM能够自我监督，通过对自身输出的反思，改进其所做的决策，以便在处理复杂、有歧义的任务时，能有更好的表现； 4、复杂图片理解：电商场景存在大量复杂的图片，包括运费订单实拍图、银行流水截图、买家收货缺件破损的、商家各类资质证明等，这类图片往往包含重要的信息，对提升Chatbot解决能力非常重要。
•Participate in the development of knowledge-based Q&A or conversational AI products, enhancing the capabilities of large language models in areas such as RAG (Retrieval-Augmented Generation) and Agents, including data systems, algorithm optimization, prompt engineering, and evaluation iteration. •Build a systematic and specialized knowledge base, optimize the full-link retrieval technology, and continuously improve retrieval precision and recall. •Improve the performance of large language models in Q&A and conversational scenarios through techniques such as RAG, SFT (Supervised Fine-Tuning), and RLHF (Reinforcement Learning from Human Feedback).
The ideal candidate will have experience in customer-facing roles and success in leading in-depth technical Application development architecture discussions with senior customer executives, Architects, IT Management, and Developers to drive value to our customers and is open to travel to customer site as needed by business. • Understanding Customer/Partner Technical Environment (Insights about Customer/Partner and Industry): Gather customer/partner insights (e.g., feedback around technical preferences, environments, business needs, competitive landscape), and map architecture and digital transformation solutions to customer/partner business outcomes. Adapt business models, plans, and solutions to insights. • Understanding Customer/Partner Technical Environment (Internal Advocacy): Act as the voice of the customer (VOC)/partner by driving new feedback, gaps, blockers, insights, resources, etc. across communities to track, add, and prioritize, using established channels (e.g., UAT/TFT). • Architecture Design and Deployment (Architecture Proposals): Receive and synthesize data about customer/partner business and technical requirements, address them with technical architecture(s), demonstrate and prove those solutions capability and business value through design collaboration sessions with the customer/partner. • Architecture Design and Deployment (Requirements and Constraints): Apply broad technical knowledge across various architecture solutions to meet business and information technology (IT) requirements and resolve identified technical constraints. Help to shape and enhance customers' requirements. • Application Development, Architecture Design and Deployment (Resolving Blockers): Identify, escalate, and work to resolve technical blockers (e.g., changing configurations, sample coding) to accelerate architecture implementations and routes non-technical issues for removal by the appropriate party. • Trusted Advisor (Challenger Mindset): Respectfully challenge customers/partners when going in the wrong direction and escalate appropriately. • Trusted Advisor (Competitor Insights/ Differentiated Value Proposition): Understand the competitor's architecture solutions and identify Microsoft's strengths over competitive solutions to drive conversations with customers/partners and convince them of solution. • Customer Usage: Lead architecture design, resiliency reviews, and technical optimization that result in production deployment application and increase customer business value. Drive efforts to ensure that the customer's environment and applications are well-architected. • Customer Satisfaction – Deliver positive Customer Satisfaction and become trusted advisors to customers by leveraging solution area expertise to enable defined Customer Success Plan outcomes.
We are seeking a skilled developer to build production-grade AI applications, focusing on LLM-based agents and tool-using systems. You will integrate large language models (LLMs), retrieval-augmented generation (RAG), and external tools/APIs on GPU-accelerated stacks, enhancing agent frameworks for reliability, scalability, and safety. What You’ll Be Doing: • Design, implement, and deploy AI-powered features using LLMs, including autonomous and multi-agent workflows. • Build agent toolchains, including planning, tool/function calling, memory management, RAG integration, and enterprise API connectivity. • Enhance agent frameworks with custom planners, routers, concurrency control, state management, and retry mechanisms. • Develop evaluation and observability systems to monitor agent performance (success rates, tool-call accuracy, latency, cost, traces). • Implement safety and compliance measures, including content filtering, PII handling, and policy enforcement using guardrail frameworks. • Optimize inference pipelines for GPU performance, latency, and cost; deploy via microservices and APIs. • Manage CI/CD, containerization, and deployment; maintain monitoring, logging, and alerting; and produce clear documentation.
• You will develop relationships with key customer IT and business decision makers, to understand their data estate, priorities, and success measures, and design secure, scalable Data & AI solutions that deliver measurable business value. • You will understand customers’ overall data estate Business and IT priorities and success measures to design Data & Analytics solutions that drive business value and drive positive Customer Satisfaction & become a trusted advisor. • You will ensure that solution exhibits high levels of performance, security, scalability, maintainability, repeatability, appropriate reusability, and reliability upon deployment and provide feedback and insights from customers/partners. • You will develop opportunities to drive Customer Success business results & help Customers get value from their Microsoft investments and identify resolutions to Customer blockers by leveraging subject matter expertise. Deliver according to MS best practices & using repeatable Intellectual Property (IP). • You will apply technical knowledge to architect and design solutions that meet business and IT needs, create Data & Analytics roadmaps, drive Proof of Concepts (POC) and Minimal Viable Product (MVP), and ensure long term technical viability of new deployments, infusing key AI technologies where appropriate. • You will maintain technical skills and knowledge, keep up to date with market trends and competitive insights; collaborate and share with the AI technical community while educating customers on Azure platform. • You will accelerate customer outcomes - Share expertise, contribute to IP creation & re-use to accelerate customer outcomes and obtain relevant accreditations and certifications. Proactively identify gaps through delivery and communicating those gaps to others (e.g., Leadership, managed intellectual property [MIP], Design, and Governance). Applies subject matter expertise to develop new IP that fills identified gaps • Drive Azure consumption through accurate and complete Azure Consumption Plans and support Unified/Factory delivery models. • Lead technical workshops, build MVPs, and guide customers through production deployments. • Collaborate with internal teams to remove blockers and share customer feedback. • Evangelize Azure technologies within customer, partner, and community ecosystems. • Own the end-to-end technical delivery results, ensuring completeness and accuracy of consumption and customer success plans in collaboration with the Customer Success Account Managers. • Provide delivery oversight and escalation support for key consumption engagements across Data & Analytics workloads. • Drive technical excellence by leading the health, resiliency, security, and optimization of mission-critical data workloads, ensuring readiness for production-scale AI use cases. • You will identify resolutions to issues blocking go-live of migration and modernization projects by leveraging Azure Infrastructure technical subject matter expertise and lead technical conversations with customers to drive value from their Microsoft investments and deliver all work according to MS best practices and policies. • You will demonstrate a self-learner mindset through continuous alignment of individual skilling to team/area demands and Customer Success goals as you accelerate customer outcomes and engage in relevant communities to share expertise, contribute to IP creation, prioritize IP reuse and learn from others to help accelerate your customers transformation journey. • Be accredited and certified to deliver with advanced and expert-level proficiency in priority workloads including Microsoft Fabric, Azure Databricks, Microsoft Purview, Azure SQL, PostgreSQL, MySQL, and Cosmos DB. Technical leadership • Design the solution using your Data Platform technical knowledge, architectural approach, consultancy skills and our methodology to win a customer’s technical decision and meet the customer’s needs; drive proof of concepts (POCs)/production deployments to create momentum for MVPs, infusing key AI technologies where appropriate and being technically proficient to do POC with hands-on-skills. • Data Platform technologies, including SQL Server, Azure SQL Database, and Azure Cosmos DB, as well as Analytics, Data Warehousing, and Data Engineering scenarios. Focus on Intelligent Data and Analytics Platforms such as Synapse, Lakehouse architecture, Fabric, Azure Databricks, and Power BI. Stay informed on market trends and competitive insights and actively collaborate and share knowledge within the Data & AI technical community. • Experience in Azure architecture design, Azure IaaS, Azure SQL Database, Azure SQL Managed Instance, High Availability, service resilience and distributed systems. • Experience as a Database Administrator, data platform design, SQL Server at scale, on-premises to Azure migrations, complex SQL Server troubleshooting, performance tuning and optimization, scaling OLTP and OLAP workloads, and securing SQL Server on-premises or in Azure • Experience with High Availability/Disaster Recovery solutions such as Failover Clustering, Database Mirroring, Always-On Availability Groups, and Replication • Use trace analysis, debug skills, source code, and other proprietary tools, to analyse problems and develop solutions to meet customer needs; this may involve writing code • Consultative experience along with presentation skills with a high degree of comfort with both large and small audiences • Learns to identify and communicate areas in intellectual property (IP) that need refreshing or have gaps. Contributes to IP development with guidance/supervision. • Demonstrates strong industry knowledge and increases recognition for Microsoft solutions by contributing to and sometimes leading presentations and engagements with external and internal audiences (e.g., Tech Connect, Build, Ignite)