字节跳动Machine Learning Algorithm Research Engineer | 机器学习算法研究员-TikTok直播-筋斗云人才计划

Team Introduction: TikTok Content Security Algorithm Research Team The International Content Safety Algorithm Research Team is dedicated to maintaining a safe and trustworthy environment for users of ByteDance's international products. We develop and iterate on machine learning models and information systems to identify risks earlier, respond to incidents faster, and monitor potential threats more effectively. The team also leads the development of foundational large models for products. In the R&D process, we tackle key challenges such as data compliance, model reasoning capability, and multilingual performance optimization. Our goal is to build secure, compliant, and high-performance models that empower various business scenarios across the platform, including content moderation, search, and recommendation. Research Project Background: In recent years, Large Language Models (LLMs) have achieved remarkable progress across various domains of natural language processing (NLP) and artificial intelligence. These models have demonstrated impressive capabilities in tasks such as language generation, question answering, and text translation. However, reasoning remains a key area for further improvement. Current approaches to enhancing reasoning abilities often rely on large amounts of Supervised Fine-Tuning (SFT) data. However, acquiring such high-quality SFT data is expensive and poses a significant barrier to scalable model development and deployment. To address this, OpenAI's o1 series of models have made progress by increasing the length of the Chain-of-Thought (CoT) reasoning process. While this technique has proven effective, how to efficiently scale this approach in practical testing remains an open question. Recent research has explored alternative methods such as Process-based Reward Model (PRM), Reinforcement Learning (RL), and Monte Carlo Tree Search (MCTS) to improve reasoning. However, these approaches still fall short of the general reasoning performance achieved by OpenAI's o1 series of models. Notably, the recent DeepSeek R1 paper suggests that pure RL methods can enable LLM to autonomously develop reasoning skills without relying on the expensive SFT data, revealing the substantial potential of RL in advancing LLM capabilities. 团队介绍： 国际化内容安全算法研究团队致力于为字节跳动国际化产品的用户维护安全可信赖环境，通过开发、迭代机器学习模型和信息系统以更早、更快发掘风险、监控风险、响应紧急事件，团队同时负责产品基座大模型的研发，我们在研发过程中需要解决数据合规、模型推理能力、多语种性能优化等方面的问题，从而为平台上的内容审核、搜索、推荐等多项业务提供安全合规，性能优越的基座模型。 课题介绍: 课题背景： 近年来，大规模语言模型（Large Language Models, LLM）在自然语言处理和人工智能的各个领域都取得了显著的进展。这些模型展示了强大的能力，例如在生成语言、回答问题、翻译文本等任务上表现优异。然而，LLM 的推理能力仍有很大的提升空间。在现有的研究中，通常依赖于大量的监督微调（Supervised Fine-Tuning, SFT）数据来增强模型的推理性能。然而，高质量 SFT 数据的获取成本高昂，这对模型的开发和应用带来了极大的限制。 为了提升推理能力，OpenAI 的 o1 系列模型通过增加思维链（Chain-of-Thought, CoT）的推理过程长度取得了一定的成功。这种方法虽然有效，但在实际测试时如何高效地进行扩展仍是一个开放的问题。一些研究尝试使用基于过程的奖励模型（Process-based Reward Model, PRM）、强化学习（Reinforcement Learning, RL）以及蒙特卡洛树搜索算法（Monte Carlo Tree Search, MCTS）等方法来解决推理问题，然而这些方法尚未能达到 OpenAI o1 系列模型的通用推理性能水平。最近deepseek r1在论文中提到通过纯强化学习的方法，可以使得 LLM 自主发展推理能力，而无需依赖昂贵的 SFT 数据。这一系列的工作都揭示着强化学习对LLM的巨大潜力。 课题挑战： 1、Reward模型的设计：在强化学习过程中，设计一个合适的reward模型是关键。Reward模型需要准确地反映推理过程的效果，并引导模型逐步提升其推理能力。这不仅要求对不同任务精准设定评估标准，还要确保reward模型能够在训练过程中动态调整，以适应模型性能的变化和提高。 2、稳定的训练过程：在缺乏高质量SFT数据的情况下，如何确保强化学习过程中的稳定训练是一个重大挑战。强化学习过程通常涉及大量的探索和试错，这可能导致训练不稳定甚至模型性能下降。需要开发具有鲁棒性的训练方法，以保证模型在训练过程中的稳定性和效果。 3、如何从数学和代码任务上拓展到自然语言任务上：现有的推理强化方法主要应用在数学和代码这些CoT数据量相对丰富的任务上。然而，自然语言任务的开放性和复杂性更高，如何将成功的RL策略从这些相对简单的任务拓展到自然语言处理任务上，要求对数据处理和RL方法进行深入的研究和创新，以实现跨任务的通用推理能力。 4、推理效率的提升：在保证推理性能的前提下，提升推理效率也是一个重要挑战。推理过程的效率直接影响到模型在实际应用中的可用性和经济性。可以考虑利用知识蒸馏技术，将复杂模型的知识传递给较小的模型，以减少计算资源消耗。另外，使用长思维链（Long Chain-of-Thought, Long-CoT）技术来改进短思维链（Short-CoT）模型，也是一种潜在的方法，以在保证推理质量的同时提升推理速度。

Team Introduction: Data AML is ByteDance's machine learning middle platform, providing training and inference systems for recommendation, advertising, CV (computer vision), speech, and NLP (natural language processing) across businesses such as Douyin, Toutiao, and Xigua Video. AML provides powerful machine learning computing capabilities to internal business units and conducts research on general and innovative algorithms to solve key business challenges. Additionally, through Volcano Engine, it delivers core machine learning and recommendation system capabilities to external enterprise clients. Beyond business applications, AML is also engaged in cutting-edge research in areas such as AI for Science and scientific computing. Research Project Introduction: Large-scale recommendation systems are being increasingly applied to short video, text community, image and other products, and the role of modal information in recommendation systems has become more prominent. ByteDance's practice has found that modal information can serve as a generalization feature to support business scenarios such as recommendation, and the research on end-to-end ultra-large-scale multimodal recommendation systems has enormous potential. It is expected to further explore directions such as multimodal cotraining, 7B/13B large-scale parameter models, and longer sequence end-to-end based on algorithm-engineering CoDesign. Engineering research directions include: Representation of multimodal samples Construction of high-performance multimodal inference engines based on the PyTorch framework Development of high-performance multimodal training frameworks Application of heterogeneous hardware in multimodal recommendation systems 1. Algorithmic research directions include: 2. Design of reasonable recommendation-advertising and multimodal cotraining architectures 3. Sparse Mixture of Experts (Sparse MOE) 4. Memory Network 5. Hybrid precision techniques 团队介绍： Data AML是字节跳动公司的机器学习中台，为抖音/今日头条/西瓜视频等业务提供推荐/广告/CV/语音/NLP的训练和推理系统。为公司内业务部门提供强大的机器学习算力，并在这些业务的问题上研究一些具有通用性和创新性的算法。同时，也通过火山引擎将一些机器学习/推荐系统的核心能力提供给外部企业客户。此外，AML还在AI for Science，科学计算等领域做一些前沿研究。 课题介绍： 大规模推荐系统正在越来越多的应用到短视频、文本社区、图像等产品上，模态信息在推荐系统中的作用也越来越大。 字节实践中发现模态信息能够很好的作为泛化特征支持推荐等业务场景，端到端的超大规模多模态推荐系统的研究具有非常大的想象空间。 期望在算法和工程CoDesign基础上，对多模态Cotrain、7B/13B大规模参数模型、更长序列端到端等方向进一步进行探索。 工程上研究方向包括多模态样本的表征、基于 pytorch 框架的高性能多模态推理引擎、高性能多模态训练框架的构建、异构硬件在多模态推荐系统上的应用；算法上的研究方向包括设计合理的推荐广告和多模态Cotrain结构、Sparse MOE、Memory Network、混合精度等。 1、负责机器学习系统架构的设计开发，以及系统性能调优； 2、负责解决系统高并发、高可靠性、高可扩展性等技术难关； 3、覆盖机器学习系统多个子方向领域的工作，包括：资源调度、任务编排、模型训练、模型推理、模型管理、数据集管理、工作流编排、ML for System等； 4、负责机器学习系统前瞻技术的调研和引入，比如：最新硬件架构、异构计算系统、GPU优化技术的引入落地； 5、研究基于机器学习方法，实现对集群/服务资源使用情况的分析和优化。

Team Introduction: TikTok is a global short-video platform available in 150 countries and regions. Our mission is to inspire creativity and bring joy by helping users discover real and interesting moments that make life better. TikTok's global headquarters are in Los Angeles and Singapore, and we also have offices in New York City, London, Dublin, Paris, Berlin, Dubai, Jakarta, Seoul, and Tokyo. TikTok Research & Development (R&D) Team: The TikTok R&D team is dedicated to building and maintaining industry-leading products that drive the success of TikTok’s global business. By joining us, you'll work on core scenarios such as user growth, social features, live streaming, e-commerce consumer side, content creation, and content consumption, helping our products scale rapidly across global markets. You'll also face deep technical challenges in areas like service architecture and infrastructure engineering, ensuring our systems operate with high quality, efficiency, and security. Meanwhile, our team also provides comprehensive technical solutions across diverse business needs, continuously optimizing product metrics and improving user experience. Here, you'll collaborate with leading experts in exploring cutting-edge technologies and pushing the boundaries of what's possible. Every line of your code will serve hundreds of millions of users. Our team is professional and goal-oriented, with an egalitarian and easy-going collaborative environment. Research Project Introduction: With the advancement of hardware computing and the continuous breakthroughs of large models in CV/NLP/multimodal learning and even AGI fields, the large computing driven in recommendation scenarios are increasingly capable of capturing user preferences in a more comprehensive and nuanced way. This enables a deeper understanding of user needs and the discovery of latent interests, ultimately leading to enhanced user experiences. As a critical component of short video recommendation systems, the ranking module is responsible for fine-grained matching between users and content, selecting the videos users are most likely to be engaged with. In this context, the key research focus is how to best leverage large computing to maximize the model’s memory, generalization, and reasoning capabilities. 团队介绍： TikTok是一个覆盖150个国家和地区的国际短视频平台，我们希望通过TikTok发现真实、有趣的瞬间，让生活更美好。TikTok 在全球各地设有办公室，全球总部位于洛杉矶和新加坡，办公地点还包括纽约、伦敦、都柏林、巴黎、柏林、迪拜、雅加达、首尔和东京等多个城市。 TikTok研发团队，旨在实现TikTok业务的研发工作，搭建及维护业界领先的产品。加入我们，你能接触到包括用户增长、社交、直播、电商C端、内容创造、内容消费等核心业务场景，支持产品在全球赛道上高速发展；也能接触到包括服务架构、基础技术等方向上的技术挑战，保障业务持续高质量、高效率、且安全地为用户服务；同时还能为不同业务场景提供全面的技术解决方案，优化各项产品指标及用户体验。 在这里， 有大牛带队与大家一同不断探索前沿， 突破想象空间。 在这里，你的每一行代码都将服务亿万用户。在这里，团队专业且纯粹，合作氛围平等且轻松。 课题介绍： 随着硬件算力的发展以及大模型在CV/NLP/多模态以至于AGI领域的不断突破，推荐场景下的大算力驱动能够帮助模型更全面深刻理解用户偏好，进而更好地理解用户需求，挖掘用户潜在兴趣，进而带来更好地用户体验。排序模块作为整个短视频推荐系统中非常重要的一环，承载着用户与视频之间的细粒度匹配挖掘进而挑选出用户最感兴趣的视频。如何找到合适的路径来最大化大算力下模型的记忆、泛化、推理能力，成为了研究的重中之重。

Team Introduction： Through algorithm optimization and collaboration with business teams, the team conducts comprehensive quality and ecosystem governance for ByteDance's e-commerce products. This involves combating risks, violations, and low-quality issues, as well as constructing and optimizing a healthy e-commerce ecosystem. The team aims to maximize platform governance effectiveness while improving operational efficiency and reducing costs. Additionally, the team is dedicated to advancing cutting-edge AI technologies to drive business transformation and development through technical innovation, covering diverse fields including but not limited to NLP, CV, multimodal models, large models, graph algorithms, and sequence algorithms. 团队介绍： 平台治理算法团队，通过优化算法，和业务团队协作，对字节旗下的电商产品进行全方位的质量和生态的治理，既包括风险、违规和低质问题的打击，也包括健康电商生态的建设和优化，在最大程度的优化平台治理的效果的同时提升治理的工作效率，降低成本。另外一方面，平台治理算法团队致力于攻坚前沿的AI技术，以技术驱动推动业务的变革和发展，领域涉及广泛，包括但不限于NLP/CV/多模态/大模型/图算法/序列算法等。 课题目标和必要性： 电商智能审核业务比较复杂，随着审核技术的不断演进，各个领域面临着新的风险问题和对抗形式，这对大模型的应用提出了新的挑战。例如，在电商审核业务中，涉及审核PBR变更、长文本、长时序、多语言、少样本和AIGC生成对抗等问题时，现有的开源大模型表现往往不尽人意。因此，针对这些挑战，我们亟需研发专门针对电商智能审核的大模型，以提升其在电商治理中的有效性和适应性。特别的，针对电商业务特点，我们需要探索高质量的数据自动生成、高效的MOE Embedding、Auto-prompt生成、高质量 COT输出、大模型知识蒸馏等。此外，该模型应能够满足电商审核业务的需求，实现高准确率的自主决策和可解释性的COT生成，显著减少误判。针对动态变化的审核PBR变更，它能够通过RAG模块自动检索类似的审核案例，将复杂的审核PBR分解为简单的原子任务，自动拆分出驳回和豁免原子任务，并自动调用相应的Tools来解决这些任务，从而建立“知道拒绝并且知道为何拒绝”的业内领先智能审核系统。最终，大模型智能审核系统的审核效果需要接近或者超过人工审核，往全机审的路线上演进。 课题内容： 电商智能审核多模态大模型，主要研究点包括但不限于： 1、模态融合能力：提升文本、音频、图像、视频和直播等多模态的细粒度理解能力，实现高准确率的自主决策和可解释性的COT生成； 2、Few-Shot能力：探索电商多语言、长时序和少样本问题，增强Few-Shot和Zero-Shot能力，针对多变的业务规则具备复杂指令和Auto-prompt生成能力； 3、攻防对抗能力：研究AIGC图像视频的判别，增强审核大模型对隐晦、抽象的生成式内容的攻防对抗能力； 4、 Agent能力：具备调用RAG模块，使用Tools，和Auto-planning能力；提升大模型的动态推理和反思能力。 涉及的研究方向：大模型，多模态大模型，Few-Shot，AIGC判别，AIGC数据生成，强化学习，Agent。