快手【留用实习】大模型训练引擎性能优化工程师
实习兼职J1020地点:北京状态:招聘
任职要求
1.本科及以上学历,计算机科学、软件工程或相关专业优先,具备扎实的计算机基础知识。 2.熟练掌握C++和Python编程语言,熟悉Linux开发环境,具备良好的代码编写习惯。 3.了解深度学习框架(如PyTorch、TensorFlow等),熟悉Transformer架构及其优化技术。 4.具备使用性能分析工具(如nsys、torch profiler等)的经验,能够快速定位并解决性能问题。 5.具备良好的团队合作精神和沟通能力,能够快速融入团队并高效协作。 6.具备较强的学习能力和抗压能力,能够在快节奏的环境中成长。 加分项: 1.具备CUDA编程能力,熟悉GPU加速技术。 2.有大模型训练或分布式系统开发经验。 3.在相关领域发表过学术论文或参与过开源项目。
工作职责
1.性能分析与优化:使用先进的性能分析工具(如nsys、torch profiler等),对大模型训练引擎进行性能剖析,定位瓶颈并提出优化方案。 2.分布式训练优化:深入研究分布式训练技术,探索并实现高效的并行策略,提升大规模模型训练的速度与效率。 3.代码优化与重构:参与训练引擎代码的优化与重构工作,提升代码质量与运行效率,确保系统的稳定性和可扩展性。 4.前沿技术研究:跟踪大模型训练领域的最新研究成果,复现并集成先进的优化技术,推动训练引擎的性能提升。
包括英文材料
学历+
C+++
https://www.learncpp.com/
LearnCpp.com is a free website devoted to teaching you how to program in modern C++.
https://www.youtube.com/watch?v=ZzaPdXTrSb8
Python+
https://liaoxuefeng.com/books/python/introduction/index.html
中文,免费,零起点,完整示例,基于最新的Python 3版本。
https://www.learnpython.org/
a free interactive Python tutorial for people who want to learn Python, fast.
https://www.youtube.com/watch?v=K5KVEU3aaeQ
Master Python from scratch 🚀 No fluff—just clear, practical coding skills to kickstart your journey!
https://www.youtube.com/watch?v=rfscVS0vtbw
This course will give you a full introduction into all of the core concepts in python.
Linux+
https://ryanstutorials.net/linuxtutorial/
Ok, so you want to learn how to use the Bash command line interface (terminal) on Unix/Linux.
https://ubuntu.com/tutorials/command-line-for-beginners
The Linux command line is a text interface to your computer.
https://www.youtube.com/watch?v=6WatcfENsOU
In this Linux crash course, you will learn the fundamental skills and tools you need to become a proficient Linux system administrator.
https://www.youtube.com/watch?v=v392lEyM29A
Never fear the command line again, make it fear you.
https://www.youtube.com/watch?v=ZtqBQ68cfJc
深度学习+
https://d2l.ai/
Interactive deep learning book with code, math, and discussions.
PyTorch+
https://datawhalechina.github.io/thorough-pytorch/
PyTorch是利用深度学习进行数据科学研究的重要工具,在灵活性、可读性和性能上都具备相当的优势,近年来已成为学术界实现深度学习算法最常用的框架。
https://www.youtube.com/watch?v=V_xro1bcAuA
Learn PyTorch for deep learning in this comprehensive course for beginners. PyTorch is a machine learning framework written in Python.
TensorFlow+
https://www.youtube.com/watch?v=tpCFfeUEGs8
Ready to learn the fundamentals of TensorFlow and deep learning with Python? Well, you’ve come to the right place.
https://www.youtube.com/watch?v=ZUKz4125WNI
This part continues right where part one left off so get that Google Colab window open and get ready to write plenty more TensorFlow code.
Transformer+
https://huggingface.co/learn/llm-course/en/chapter1/4
Breaking down how Large Language Models work, visualizing how data flows through.
https://poloclub.github.io/transformer-explainer/
An interactive visualization tool showing you how transformer models work in large language models (LLM) like GPT.
https://www.youtube.com/watch?v=wjZofJX0v4M
Breaking down how Large Language Models work, visualizing how data flows through.
CUDA+
https://developer.nvidia.com/blog/even-easier-introduction-cuda/
This post is a super simple introduction to CUDA, the popular parallel computing platform and programming model from NVIDIA.
https://www.youtube.com/watch?v=86FAWCzIe_4
Lean how to program with Nvidia CUDA and leverage GPUs for high-performance computing and deep learning.
大模型+
https://www.youtube.com/watch?v=xZDB1naRUlk
You will build projects with LLMs that will enable you to create dynamic interfaces, interact with vast amounts of text data, and even empower LLMs with the capability to browse the internet for research papers.
https://www.youtube.com/watch?v=zjkBMFhNj_g
分布式系统+
https://www.distributedsystemscourse.com/
The home page of a free online class in distributed systems.
https://www.youtube.com/watch?v=7VbL89mKK3M&list=PLOE1GTZ5ouRPbpTnrZ3Wqjamfwn_Q5Y9A
相关职位
实习J1014
1、参与快手大规模深度学习推理引擎、大模型训练解决方案的研发与优化,包括大模型推理、模型训练框架、微调平台等; 2、参与底层算子的优化、通过优化访存pattern、计算提升推理性能,与算法部门合作,为公司大模型定制训练方案,探索RLHF、MoE、多模态、longcontext等前沿方向,提升训练性能; 3、优化推理框架上层调度策略,通过机内、机间的计算任务调度和通讯优化提升引擎性能;优化现有大语言模型相关工具和平台,提高模型训练、维护效率,降低成本,提升训练服务稳定性。
更新于 2025-05-23
实习J1020
1、负责依据不同业务场景的特点和新硬件特性,结合系统软硬件栈的整体调优,提出并实施性能优化方案; 2、负责持续跟踪业内软硬件相关领域的技术发展趋势,结合不同业务场景未来需求,开展方案预研以及推广应用工作。 具体包括以下两种场景或者两种之一: 1)AI计算相关场景,例如:大模型训练场景,AIGC、NLP、推荐等常规推理场景; 2)以容器云、大数据计算平台为例的通用计算平台场景。
更新于 2025-03-11
实习J1020
1、负责依据不同业务场景的特点和新硬件特性,结合系统软硬件栈的整体调优,提出并实施性能优化方案; 2、负责持续跟踪业内软硬件相关领域的技术发展趋势,结合不同业务场景未来需求,开展方案预研以及推广应用工作; 具体包括以下三种场景或者三种之一: 1)以容器云、大数据计算平台为例的通用计算平台场景; 2)AI计算相关场景,例如:大模型训练场景,AIGC、NLP、推荐等常规推理场景; 3)结构化以及非结构化数据存储场景。
更新于 2025-03-04