字节跳动大数据计算引擎开发工程师-数据平台
校招全职A58657地点：北京状态：招聘
任职要求
1、2026届获得本科及以上学历，计算机相关专业优先； 2、熟练掌握Java或C++编程，具备扎实的计算机基础知识（包含数据结构、算法、多线程编程、I/O、操作系统等）； 3、熟悉Spark、Doris、StarRocks、Presto、Kylin、Hive、Impala、Flink、ClickHouse、Ray、Lance、Paimon、Iceberg、Hudi等主流大数据系统原理及源码优先； 4、熟悉主流的OLAP引擎的优化原理，如CBO、向量化执行、物化视图、列式存储、Shuffle优化等。
工作职责
团队介绍：数据平台是字节跳动数据中台部门，为公司多业务线（包括抖音、电商、直播和生活服务等）提供一站式大数据解决方案，涵盖数据的生产、清洗、传输、建模、分析等全流程链路，提供数据开发、实验评估、画像标签、增强分析等多元场景解决能力。同时，数据平台部门也致力于把字节跳动积累沉淀的数据中台解决方案做商业化输出，让更多行业能够应用我们的产品能力构建自己的数据中台。在火山引擎上，我们提供了营销增长套件，数据中台等相关产品解决方案，为泛互联网、金融、汽车、新零售等行业提供了行业解决方案。 1、打造业界PB级OLAP引擎，支撑字节跳动主要产品线（如抖音、今日头条）的ETL、湖仓分析场景等； 2、参与以Spark、Doris、Presto、Ray、Lance等为代表的大数据引擎内核优化； 3、打造基于高并发大流量业务场景的大数据查询服务、任务调度系统、异构数据湖统一元数据服务及权限管理服务的设计和研发； 4、参与字节跳动数据引擎的架构设计、业务场景支持、性能优化与降本、稳定性提升。
包括英文材料
学历+
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
C+++
https://www.learncpp.com/
LearnCpp.com is a free website devoted to teaching you how to program in modern C++.
https://www.youtube.com/watch?v=ZzaPdXTrSb8
数据结构+
https://www.youtube.com/watch?v=8hly31xKli0
In this course you will learn about algorithms and data structures, two of the fundamental topics in computer science.
https://www.youtube.com/watch?v=B31LgI4Y4DQ
Learn about data structures in this comprehensive course. We will be implementing these data structures in C or C++.
https://www.youtube.com/watch?v=CBYHwZcbD-s
Data Structures and Algorithms full course tutorial java
算法+
https://roadmap.sh/datastructures-and-algorithms
Step by step guide to learn Data Structures and Algorithms in 2025
https://www.hellointerview.com/learn/code
A visual guide to the most important patterns and approaches for the coding interview.
https://www.w3schools.com/dsa/
多线程+
https://liaoxuefeng.com/books/java/threading/basic/index.html
和单线程相比，多线程编程的特点在于：多线程经常需要读写共享数据，并且需要同步。
https://www.youtube.com/watch?v=_uQgGS_VIXM&list=PLsc-VaxfZl4do3Etp_xQ0aQBoC-x5BIgJ
https://www.youtube.com/watch?v=IEEhzQoKtQU
https://www.youtube.com/watch?v=mTGdtC9f4EU&list=PLL8woMHwr36EDxjUoCzboZjedsnhLP1j4
https://www.youtube.com/watch?v=TPVH_coGAQs&list=PLk6CEY9XxSIAeK-EAh3hB4fgNvYkYmghp
https://www.youtube.com/watch?v=xPqnoB2hjjA
This video is an introduction to multithreading in modern C++.
https://www.youtube.com/watch?v=YKBwKy5PrpQ
Rust threading is easy to implement and improves the efficiency of your applications on multi-core systems!
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Doris+
https://doris.apache.org/docs/gettingStarted/what-is-apache-doris
StarRocks+
https://docs.starrocks.io/docs/quick_start/
These Quick Start guides will help you get going with a small StarRocks environment.
https://itnext.io/introduction-to-starrocks-a-new-modern-analytical-database-1db2177d26e1
Recently, I had the opportunity to explore StarRocks which is the new kid in the block when talking about massive scale databases which are able to handle petabytes of data.
Presto+
[英文] What is Presto?
https://prestodb.io/what-is-presto/
https://www.tutorialspoint.com/apache_presto/index.htm
Hive+
[英文] Hive Tutorial
https://www.tutorialspoint.com/hive/index.htm
Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.
https://www.youtube.com/watch?v=D4HqQ8-Ja9Y
Impala+
[英文] Impala Tutorials
https://impala.apache.org/docs/build/html/topics/impala_tutorial.html
This section includes tutorial scenarios that demonstrate how to begin using Impala.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
ClickHouse+
[英文] Advanced Tutorial
https://clickhouse.com/docs/tutorial
Learn how to ingest and query data in ClickHouse using the New York City taxi example dataset.
https://www.youtube.com/watch?v=FtoWGT7kS-c
ClickHouse is an open-source column-oriented DBMS for online analytical processing that allows users to generate analytical reports using SQL queries in real-time.
https://www.youtube.com/watch?v=Rhe-kUyrFUE&list=PL0Z2YDlm0b3gcY5R_MUo4fT5bPqUQ66ep
Ray+
https://github.com/ray-project/ray
Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://www.youtube.com/watch?v=FhXfEXUUQp0
In this video, I'll teach you everything you need to know about Apache Ray!
https://www.youtube.com/watch?v=fMiAyj2kgac
Using powerful machine learning algorithms is easy using Ray.io and Python.
https://www.youtube.com/watch?v=q_aTbb7XeL4
Parallel and Distributed computing sounds scary until you try this fantastic Python library.
Iceberg+
https://iceberg.apache.org/spark-quickstart/
This guide will get you up and running with Apache Iceberg™ using Apache Spark™, including sample code to highlight some powerful features.
https://www.baeldung.com/apache-iceberg-intro
This tutorial will discuss Apache Iceberg, a popular open table format in today’s big data landscape.
https://www.youtube.com/watch?v=TsmhRZElPvM
You’ve probably heard about Apache Iceberg™—after all, it’s been getting a lot of buzz.
Hudi+
[英文] Spark Quick Start
https://hudi.apache.org/docs/quick-start-guide
we will walk through code snippets that allows you to insert, update, delete and query a Hudi table.
https://www.oreilly.com/library/view/apache-hudi-the/9781098173821/
Overcome challenges in building transactional guarantees on rapidly changing data by using Apache Hudi.
https://www.youtube.com/watch?v=pyK18sDYnS0
In this video, I'll introduce you to one of the most popular Data Lake solutions out there, Apache Hudi!
OLAP+
https://www.youtube.com/watch?v=iw-5kFzIdgY
OLAP (for online analytical processing) is software for performing multidimensional analysis at high speeds on large volumes of data from a data warehouse, data mart, or some other unified, centralized data store.
相关职位
校招数据引擎
【业务介绍】 数据引擎是大数据Infra团队，提供大数据存储、计算、平台基础能力，支持EB级别数据的实时集成、开发加工、消费使用的全链路数据解决方案。 【职位描述】 1. 负责大数据计算引擎以及数据平台的研发工作，支撑公司内社区、电商以及商业化业务； 2. 面向PB级海量数据的解决方案设计，处理每日千亿级增量的用户行为数据； 3. 通过提供优秀的引擎和平台产品，配合数据产品及数据仓库团队，建设高效的数据研发体系，提供高效稳定的数据服务，降低数据的使用门槛，实现数据的最大价值。
社招3年以上D7209
1、参与快手大数据运维产品建设，包括大数据计算引擎运维平台与业务控制台开发落地，保障引擎运维效率以及提升业务使用计算引擎易用性； 2、接受大数据平台系统设计与实现复杂度的挑战，分析和发现系统的优化点，负责推动系统的合理性、可靠性、可用性的提升； 3、为团队引入创新的技术、创新的解决方案，用创新的思路解决问题。
更新于 2025-03-07
社招3年以上D11431
1、参与公司大数据开发与管理平台、数据资产及营销平台的研发工作，包括大数据开发平台、质量平台、安全中心、数据资产平台、营销平台等； 2、充分利用大数据技术、微服务、分布式系统等技术，构建高可用、高扩展和低耦合高内聚的数据中台服务； 3、熟悉业界技术体系，为快手数据产品研发引入创造性的技术方案，解决面临的各种复杂问题和挑战。
更新于 2024-10-10
社招3年以上技术类-开发
1、分布式 NoSQL 存储架构设计与优化：负责大规模分布式 NoSQL存储系统的架构设计、核心模块开发及性能调优，支持 PB 级数据存储需求。 2、实现多租户资源隔离、优先级调度与 SLA 保障，支撑智能信息搜索/推荐/LLM 等高性能业务场景。 3、大数据计算引擎研发与优化：设计并开发流、批计算框架，构建基于 DAG 的动态任务调度系统，实现万级并发任务的资源利用率与 SLA 平衡。 4、攻克每日 PB 级流、批数据处理的稳定性挑战，优化大规模 Join/Aggregation 场景下的 IO 与网络带宽瓶颈。
更新于 2025-09-08