YY直播大数据计算引擎研发工程师(J10508)
社招全职3年以上地点：广州状态：招聘
任职要求
1、本科及以上学历，计算机相关专业，3年及以上大数据平台研发经验； 2、熟悉操作系统、计算机网络、分布式系统原理、Java后端程序设计； 3、熟悉大数据技术栈,熟悉HDFS、Kafka、Hive、HBase、Flink、Yarn等多种组件和技术,有一定的系统性能优化及故障排除能力； 4、有Owner精神，善于沟通，主动性和责任心强，对数据敏感，逻辑性强，有良好的抗压能力，有良好的团队合作精神； 5、具备Spark，Flink，Presto至少一种源码分析经验或者开源社区有代码贡献者优先。
工作职责
1、负责公司大数据平台及应用平台的设计、开发、环境搭建、调优及故障诊断； 2、负责公司大数据计算组件平台级支持服务，以及大数据计算组件的研发和性能优化工作； 3、跟进相关计算组件社区最新动态，在确保平台稳定运行的同时升级新特性。
包括英文材料
学历+
大数据+
https://www.youtube.com/watch?v=bAyrObl7TYE
https://www.youtube.com/watch?v=H4bf_uuMC-g
With all this talk of Big Data, we got Rebecca Tickle to explain just what makes data into Big Data.
分布式系统+
https://www.distributedsystemscourse.com/
The home page of a free online class in distributed systems.
https://www.youtube.com/watch?v=7VbL89mKK3M&list=PLOE1GTZ5ouRPbpTnrZ3Wqjamfwn_Q5Y9A
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
HDFS+
https://hadoop.apache.org/docs/r1.2.1/hdfs_design.html
The Hadoop Distributed File System (HDFS) is a distributed file system designed to run on commodity hardware.
https://www.ibm.com/cn-zh/think/topics/hdfs
Hadoop 分布式文件系统 (HDFS) 是一种管理大型数据集的文件系统，可在商用硬件上运行。
Kafka+
https://developer.confluent.io/what-is-apache-kafka/
https://www.youtube.com/watch?v=CU44hKLMg7k
https://www.youtube.com/watch?v=j4bqyAMMb7o&list=PLa7VYi0yPIH0KbnJQcMv5N9iW8HkZHztH
In this Apache Kafka fundamentals course, we introduce you to the basic Apache Kafka elements and APIs, as well as the broader Kafka ecosystem.
Hive+
[英文] Hive Tutorial
https://www.tutorialspoint.com/hive/index.htm
Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.
https://www.youtube.com/watch?v=D4HqQ8-Ja9Y
HBase+
[英文] HBase Tutorial
https://www.tutorialspoint.com/hbase/index.htm
HBase is a data model that is similar to Google's big table designed to provide quick random access to huge amounts of structured data. This tutorial provides an introduction to HBase, the procedures to set up HBase on Hadoop File Systems, and ways to interact with HBase shell.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
Yarn+
[英文] Introduction
https://yarnpkg.com/getting-started
Yarn is an established open-source package manager used to manage dependencies in JavaScript projects.
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Presto+
[英文] What is Presto?
https://prestodb.io/what-is-presto/
https://www.tutorialspoint.com/apache_presto/index.htm
相关职位
社招D7195
1、参与快手EB级大数据平台计算引擎相关系统的研发与优化工作，解决实际业务需求与性能问题； 2、接受大数据平台系统设计与实现复杂度的挑战，分析和发现系统的优化点，负责推动系统的合理性、可靠性、可用性的提升； 3、和开源社区保持交流，从社区引入对公司业务场景有帮助的特性与系统，或将内部研发的功能贡献到社区。
更新于 2025-03-07
社招D7195
1、参与快手EB级大数据平台计算引擎相关系统的研发与优化工作，解决实际业务需求与性能问题； 2、接受大数据平台系统设计与实现复杂度的挑战，分析和发现系统的优化点，负责推动系统的合理性、可靠性、可用性的提升； 3、和开源社区保持交流，从社区引入对公司业务场景有帮助的特性与系统，或将内部研发的功能贡献到社区。
更新于 2025-03-07
社招公共技术
1.负责大数据相关计算引擎核心研发，为腾讯的智能湖仓打造领先业界的计算内核， 推进大数据业务的高效发展； 2.负责腾讯计算内核领域前沿技术调研，与开源社区保持交流，根据业务特性和需求，引入前沿技术。
更新于 2025-07-22
