字节跳动大数据开发工程师
社招全职A166444A地点:北京状态:招聘
任职要求
1、熟悉一个大数据领域的开源框架,Hadoop/Hive/Flink/FlinkSQL/Spark/Kafka/Hbase/Redis rocksdb/Elasticsearch/Parquet; 2、熟悉Java、C++等编程语言; 3、出色的编码和Trouble-shoting能力; 4、乐于挑战没有明显答案的问题,对新技术有强烈的学习热情; 5、有PB级别数据处理经验加分; 6、对某一大数据框架有原理/源码级别理解的加分。
工作职责
1、为大规模推荐系统设计和实现合理的离线/实时数据架构; 2、设计和实现灵活可扩展、稳定、高性能的存储系统和计算模型; 3、生产系统的Trouble-shoting,设计和实现必要的机制和工具保障生产系统整体运行的稳定性; 4、打造业界领先的离在线存储、批式流式计算框架等分布式系统,为海量数据和大规模业务系统提供可靠的基础设施。
包括英文材料
大数据+
https://www.youtube.com/watch?v=bAyrObl7TYE
https://www.youtube.com/watch?v=H4bf_uuMC-g
With all this talk of Big Data, we got Rebecca Tickle to explain just what makes data into Big Data.
Hadoop+
https://www.runoob.com/w3cnote/hadoop-tutorial.html
Hadoop 为庞大的计算机集群提供可靠的、可伸缩的应用层计算和存储支持,它允许使用简单的编程模型跨计算机群集分布式处理大型数据集,并且支持在单台计算机到几千台计算机之间进行扩展。
[英文] Hadoop Tutorial
https://www.tutorialspoint.com/hadoop/index.htm
Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models.
Hive+
[英文] Hive Tutorial
https://www.tutorialspoint.com/hive/index.htm
Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.
https://www.youtube.com/watch?v=D4HqQ8-Ja9Y
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Kafka+
https://developer.confluent.io/what-is-apache-kafka/
https://www.youtube.com/watch?v=CU44hKLMg7k
https://www.youtube.com/watch?v=j4bqyAMMb7o&list=PLa7VYi0yPIH0KbnJQcMv5N9iW8HkZHztH
In this Apache Kafka fundamentals course, we introduce you to the basic Apache Kafka elements and APIs, as well as the broader Kafka ecosystem.
HBase+
[英文] HBase Tutorial
https://www.tutorialspoint.com/hbase/index.htm
HBase is a data model that is similar to Google's big table designed to provide quick random access to huge amounts of structured data. This tutorial provides an introduction to HBase, the procedures to set up HBase on Hadoop File Systems, and ways to interact with HBase shell.
Redis+
[英文] Developer Hub
https://redis.io/dev/
Get all the tutorials, learning paths, and more you need to start building—fast.
https://www.runoob.com/redis/redis-tutorial.html
REmote DIctionary Server(Redis) 是一个由 Salvatore Sanfilippo 写的 key-value 存储系统,是跨平台的非关系型数据库。
https://www.youtube.com/watch?v=jgpVdJB2sKQ
In this video I will be covering Redis in depth from how to install it, what commands you can use, all the way to how to use it in a real world project.
RocksDB+
https://rocksdb.org/docs/getting-started.html
The RocksDB library provides a persistent key value store.
ElasticSearch+
https://www.youtube.com/watch?v=a4HBKEda_F8
Learn about Elasticsearch with this comprehensive course designed for beginners, featuring both theoretical concepts and hands-on applications using Python (though applicable to any programming language). The course is structured in two parts: first covering essential Elasticsearch fundamentals including index management, document storage, text analysis, pipeline creation, search functionality, and advanced features like semantic search and embeddings; followed by a practical section where you'll build a real-world website using Elasticsearch as a search engine, working with the Astronomy Picture of the Day (APOD) dataset to implement features such as data cleaning pipelines, tokenization, pagination, and aggregations.
Parquet+
https://www.youtube.com/watch?v=KLFadWdomyI
Learn all about Apache Parquet, a column-based file format that's popular in the Hadoop/Spark ecosystem.
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
C+++
https://www.learncpp.com/
LearnCpp.com is a free website devoted to teaching you how to program in modern C++.
https://www.youtube.com/watch?v=ZzaPdXTrSb8
相关职位
社招网易数智
1、负责网易大数据平台的Iceberg等大数据组件迭代研发。 2、负责Iceberg等技术在业务上的实践落地以及问题分析诊断。 3、 参与Hive等组件在大数据元数据服务方面的稳定性建设以及问题诊断。
更新于 2025-04-17

社招5年以上技术
1、负责哈啰街猫业务基础数据的建设,包括基础数据模型建立和维护,报表的开发,业务系统的数据开发等; 2、理解哈啰街猫投喂、电商等业务,根据业务需求建立用户画像体系和标签体系,支持推荐和用户运营; 3、参与数据产品及应用的研发工作,挖掘数据业务价值,助力数据化运营;
更新于 2025-02-12

社招
团队内80%+都活跃在开源社区,有多名Committer. 欢迎对大数据底层技术有兴趣的小伙伴,一起挑战自我!(非数据仓库方向) 工作base可选:苏州/北京/成都 岗位描述: 基于hadoop/flink/spark/hive/cloud native等开源技术 1. 负责大数据集群规划、运维工作;负责大数据集群技术问题攻关,集群调优,源码解读,Bug fix等; 2. 负责大数据公共组件、中间件的开发工作; 3. 负责存储组件、批处理、流计算、OLAP、ML/DL,通过技术和业务场景的紧密结合,让数据发挥最大业务价值 4. 支撑数据中台建设;支撑业务结合需求设计高扩展、高性能、高可用的大数据业务系统;
更新于 2025-02-26