微店实时计算开发工程师(J10657)
社招全职2年以上地点：杭州状态：招聘
任职要求
任职要求： 1. 本科以上，计算机相关专业；两年以上大数据研发经验 2. 熟悉主流大数据技术，如Flink，Kafka，Hadoop、Spark、Hive、Tez、Presto、Hbase等 3. Java编程背景，有丰富的进程间通信\高并发\多线程研等研发经验 4. 熟悉Linux底层操作系统，能独立管理并快速搭建大数据相关运行环境 5. 对于K8S等容器化技术了解者优先 6. 有大规模集群管理、运维、优化迁移经验的优先 7. 参与Apche开源社区技术攻坚，有源码贡献者优先 8. 有使用VUE等框架进行前端开发经验加分
工作职责
职责描述： 1. 负责实时数据中台融合建设，参与实时离线混部项目, 负责相关系统架构设计，资源环境规划 2. 负责大规模集群管理，架构优化，资源深化 3. 负责平台服务接口设计&研发，标准&规范制定
包括英文材料
大数据+
https://www.youtube.com/watch?v=bAyrObl7TYE
https://www.youtube.com/watch?v=H4bf_uuMC-g
With all this talk of Big Data, we got Rebecca Tickle to explain just what makes data into Big Data.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
Kafka+
https://developer.confluent.io/what-is-apache-kafka/
https://www.youtube.com/watch?v=CU44hKLMg7k
https://www.youtube.com/watch?v=j4bqyAMMb7o&list=PLa7VYi0yPIH0KbnJQcMv5N9iW8HkZHztH
In this Apache Kafka fundamentals course, we introduce you to the basic Apache Kafka elements and APIs, as well as the broader Kafka ecosystem.
Hadoop+
https://www.runoob.com/w3cnote/hadoop-tutorial.html
Hadoop 为庞大的计算机集群提供可靠的、可伸缩的应用层计算和存储支持，它允许使用简单的编程模型跨计算机群集分布式处理大型数据集，并且支持在单台计算机到几千台计算机之间进行扩展。
[英文] Hadoop Tutorial
https://www.tutorialspoint.com/hadoop/index.htm
Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models.
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Hive+
[英文] Hive Tutorial
https://www.tutorialspoint.com/hive/index.htm
Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.
https://www.youtube.com/watch?v=D4HqQ8-Ja9Y
Presto+
[英文] What is Presto?
https://prestodb.io/what-is-presto/
https://www.tutorialspoint.com/apache_presto/index.htm
HBase+
[英文] HBase Tutorial
https://www.tutorialspoint.com/hbase/index.htm
HBase is a data model that is similar to Google's big table designed to provide quick random access to huge amounts of structured data. This tutorial provides an introduction to HBase, the procedures to set up HBase on Hadoop File Systems, and ways to interact with HBase shell.
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
高并发+
https://www.baeldung.com/concurrency-principles-patterns
In this tutorial, we’ll discuss some of the design principles and patterns that have been established over time to build highly concurrent applications.
https://www.baeldung.com/java-concurrency
Handling concurrency in an application can be a tricky process with many potential pitfalls. A solid grasp of the fundamentals will go a long way to help minimize these issues.
https://www.oreilly.com/library/view/concurrency-in-go/9781491941294/
You’ll understand how Go chooses to model concurrency, what issues arise from this model, and how you can compose primitives within this model to solve problems.
https://www.oreilly.com/library/view/modern-concurrency-in/9781098165406/
With this book, you'll explore the transformative world of Java 21's key feature: virtual threads.
https://www.youtube.com/watch?v=qyM8Pi1KiiM
https://www.youtube.com/watch?v=wEsPL50Uiyo
多线程+
https://liaoxuefeng.com/books/java/threading/basic/index.html
和单线程相比，多线程编程的特点在于：多线程经常需要读写共享数据，并且需要同步。
https://www.youtube.com/watch?v=_uQgGS_VIXM&list=PLsc-VaxfZl4do3Etp_xQ0aQBoC-x5BIgJ
https://www.youtube.com/watch?v=IEEhzQoKtQU
https://www.youtube.com/watch?v=mTGdtC9f4EU&list=PLL8woMHwr36EDxjUoCzboZjedsnhLP1j4
https://www.youtube.com/watch?v=TPVH_coGAQs&list=PLk6CEY9XxSIAeK-EAh3hB4fgNvYkYmghp
https://www.youtube.com/watch?v=xPqnoB2hjjA
This video is an introduction to multithreading in modern C++.
https://www.youtube.com/watch?v=YKBwKy5PrpQ
Rust threading is easy to implement and improves the efficiency of your applications on multi-core systems!
Linux+
https://ryanstutorials.net/linuxtutorial/
Ok, so you want to learn how to use the Bash command line interface (terminal) on Unix/Linux.
https://ubuntu.com/tutorials/command-line-for-beginners
The Linux command line is a text interface to your computer.
https://www.youtube.com/watch?v=6WatcfENsOU
In this Linux crash course, you will learn the fundamental skills and tools you need to become a proficient Linux system administrator.
https://www.youtube.com/watch?v=v392lEyM29A
Never fear the command line again, make it fear you.
https://www.youtube.com/watch?v=ZtqBQ68cfJc
Kubernetes+
https://kubernetes.io/docs/tutorials/kubernetes-basics/
This tutorial provides a walkthrough of the basics of the Kubernetes cluster orchestration system.
https://kubernetes.io/zh-cn/docs/tutorials/kubernetes-basics/
本教程介绍 Kubernetes 集群编排系统的基础知识。每个模块包含关于 Kubernetes 主要特性和概念的一些背景信息，还包括一个在线教程供你学习。
https://www.youtube.com/watch?v=s_o8dwzRlu4
Hands-On Kubernetes Tutorial | Learn Kubernetes in 1 Hour - Kubernetes Course for Beginners
https://www.youtube.com/watch?v=X48VuDVv0do
Full Kubernetes Tutorial | Kubernetes Course | Hands-on course with a lot of demos
Vue+
[英文] Tutorial | Vue.js
https://vuejs.org/tutorial/
The goal of this tutorial is to quickly give you an experience of what it feels like to work with Vue, right in the browser.
https://www.youtube.com/watch?v=VeNfHj6MhgA
3 Hour crash course of the Vue.js framework. You will learn all the fundamentals including components, directives, lifecycle, events and much more.
前端开发+
https://roadmap.sh/frontend
Step by step guide to becoming a modern frontend developer
相关职位
社招技术类
1、负责电商搜索、广告、推荐等核心业务数据体系及相关工具平台的规划、建设与持续优化，高效支持算法、分析、工程等团队的数据需求； 2、深入理解业务逻辑，抽象业务需求并设计可扩展、高性能的数据技术架构，快速响应业务变化，构建高效、可靠的数据互通与共享机制； 3、负责数据处理链路（离线/实时）的日常运维、监控与保障，确保数据稳定、高效产出，及时解决数据问题。
更新于 2025-07-23
社招66D2
字节跳动推荐架构团队，负责字节跳动旗下相关产品的推荐系统架构的设计和开发，保障系统稳定和高可用；负责在线服务、离线数据流性能优化，解决系统瓶颈，降低成本开销；抽象系统通用组件和服务，建设推荐中台、数据中台，支撑新产品快速孵化以及为ToB赋能。 1、为大规模推荐系统设计和实现合理的实时（流式计算）数据系统； 2、设计和实现灵活可扩展、稳定、高性能的存储系统和计算模型； 3、生产系统的trouble-shooting，设计和实现必要的机制和工具保障生产系统整体运行的稳定性； 4、打造业界领先的流式计算框架等分布式系统，为海量数据和大规模业务系统提供可靠的基础设施。
更新于 2021-09-26
社招A11239
1、为大规模推荐系统设计和实现合理的实时（流式计算）数据系统； 2、设计和实现灵活可扩展、稳定、高性能的存储系统和计算模型； 3、生产系统的Trouble-shooting，设计和实现必要的机制和工具保障生产系统整体运行的稳定性； 4、打造业界领先的流式计算框架等分布式系统，为海量数据和大规模业务系统提供可靠的基础设施。
更新于 2024-01-29
社招
团队介绍： 小鹏汽车自动驾驶的大数据方向，负责所有自动驾驶数据的云端处理，为自动驾驶业务提供高性能，高质量的数据加工，保证整个数据生产的稳定性，及时性，高可用。 1. 负责自动驾驶大数据多模态（如视频、图像、雷达信号等）湖仓平台的架构设计、开发与建设，包括数据处理、资源调度、算子管理、部署服务等；负责数据采集、清洗、转换和加载（ETL）流程的开发，处理多源异构数据 2. 基于大数据多模态湖仓平台，协助客户处理生产业务中的海量数据，解决疑难问题，支持百亿级自动驾驶感知和全栈数据的快速定位和分析，赋能上层业务发展。 3. 协助设计和优化数据仓库模型，参与数据治理工作（如数据质量核查、元数据管理等） 4. 负责自动驾驶离线和实时数据仓库的构建和性能优化；负责车端信号数据仓库体系和数据指标体系的架构设计与开发，为算法和数据闭环提供框架支持； 5. 调优分布式计算引擎（Spark/Flink/Presto）及存储系统（HDFS/OSS），构建OLAP引擎（Doris/StarRocks），解决海量数据场景下的资源瓶颈。 6. 跟踪Iceberg、Paimon、Flink、Spark、Lance等开源技术演进，主导关键组件二次开发或源码级优化；负责前沿技术的跟踪研究，工具链的选型测试，解决、攻克数据平台的核心技术难题。 7. 建立监控和反馈指标，持续优化改进产品的架构及性能，保证PB级数仓的数据质量和平台稳定性。
更新于 2024-12-24