百度大数据平台研发工程师(J85756)
社招全职1年以上ACG地点:北京状态:招聘
任职要求
-本科及以上学历,计算机专业优先 -熟悉Hadoop/Spark/Flink/Hive/Doris等任一项主流开源技术,具备类似技术的引擎研发经验 -有1年以上大规模系统软件的研发经验,如操作系统、存储系统、数据库系统、计算系统等,熟悉网络编程、多线程编程、操作系统原理 -对分布式数据计算或数据分析的原理、架构及使用调优有一定经验和深入见解者优先,熟悉Spark、Doris、Flink优先 -具备扎实的编程能力,良好的工程素养,团队合作能力和沟通能力强,抗压能力强,具有较强的自我驱动力
工作职责
-研发大规模分布式计算系统,包括高吞吐的批量计算、高时效的流式计算、高效能的OLAP等 -为全百度提供高性能、高可用、低成本的大数据计算服务 -探索新架构、新技术、新硬件,设计面向大模型的数据处理引擎和服务 -深入理解业务系统,理解项目客户需求,设计和实现符合用户需求的大数据平台系统,确保项目顺利验收
包括英文材料
学历+
Hadoop+
https://www.runoob.com/w3cnote/hadoop-tutorial.html
Hadoop 为庞大的计算机集群提供可靠的、可伸缩的应用层计算和存储支持,它允许使用简单的编程模型跨计算机群集分布式处理大型数据集,并且支持在单台计算机到几千台计算机之间进行扩展。
[英文] Hadoop Tutorial
https://www.tutorialspoint.com/hadoop/index.htm
Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models.
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
Hive+
[英文] Hive Tutorial
https://www.tutorialspoint.com/hive/index.htm
Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.
https://www.youtube.com/watch?v=D4HqQ8-Ja9Y
Doris+
https://doris.apache.org/docs/gettingStarted/what-is-apache-doris
网络编程+
https://www.youtube.com/watch?v=2HrYIl6GpYg
I will make a simple HTTP web server with the C Programming Language.
https://www.youtube.com/watch?v=8z6okCgdREo
This tutorial is for Gophers who have written a command line or an API application, but have little to no experience in lower-level concepts like reading and writing to sockets, working with channels, and managing multiple goroutines.
https://www.youtube.com/watch?v=bdIiTxtMaKA&list=PL9IEJIKnBJjH_zM5LnovnoaKlXML5qh17
https://www.youtube.com/watch?v=bzja9fQWzdA
Implement the ubiquitous TCP protocol that underlies much of the traffic on the internet!
[英文] 📺Network Programming with Python Course (build a port scanner, mailing client, chat room, DDOS)
https://www.youtube.com/watch?v=FGdiSJakIS4
Learn network programming in Python by building four projects. You will learn to build a mailing client, a DDOS script, a port scanner, and a TCP Chat Room.
https://www.youtube.com/watch?v=gntyAFoZp-E
https://www.youtube.com/watch?v=JiuouCJQzSQ
Explore the fundamentals of networking in Rust by building a simple TCP server.
https://www.youtube.com/watch?v=JRTLSxGf_6w
https://www.youtube.com/watch?v=sFizpxHkIlI
In this video we'll cover SOCKET PROGRAMMING in JAVA.
https://www.youtube.com/watch?v=sXW_sNGvqcU
多线程+
https://liaoxuefeng.com/books/java/threading/basic/index.html
和单线程相比,多线程编程的特点在于:多线程经常需要读写共享数据,并且需要同步。
https://www.youtube.com/watch?v=_uQgGS_VIXM&list=PLsc-VaxfZl4do3Etp_xQ0aQBoC-x5BIgJ
https://www.youtube.com/watch?v=IEEhzQoKtQU
https://www.youtube.com/watch?v=mTGdtC9f4EU&list=PLL8woMHwr36EDxjUoCzboZjedsnhLP1j4
https://www.youtube.com/watch?v=TPVH_coGAQs&list=PLk6CEY9XxSIAeK-EAh3hB4fgNvYkYmghp
https://www.youtube.com/watch?v=xPqnoB2hjjA
This video is an introduction to multithreading in modern C++.
https://www.youtube.com/watch?v=YKBwKy5PrpQ
Rust threading is easy to implement and improves the efficiency of your applications on multi-core systems!
数据分析+
[英文] Data Analyst Roadmap
https://roadmap.sh/data-analyst
Step by step guide to becoming an Data Analyst in 2025
相关职位
社招3年以上ACG
-负责大数据管理开发平台的研究、设计和开发,包括数据采集、存储、开发、处理、作业调度等工作 -负责大数据管理开发平台的设计和优化,提高产品的实用性和用户体验 -持续优化平台的稳定性和性能,维护系统的稳定运行 -深入理解业务系统,理解项目客户需求,设计和实现符合用户需求的大数据平台系统,确保项目顺利验收
更新于 2025-06-09
社招A118197
团队介绍:BABI平台是通过内部商业化/市场化的方式,以商品化为主要核心手段,围绕IT基础设施和人力成本提供一站式的平台支持,服务公司看清楚各业务部门从预算到成本到收入的经营过程,结合数据化、智能化的方式帮助业务线做好成本经营。 1、参与字节跳动技术团队及火山引擎业务等经营分析系统的大数据研发工作,包括但不限于BI可视化、OLAP分析、数据流调度、数据智能化服务等核心模块; 2、参与FinOps数据治理工作,持续提升数据易用性及数据质量,为经营分析提效赋能; 3、深入了解业务和系统痛点,给出优质的解决方案,并持续提升系统稳定性、可靠性、可扩展性,解决技术难题。
更新于 2024-12-03