
文远知行大数据开发工程师
社招全职3年以上地点:广州状态:招聘
任职要求
熟悉至少一个分布式大数据处理技术原理,包括但不限于Apache Flink/Apache Spark/Ray data/Google dataflow/MapReduce/Dask/Risingwave等计算引擎,熟读源码者尤佳; 熟练掌握c++/java/go中的一种开发语言; 了解分布式系统原理,有3年以上分布式环境开发经验,具备大规模数据处理的实践经验尤佳; 熟悉k8s/docker/bazel等开发部署环境; 了解分布式对象存储/数据湖/向量数据库,有实际工作经验尤佳; 良好的团队合作精神,具备较强的沟通能力和问题解决能力。 加分项: 知名开源分布式计算/存储引擎的committer; 有使用Ray Data等计算引擎处理海量非结构化数据的实际项目经验,对大规模异构计算有一定深度理解与认知。 文远知行(WeRide)成立于 2017 年,是全球领先的 L4 级自动驾驶科技公司,致力于“以无人驾驶改变人类出行”,已在全球超过 25 个城市开展自动驾驶研发、测试及运营,累积自动驾驶里程超1600万公里,应用场景覆盖智慧出行、智慧货运和智慧环卫,形成自动驾驶出租车、自动驾驶小巴、自动驾驶货运车、自动驾驶环卫车、高阶智能驾驶等五大产品矩阵,提供网约车、随需公交、同城货运、智能环卫、高阶智能驾驶解决方案等多种服务。 凭借“1个平台+3大场景+5大产品”的多元商业化战略,文远知行商业营收居同类自动驾驶企业之首,已与多家全球顶级主机厂和一级供应商达成战略合作伙伴关系,包括雷诺日产三菱联盟、宇通集团、博世、广汽集团等,不断为人类出行提供更多新选择。 文远知行目前的团队既有来自谷歌、微软、亚马逊、苹果、百度、滴滴的高阶工程师,也不乏刚从顶尖高校毕业的青年才俊。来到我们当中,你会发现这是一个专业、专注、有趣、有料的队伍。大家为了一个激动人心而富有挑战的目标走到一起,互相激励、脑力碰撞,为实现产品落地、创造社会价值、推进行业技术而努力。 对有抱负的工程师,还有什么比这更有意义的呢?我们虚位以待,真诚期待技术过硬、志趣相投的小伙伴加入我们! 更多信息请访问:http://www.weride.ai,或关注官方微信号:文远知行WeRide Infra团队介绍 基础架构团队为车端提供高可用、高性能和高可靠的车载系统,以及灵活的系统应用接口;离线工具链提供各类高效的开发集成平台和仿真平台,通过收集和分析自动驾驶过程中的海量数据,设计和执行大规模仿真任务,提供强大的工具帮助算法团队快速定位问题和解决问题。 基础架构团队负责设计和开发自研车载系统及中间件、开发高性能云计算系统、搭建大规模数据处理和分析平台、利用机器学习和模仿学习设计自动驾驶仿真算法、构建大规模仿真场景、以及研发支撑算法开发、测试、集成和部署的强大工具。
工作职责
岗位内容: 在自动驾驶领域,作为平台团队,支撑面向AI的海量数据的收集、高性能处理和检索管理。 设计、开发和维护业内领先的自研高性能的分布式大数据计算引擎,支持自动驾驶领域非结构化数据的高效处理; 负责自动驾驶领域大数据计算加工/机器学习推理链路的开发和维护,持续不断提升计算加工效率; 提供海量数据的存储与全生命周期管理,支撑海量数据的数据湖管理与向量化检索等能力。
包括英文材料
Apache+
https://www.apache.org/
The Apache® Software Foundation (ASF) provides software for the public good, guided by community over code.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Ray+
https://github.com/ray-project/ray
Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://www.youtube.com/watch?v=FhXfEXUUQp0
In this video, I'll teach you everything you need to know about Apache Ray!
https://www.youtube.com/watch?v=fMiAyj2kgac
Using powerful machine learning algorithms is easy using Ray.io and Python.
https://www.youtube.com/watch?v=q_aTbb7XeL4
Parallel and Distributed computing sounds scary until you try this fantastic Python library.
MapReduce+
https://www.youtube.com/watch?v=bcjSe0xCHbE
https://www.youtube.com/watch?v=cHGaQz0E7AU
In this video I explain the basics of Map Reduce model, an important concept for any software engineer to be aware of.
Dask+
https://tutorial.dask.org/00_overview.html
Dask is a parallel and distributed computing library that scales the existing Python and PyData ecosystem.
https://www.youtube.com/watch?v=jstCmSD_LAs
In this video, you will learn how to use Dask, a Python module that enables pandas code to run in parallel on your local machine or scaled out to multiple machines.
C+++
https://www.learncpp.com/
LearnCpp.com is a free website devoted to teaching you how to program in modern C++.
https://www.youtube.com/watch?v=ZzaPdXTrSb8
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
Go+
https://www.youtube.com/watch?v=8uiZC0l4Ajw
学习Golang的完整教程!从开始到结束不到一个小时,包括如何在Go中构建API的完整演示。没有多余的内容,只有你需要知道的知识。
分布式系统+
https://www.distributedsystemscourse.com/
The home page of a free online class in distributed systems.
https://www.youtube.com/watch?v=7VbL89mKK3M&list=PLOE1GTZ5ouRPbpTnrZ3Wqjamfwn_Q5Y9A
Kubernetes+
https://kubernetes.io/docs/tutorials/kubernetes-basics/
This tutorial provides a walkthrough of the basics of the Kubernetes cluster orchestration system.
https://kubernetes.io/zh-cn/docs/tutorials/kubernetes-basics/
本教程介绍 Kubernetes 集群编排系统的基础知识。每个模块包含关于 Kubernetes 主要特性和概念的一些背景信息,还包括一个在线教程供你学习。
https://www.youtube.com/watch?v=s_o8dwzRlu4
Hands-On Kubernetes Tutorial | Learn Kubernetes in 1 Hour - Kubernetes Course for Beginners
https://www.youtube.com/watch?v=X48VuDVv0do
Full Kubernetes Tutorial | Kubernetes Course | Hands-on course with a lot of demos
Docker+
https://www.youtube.com/watch?v=GFgJkfScVNU
Master Docker in one course; learn about images and containers on Docker Hub, running multiple containers with Docker Compose, automating workflows with Docker Compose Watch, and much more. 🐳
https://www.youtube.com/watch?v=kTp5xUtcalw
Learn how to use Docker and Kubernetes in this complete hand-on course for beginners.
自动驾驶+
https://www.youtube.com/watch?v=_q4WUxgwDeg&list=PL05umP7R6ij321zzKXK6XCQXAaaYjQbzr
Lecture: Self-Driving Cars (Prof. Andreas Geiger, University of Tübingen)
https://www.youtube.com/watch?v=NkI9ia2cLhc&list=PLB0Tybl0UNfYoJE7ZwsBQoDIG4YN9ptyY
You will learn to make a self-driving car simulation by implementing every component one by one. I will teach you how to implement the car driving mechanics, how to define the environment, how to simulate some sensors, how to detect collisions and how to make the car control itself using a neural network.
HTTP+
https://developer.mozilla.org/zh-CN/docs/Web/HTTP
超文本传输协议(HTTP)是一个用于传输超媒体文档(例如 HTML)的应用层协议。它是为 Web 浏览器与 Web 服务器之间的通信而设计的,但也可以用于其他目的。
高可用+
https://redis.io/blog/high-availability-architecture/
A high available architecture is when there are a number of different components, modules, or services that work together to maintain optimal performance, irrespective of peak-time loads.
https://www.ibm.com/think/topics/high-availability
High availability (HA) is a term that refers to a system’s ability to be accessible and reliable close to 100% of the time.
算法+
https://roadmap.sh/datastructures-and-algorithms
Step by step guide to learn Data Structures and Algorithms in 2025
https://www.hellointerview.com/learn/code
A visual guide to the most important patterns and approaches for the coding interview.
https://www.w3schools.com/dsa/
中间件+
https://www.youtube.com/watch?v=1oWPUpMheGk
机器学习+
https://www.youtube.com/watch?v=0oyDqO8PjIg
Learn about machine learning and AI with this comprehensive 11-hour course from @LunarTech_ai.
https://www.youtube.com/watch?v=i_LwzRVP7bg
Learn Machine Learning in a way that is accessible to absolute beginners.
https://www.youtube.com/watch?v=NWONeJKn6kc
Learn the theory and practical application of machine learning concepts in this comprehensive course for beginners.
https://www.youtube.com/watch?v=PcbuKRNtCUc
Learn about all the most important concepts and terms related to machine learning and AI.
相关职位
社招网易数智
1、负责网易大数据平台的Iceberg等大数据组件迭代研发。 2、负责Iceberg等技术在业务上的实践落地以及问题分析诊断。 3、 参与Hive等组件在大数据元数据服务方面的稳定性建设以及问题诊断。
更新于 2025-04-17
社招A166444A
1、为大规模推荐系统设计和实现合理的离线/实时数据架构; 2、设计和实现灵活可扩展、稳定、高性能的存储系统和计算模型; 3、生产系统的Trouble-shoting,设计和实现必要的机制和工具保障生产系统整体运行的稳定性; 4、打造业界领先的离在线存储、批式流式计算框架等分布式系统,为海量数据和大规模业务系统提供可靠的基础设施。
更新于 2025-02-20

社招5年以上技术
1、负责哈啰街猫业务基础数据的建设,包括基础数据模型建立和维护,报表的开发,业务系统的数据开发等; 2、理解哈啰街猫投喂、电商等业务,根据业务需求建立用户画像体系和标签体系,支持推荐和用户运营; 3、参与数据产品及应用的研发工作,挖掘数据业务价值,助力数据化运营;
更新于 2025-02-12

社招
团队内80%+都活跃在开源社区,有多名Committer. 欢迎对大数据底层技术有兴趣的小伙伴,一起挑战自我!(非数据仓库方向) 工作base可选:苏州/北京/成都 岗位描述: 基于hadoop/flink/spark/hive/cloud native等开源技术 1. 负责大数据集群规划、运维工作;负责大数据集群技术问题攻关,集群调优,源码解读,Bug fix等; 2. 负责大数据公共组件、中间件的开发工作; 3. 负责存储组件、批处理、流计算、OLAP、ML/DL,通过技术和业务场景的紧密结合,让数据发挥最大业务价值 4. 支撑数据中台建设;支撑业务结合需求设计高扩展、高性能、高可用的大数据业务系统;
更新于 2025-02-26