小鹏汽车大数据平台研发高级/资深工程师
社招全职地点：广州状态：招聘
任职要求
-掌握hadoop、hive、datalake、spark、flink、ray、doris、kafka、es、hbase、milvus等一种或者几种平台的应用经验，并对其架构和关键底层技术原理或数据结构有深入理解，有源码经验尤佳 -掌握分布式系统设计原理，对分布式计算或查询系统有较深入研究 -精通JAVA、或熟练掌握Rust编程语言，对数据结构和算法设计有较为深刻的理解 -掌握优秀技术组件内核原理（如FAISS、Arrow、Akka Actor、Netty、Lucene、LSM Tree实现等），有一定源码经验尤佳 -熟悉云计算微服务相关技术，如Spring Cloud、Kubernets、Mesos、Docker等 -熟悉Linux操作系统，对shell编程有一定了解 -了解数据仓库和ETL相关技术 -很强的学习、分析和解决问题能力，良好的团队意识和协作精神，有较强的内外沟通能力
工作职责
-负责自驾大数据平台研发，实现分布式大数据计算和查询引擎以及分布式中间件 -支持海量数据计算以及高吞吐高并发数据应用场景，提升大数据产品性能和研发效率
包括英文材料
Hadoop+
https://www.runoob.com/w3cnote/hadoop-tutorial.html
Hadoop 为庞大的计算机集群提供可靠的、可伸缩的应用层计算和存储支持，它允许使用简单的编程模型跨计算机群集分布式处理大型数据集，并且支持在单台计算机到几千台计算机之间进行扩展。
[英文] Hadoop Tutorial
https://www.tutorialspoint.com/hadoop/index.htm
Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models.
Hive+
[英文] Hive Tutorial
https://www.tutorialspoint.com/hive/index.htm
Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.
https://www.youtube.com/watch?v=D4HqQ8-Ja9Y
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
Ray+
https://github.com/ray-project/ray
Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
https://www.youtube.com/watch?v=FhXfEXUUQp0
In this video, I'll teach you everything you need to know about Apache Ray!
https://www.youtube.com/watch?v=fMiAyj2kgac
Using powerful machine learning algorithms is easy using Ray.io and Python.
https://www.youtube.com/watch?v=q_aTbb7XeL4
Parallel and Distributed computing sounds scary until you try this fantastic Python library.
Doris+
https://doris.apache.org/docs/gettingStarted/what-is-apache-doris
Kafka+
https://developer.confluent.io/what-is-apache-kafka/
https://www.youtube.com/watch?v=CU44hKLMg7k
https://www.youtube.com/watch?v=j4bqyAMMb7o&list=PLa7VYi0yPIH0KbnJQcMv5N9iW8HkZHztH
In this Apache Kafka fundamentals course, we introduce you to the basic Apache Kafka elements and APIs, as well as the broader Kafka ecosystem.
ElasticSearch+
https://www.youtube.com/watch?v=a4HBKEda_F8
Learn about Elasticsearch with this comprehensive course designed for beginners, featuring both theoretical concepts and hands-on applications using Python (though applicable to any programming language). The course is structured in two parts: first covering essential Elasticsearch fundamentals including index management, document storage, text analysis, pipeline creation, search functionality, and advanced features like semantic search and embeddings; followed by a practical section where you'll build a real-world website using Elasticsearch as a search engine, working with the Astronomy Picture of the Day (APOD) dataset to implement features such as data cleaning pipelines, tokenization, pagination, and aggregations.
HBase+
[英文] HBase Tutorial
https://www.tutorialspoint.com/hbase/index.htm
HBase is a data model that is similar to Google's big table designed to provide quick random access to huge amounts of structured data. This tutorial provides an introduction to HBase, the procedures to set up HBase on Hadoop File Systems, and ways to interact with HBase shell.
Milvus+
[英文] Tutorials Overview
https://milvus.io/docs/tutorials-overview.md
This page provides a list of tutorials for you to interact with Milvus.
https://www.baeldung.com/milvus-tutorial-intro
In this tutorial, we’ll explore Milvus, a highly scalable open-source vector database.
https://www.youtube.com/watch?v=7ejr_ZzU9jw
Discover the power of Milvus, an open-source vector database revolutionizing AI applications.
https://www.youtube.com/watch?v=Yhv19le0sBw
Vector databases have been trending recently as they power modern search, recommendations, and AI-driven applications.
数据结构+
https://www.youtube.com/watch?v=8hly31xKli0
In this course you will learn about algorithms and data structures, two of the fundamental topics in computer science.
https://www.youtube.com/watch?v=B31LgI4Y4DQ
Learn about data structures in this comprehensive course. We will be implementing these data structures in C or C++.
https://www.youtube.com/watch?v=CBYHwZcbD-s
Data Structures and Algorithms full course tutorial java
分布式系统+
https://www.distributedsystemscourse.com/
The home page of a free online class in distributed systems.
https://www.youtube.com/watch?v=7VbL89mKK3M&list=PLOE1GTZ5ouRPbpTnrZ3Wqjamfwn_Q5Y9A
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
Rust+
https://www.youtube.com/watch?v=BpPEoZW5IiY
In this comprehensive Rust course for beginners, you will learn about the core concepts of the language and underlying mechanisms in theory.
https://www.youtube.com/watch?v=lzKeecy4OmQ
Full Rust 101 Crash Course for beginners.
https://www.youtube.com/watch?v=rQ_J9WH6CGk
算法+
https://roadmap.sh/datastructures-and-algorithms
Step by step guide to learn Data Structures and Algorithms in 2025
https://www.hellointerview.com/learn/code
A visual guide to the most important patterns and approaches for the coding interview.
https://www.w3schools.com/dsa/
内核+
https://www.youtube.com/watch?v=C43VxGZ_ugU
I rummage around the Linux kernel source and try to understand what makes computers do what they do.
https://www.youtube.com/watch?v=HNIg3TXfdX8&list=PLrGN1Qi7t67V-9uXzj4VSQCffntfvn42v
Learn how to develop your very own kernel from scratch in this programming series!
https://www.youtube.com/watch?v=JDfo2Lc7iLU
Denshi goes over a simple explanation of what computer kernels are and how they work, alonside what makes the Linux kernel any special.
Faiss+
https://faiss.ai/index.html
Faiss is a library for efficient similarity search and clustering of dense vectors.
https://huggingface.co/learn/llm-course/en/chapter5/6
In this section we’ll use this information to build a search engine that can help us find answers to our most pressing questions about the library!
Akka+
https://doc.akka.io/libraries/akka-core/current/typed/guide/index.html
Akka is a toolkit for building highly concurrent, distributed, and resilient message-driven applications for Java and Scala.
https://www.baeldung.com/akka-actors-java
Akka is an open-source library that helps to easily develop concurrent and distributed applications using Java or Scala by leveraging the Actor Model.
微服务+
https://learn.microsoft.com/en-us/training/modules/dotnet-microservices/
Microservice applications are composed of small, independently versioned, and scalable customer-focused services that communicate with each other by using standard protocols and well-defined interfaces.
https://microservices.io/
Microservices - also known as the microservice architecture - is an architectural style that structures an application as a collection of two or more services.
https://spring.io/microservices
Building small, self-contained, ready to run applications can bring great flexibility and added resilience to your code.
https://www.ibm.com/think/topics/microservices
Microservices, or microservices architecture, is a cloud-native architectural approach in which a single application is composed of many loosely coupled and independently deployable smaller components or services.
https://www.youtube.com/watch?v=CqCDOosvZIk
https://www.youtube.com/watch?v=hmkF77F9TLw
Learn about software system design and microservices.
Spring+
https://liaoxuefeng.com/books/java/spring/index.html
Spring是一个支持快速开发Java EE应用程序的框架。它提供了一系列底层容器和基础设施，并可以和大量常用的开源框架无缝集成，可以说是开发Java EE应用程序的必备。
https://spring.io/guides/gs/rest-service
https://spring.io/quickstart
Level up your Java code and explore what Spring can do for you.
Mesos+
https://www.baeldung.com/apache-mesos
Apache Mesos is a platform that allows effective resource sharing between such applications.
https://www.oreilly.com/library/view/learn-apache-mesos/9781789137385/
Learn Apache Mesos is the go-to book for anyone eager to master the power of efficient resource management and cluster deployment with Apache Mesos.
Docker+
https://www.youtube.com/watch?v=GFgJkfScVNU
Master Docker in one course; learn about images and containers on Docker Hub, running multiple containers with Docker Compose, automating workflows with Docker Compose Watch, and much more. 🐳
https://www.youtube.com/watch?v=kTp5xUtcalw
Learn how to use Docker and Kubernetes in this complete hand-on course for beginners.
Linux+
https://ryanstutorials.net/linuxtutorial/
Ok, so you want to learn how to use the Bash command line interface (terminal) on Unix/Linux.
https://ubuntu.com/tutorials/command-line-for-beginners
The Linux command line is a text interface to your computer.
https://www.youtube.com/watch?v=6WatcfENsOU
In this Linux crash course, you will learn the fundamental skills and tools you need to become a proficient Linux system administrator.
https://www.youtube.com/watch?v=v392lEyM29A
Never fear the command line again, make it fear you.
https://www.youtube.com/watch?v=ZtqBQ68cfJc
Bash+
[英文] The Bash Guide
https://guide.bash.academy/
A quality-driven guide through the shell's many features.
https://www.youtube.com/watch?v=tK9Oc6AEnR4
Understanding how to use bash scripting will enhance your productivity by automating tasks, streamlining processes, and making your workflow more efficient.
数据仓库+
https://www.youtube.com/watch?v=9GVqKuTVANE
From Zero to Data Warehouse Hero: A Full SQL Project Walkthrough and Real Industry Experience!
https://www.youtube.com/watch?v=k4tK2ttdSDg
ETL+
https://www.ibm.com/think/topics/etl
ETL—meaning extract, transform, load—is a data integration process that combines, cleans and organizes data from multiple sources into a single, consistent data set for storage in a data warehouse, data lake or other target system.
https://www.youtube.com/watch?v=OW5OgsLpDCQ
It explains what ETL is and what it can do for you to improve your data analysis and productivity.
相关职位
社招5年以上技术
关于我们： 滴滴国际化Fintech业务，是滴滴国际化战略的重要组成板块。近年来，滴滴Fintech在拉美地区积极探索和开展电子支付、信贷、信用卡、商户收单等业务，为当地用户带来更便捷、优质、更高性价比的金融服务。我们诚挚邀请真诚、可靠、勇于挑战的您和我们一起，携手并肩，拥抱金融出海的浪潮，和滴滴Fintech一起快速成长。 职位描述： 1、参与并完成风控平台基建研发，包括决策引擎、特征平台、核身、模型、名单、图数据库、监控平台、Databus等多个方向 2、建设提效工具，提升风控研发流程的效率。 3、积极跟其他团队沟通和配合，推动项目进展，讨论并提出有建设性的意见。
更新于 2025-09-30
社招5年以上技术
滴滴国际化Fintech业务，是滴滴国际化战略的重要组成板块。其支付业务，已经覆盖了全球十多个国家，在中国互联网公司出海中出类拔萃。 自2021年开始，滴滴Fintech在拉美地区大力发展电子支付和信贷业务。短短2年时间，其个人信贷业务，已经在墨西哥的Fintech玩家中位于第一梯队；其电子钱包业务也在巴西的Fintech玩家中也名列前茅，实现快速增长。此外，滴滴Fintech还在拉美地区积极探索和开展信用卡、商户收单等业务，为当地用户带来更便捷、优质、更高性价比的金融服务，实现多个从0到1的突破。 我们诚挚邀请真诚、 可靠、勇于挑战的您和我们一起，携手并肩，拥抱金融出海的浪潮。和滴滴Fintech一起，实现从0到1，从1到100的快速成长 职位描述： 1、参与并完成风控平台基建研发，包括决策引擎、特征平台、核身、模型、名单、图数据库、监控平台、Databus等多个方向 2、建设提效工具，提升风控研发流程的效率。 3、积极跟其他团队沟通和配合，推动项目进展，讨论并提出有建设性的意见。
更新于 2025-03-19
社招5年以上技术
滴滴国际化Fintech业务，是滴滴国际化战略的重要组成板块。其支付业务，已经覆盖了全球十多个国家，在中国互联网公司出海中出类拔萃。 自2021年开始，滴滴Fintech在拉美地区大力发展电子支付和信贷业务。短短2年时间，其个人信贷业务，已经在墨西哥的Fintech玩家中位于第一梯队；其电子钱包业务也在巴西的Fintech玩家中也名列前茅，实现快速增长。此外，滴滴Fintech还在拉美地区积极探索和开展信用卡、商户收单等业务，为当地用户带来更便捷、优质、更高性价比的金融服务，实现多个从0到1的突破。 我们诚挚邀请真诚、 可靠、勇于挑战的您和我们一起，携手并肩，拥抱金融出海的浪潮。和滴滴Fintech一起，实现从0到1，从1到100的快速成长 职位描述： 1、参与并完成风控平台基建研发，包括决策引擎、特征平台、核身、模型、名单、图数据库、监控平台、Databus等多个方向 2、建设提效工具，提升风控研发流程的效率。 3、积极跟其他团队沟通和配合，推动项目进展，讨论并提出有建设性的意见。
更新于 2025-03-19
社招5年以上技术
关于我们： 滴滴国际化Fintech业务，是滴滴国际化战略的重要组成板块。近年来，滴滴Fintech在拉美地区积极探索和开展电子支付、信贷、信用卡、商户收单等业务，为当地用户带来更便捷、优质、更高性价比的金融服务。我们诚挚邀请真诚、可靠、勇于挑战的您和我们一起，携手并肩，拥抱金融出海的浪潮，和滴滴Fintech一起快速成长。 base：北京、上海、杭州 职位描述： 1、助力现金贷、信用卡、钱包、出行外卖业务，帮助各业务场景高效接入集成风控系统，以及升级迭代。积极跟其他团队沟通和配合，推动项目进展，讨论并提出有建设性的意见。 2、参与并完成风控平台基建研发，完善平台功能，包括决策引擎、特征平台、核身、模型、名单、图数据库、监控平台、Databus等多个方向。 3、提升系统的效率、可维护性、稳定性。
更新于 2025-08-08