小鹏汽车大数据开发工程师-数仓方向-广州
社招全职自动驾驶板块地点：广州状态：招聘
任职要求
职位要求 - 计算机相关专业本科及以上学历，能写高质量SQL代码，至少熟练掌握Python，Java，scala等语言中的一种； - 熟悉Hadoop生态相关技术，如Hive、HBase、Spark、Flink、Elasticsearch、Trino、Doris等，有基于分布式数据存储与计算平台应用开发经验； - 熟悉数据仓库模型设计，熟悉数据治理流程与方法，有实际模型设计及ETL开发经验，熟悉Kimball维度建模方法论，有自己的数据治理方法论； - 熟悉开放表格式Delta lake/Iceberg/Hudi/Paimon其中任意一种； - 熟悉至少一种主流湖仓一体平台(如dataworks等)的使用与优化，包括平台的部署，配置，监控和调优等操作； - 熟悉报表工具(如PowerBI/Superset等)的使用，能够独立设计和开发交互式的数据产品； - 有良好的汽车业务Sense，对数据业务场景非常敏感，能够横向协同，跨界整合资源，有效结合业务和技术创新，形成完整的数据解决方案，全局地规划或完善数据服务体系以解决业务/产品的问题； - 掌握常用的数据分析工具、数据挖掘、机器学习算法是加分项； - 具有良好的逻辑思维能力、分析能力、表达呈现能力，主动性强，能与他人合作，共同完成目标； - 有自动驾驶或大型互联网公司相关从业经验，具有大型跨部门的复杂数据项目实施经验优先。
工作职责
1. 数据指标体系搭建：深挖数据价值，构建和维护车端信号数据仓库体系和数据指标体系，为算法和数据闭环提供PB级共享平台和框架支持；负责核心数据指标体系（包括业务分类、生产状态、功能指标等）的搭建、监控与运营；快速输出并不断沉淀标准化的产品数据体系，让业务的数据化运营更加高效、便捷； 2. 数据治理：梳理上下游的数据资产，制定及推广数据标准（如研发规范、质量规范、保障规范）和治理流程，确保数据准 确性、完整性和一致性。 3. 数据管理：负责元数据管理、数据质量检查、数据分级管理等系统的设计、开发及应用，提升数据易用性、可用性及稳定性； 4. 业务团队数据需求的研发支撑：如日志埋点、车联网数据、内部与外部数据的采集、数据同步、数据清洗与标准化、数据模型设计、离线数据处理、实时数据处理、数据服务化、数据可视化等；
包括英文材料
学历+
SQL+
https://liaoxuefeng.com/books/sql/introduction/index.html
什么是SQL？简单地说，SQL就是访问和处理关系数据库的计算机标准语言。
https://sqlbolt.com/
Learn SQL with simple, interactive exercises.
https://www.youtube.com/watch?v=p3qvj9hO_Bo
In this video we will cover everything you need to know about SQL in only 60 minutes.
Python+
https://liaoxuefeng.com/books/python/introduction/index.html
中文，免费，零起点，完整示例，基于最新的Python 3版本。
https://www.learnpython.org/
a free interactive Python tutorial for people who want to learn Python, fast.
https://www.youtube.com/watch?v=K5KVEU3aaeQ
Master Python from scratch 🚀 No fluff—just clear, practical coding skills to kickstart your journey!
https://www.youtube.com/watch?v=rfscVS0vtbw
This course will give you a full introduction into all of the core concepts in python.
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
Scala+
Hadoop+
https://www.runoob.com/w3cnote/hadoop-tutorial.html
Hadoop 为庞大的计算机集群提供可靠的、可伸缩的应用层计算和存储支持，它允许使用简单的编程模型跨计算机群集分布式处理大型数据集，并且支持在单台计算机到几千台计算机之间进行扩展。
[英文] Hadoop Tutorial
https://www.tutorialspoint.com/hadoop/index.htm
Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models.
Hive+
[英文] Hive Tutorial
https://www.tutorialspoint.com/hive/index.htm
Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.
https://www.youtube.com/watch?v=D4HqQ8-Ja9Y
HBase+
[英文] HBase Tutorial
https://www.tutorialspoint.com/hbase/index.htm
HBase is a data model that is similar to Google's big table designed to provide quick random access to huge amounts of structured data. This tutorial provides an introduction to HBase, the procedures to set up HBase on Hadoop File Systems, and ways to interact with HBase shell.
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
ElasticSearch+
https://www.youtube.com/watch?v=a4HBKEda_F8
Learn about Elasticsearch with this comprehensive course designed for beginners, featuring both theoretical concepts and hands-on applications using Python (though applicable to any programming language). The course is structured in two parts: first covering essential Elasticsearch fundamentals including index management, document storage, text analysis, pipeline creation, search functionality, and advanced features like semantic search and embeddings; followed by a practical section where you'll build a real-world website using Elasticsearch as a search engine, working with the Astronomy Picture of the Day (APOD) dataset to implement features such as data cleaning pipelines, tokenization, pagination, and aggregations.
Doris+
https://doris.apache.org/docs/gettingStarted/what-is-apache-doris
数据仓库+
https://www.youtube.com/watch?v=9GVqKuTVANE
From Zero to Data Warehouse Hero: A Full SQL Project Walkthrough and Real Industry Experience!
https://www.youtube.com/watch?v=k4tK2ttdSDg
数据治理+
https://www.ibm.com/think/topics/data-governance
Data governance is the data management discipline that focuses on the quality, security and availability of an organization’s data.
https://www.youtube.com/watch?v=uPsUjKLHLAg
Building data fabric eliminates the technological complexities of data governance so users can connect to the right data at the right time, regardless of where it resides.
ETL+
https://www.ibm.com/think/topics/etl
ETL—meaning extract, transform, load—is a data integration process that combines, cleans and organizes data from multiple sources into a single, consistent data set for storage in a data warehouse, data lake or other target system.
https://www.youtube.com/watch?v=OW5OgsLpDCQ
It explains what ETL is and what it can do for you to improve your data analysis and productivity.
Delta Lake+
https://delta.io/learn/getting-started/
This guide helps you quickly explore the main features of Delta Lake.
[英文] Delta Lake Tutorials
https://delta.io/learn/tutorials/
Try out the latest tutorials for the open-source Delta Lake project.
[英文] Tutorial: Delta Lake
https://docs.databricks.com/aws/en/delta/tutorial
This tutorial introduces common Delta Lake operations on Databricks.
https://www.youtube.com/watch?v=fkWxiesfrgk
In this Delta Lake course, we will go though all the important concepts of Delta Lake.
Iceberg+
https://iceberg.apache.org/spark-quickstart/
This guide will get you up and running with Apache Iceberg™ using Apache Spark™, including sample code to highlight some powerful features.
https://www.baeldung.com/apache-iceberg-intro
This tutorial will discuss Apache Iceberg, a popular open table format in today’s big data landscape.
https://www.youtube.com/watch?v=TsmhRZElPvM
You’ve probably heard about Apache Iceberg™—after all, it’s been getting a lot of buzz.
Hudi+
[英文] Spark Quick Start
https://hudi.apache.org/docs/quick-start-guide
we will walk through code snippets that allows you to insert, update, delete and query a Hudi table.
https://www.oreilly.com/library/view/apache-hudi-the/9781098173821/
Overcome challenges in building transactional guarantees on rapidly changing data by using Apache Hudi.
https://www.youtube.com/watch?v=pyK18sDYnS0
In this video, I'll introduce you to one of the most popular Data Lake solutions out there, Apache Hudi!
Power BI+
[英文] Power BI Tutorial
https://www.tutorialspoint.com/power_bi/index.htm
Power BI is a Data Visualization and Business Intelligence tool that converts data from different data sources to interactive dashboards and BI reports.
https://www.youtube.com/watch?v=FwjaHCVNBWA
数据分析+
[英文] Data Analyst Roadmap
https://roadmap.sh/data-analyst
Step by step guide to becoming an Data Analyst in 2025
数据挖掘+
https://www.youtube.com/watch?v=-bSkREem8dM
Database vs Data Warehouse vs Data Lake
https://www.youtube.com/watch?v=7rs0i-9nOjo
机器学习+
https://www.youtube.com/watch?v=0oyDqO8PjIg
Learn about machine learning and AI with this comprehensive 11-hour course from @LunarTech_ai.
https://www.youtube.com/watch?v=i_LwzRVP7bg
Learn Machine Learning in a way that is accessible to absolute beginners.
https://www.youtube.com/watch?v=NWONeJKn6kc
Learn the theory and practical application of machine learning concepts in this comprehensive course for beginners.
https://www.youtube.com/watch?v=PcbuKRNtCUc
Learn about all the most important concepts and terms related to machine learning and AI.
算法+
https://roadmap.sh/datastructures-and-algorithms
Step by step guide to learn Data Structures and Algorithms in 2025
https://www.hellointerview.com/learn/code
A visual guide to the most important patterns and approaches for the coding interview.
https://www.w3schools.com/dsa/
自动驾驶+
https://www.youtube.com/watch?v=_q4WUxgwDeg&list=PL05umP7R6ij321zzKXK6XCQXAaaYjQbzr
Lecture: Self-Driving Cars (Prof. Andreas Geiger, University of Tübingen)
https://www.youtube.com/watch?v=NkI9ia2cLhc&list=PLB0Tybl0UNfYoJE7ZwsBQoDIG4YN9ptyY
You will learn to make a self-driving car simulation by implementing every component one by one. I will teach you how to implement the car driving mechanics, how to define the environment, how to simulate some sensors, how to detect collisions and how to make the car control itself using a neural network.
相关职位
社招5年以上62T1
团队介绍：TikTok是一个覆盖150个国家和地区的国际短视频平台，我们希望通过TikTok发现真实、有趣的瞬间，让生活更美好。TikTok 在全球各地设有办公室，全球总部位于洛杉矶和新加坡，办公地点还包括纽约、伦敦、都柏林、巴黎、柏林、迪拜、雅加达、首尔和东京等多个城市。 TikTok研发团队，旨在实现TikTok业务的研发工作，搭建及维护业界领先的产品。加入我们，你能接触到包括用户增长、社交、直播、电商C端、内容创造、内容消费等核心业务场景，支持产品在全球赛道上高速发展；也能接触到包括服务架构、基础技术等方向上的技术挑战，保障业务持续高质量、高效率、且安全地为用户服务；同时还能为不同业务场景提供全面的技术解决方案，优化各项产品指标及用户体验。 在这里， 有大牛带队与大家一同不断探索前沿， 突破想象空间。 在这里，你的每一行代码都将服务亿万用户。在这里，团队专业且纯粹，合作氛围平等且轻松。目前在北京，上海，杭州、广州、深圳分别开放多个岗位机会。 1、构建全面的隐私数据质量挖掘及隐私质量的度量指标体系、沉淀隐私数据质量分析方法与监控框架， 提炼隐私数据产品需求，与相关团队（质量，数据平台引擎，数据平台化开发等） 协作完成隐私风险监控的体系搭建； 2、参与大数据组件技术应用，国际化短视频数据保护的质量建设、改造和技术攻关； 3、工作范围涉及包括但不限于：任务和资源调度系统，分布式高并发系统，据资产管理，实时质量监控，流批一体技术等领域。
更新于 2020-08-06
社招3年以上A210044
团队介绍：TikTok是一个覆盖150个国家和地区的国际短视频平台，我们希望通过TikTok发现真实、有趣的瞬间，让生活更美好。TikTok 在全球各地设有办公室，全球总部位于洛杉矶和新加坡，办公地点还包括纽约、伦敦、都柏林、巴黎、柏林、迪拜、雅加达、首尔和东京等多个城市。 TikTok研发团队，旨在实现TikTok业务的研发工作，搭建及维护业界领先的产品。加入我们，你能接触到包括用户增长、社交、直播、电商C端、内容创造、内容消费等核心业务场景，支持产品在全球赛道上高速发展；也能接触到包括服务架构、基础技术等方向上的技术挑战，保障业务持续高质量、高效率、且安全地为用户服务；同时还能为不同业务场景提供全面的技术解决方案，优化各项产品指标及用户体验。 在这里， 有大牛带队与大家一同不断探索前沿， 突破想象空间。 在这里，你的每一行代码都将服务亿万用户。在这里，团队专业且纯粹，合作氛围平等且轻松。目前在北京，上海，杭州、广州、深圳分别开放多个岗位机会。 1、负责TikTok架构团队的数仓研发与建设，为稳定性、成本、性能、效能、建站等多个场景提供完整、准确、易用的数据支持； 2、负责日常TikTok数据需求支持，包括：数据需求对接、需求研发与交付； 3、积极深入了解业务，逐步实现数据驱动业务的价值输出； 4、负责架构数仓的数据清洗策略制定，保证数据准确性、完整性与易用性； 5、协助完成数仓的各项治理规范与方案的落地，确保低成本、高效率的完成数据生产与交付； 6、协助完成数仓的各项技术探索与调研，完成数仓的可持续技术沉淀与演进。
更新于 2025-05-19
社招2年以上网易游戏（互娱）
职位描述： 1. 负责Flink/Iceberg/Kafka/Pulsar等实时计算核心组件的开发，持续提升功能，性能与稳定性。 2. 基于实时计算组件与平台，为业务提供高质量的流计算服务。并主动分析持续针对下游业务进行稳定性，性能与成本方面的优化工作。 3. 负责一定的运维开发工作，为实时计算组件提供完善的指标/告警体系，容量管理等功能。 4. 参与数据湖相关平台与业务的开发。
更新于 2025-04-15
社招2年以上网易游戏（互娱）
1. 负责Flink/Iceberg/Kafka/Pulsar等实时计算核心组件的开发，持续提升功能，性能与稳定性。 2. 基于实时计算组件与平台，为业务提供高质量的流计算服务。并主动分析持续针对下游业务进行稳定性，性能与成本方面的优化工作。 3. 负责一定的运维开发工作，为实时计算组件提供完善的指标/告警体系，容量管理等功能。 4. 参与数据湖相关平台与业务的开发。
更新于 2025-07-22