
货拉拉高级数据仓库工程师(J19031)
社招全职4年以上地点:深圳状态:招聘
任职要求
任职资格 任职要求: 4年以上数据工程相关工作经验熟悉SQL和Python具有丰富的复杂ETL管道构建经验熟悉Hadoop生态系统,如Hive, Impala, Spark, Flink等。熟悉数据仓库框架和数据模型设计理论。英语流利者优先
工作职责
工作职责: 作为Lalamove的数据工程师,您将加入正在成长的数据仓库团队,设计、构建和维护我们的数据仓库。数据只有在被理解的时候才有用。公司中的其他数据团队,包括BI团队、数据科学团队、优化团队和不同功能的数据分析师,依赖于中心化数据仓库中干净和处理过的数据,以最大限度地提高业务影响。数据仓库团队将是数据仓库设计的所有者,以及处理数据仓库的数据管道的所有者。设计、实现和优化ETL数据管道与上游消费者和下游用户合作,将需求转化和集成到我们的数据模型中理解、记录和维护业务定义、数据字典和数据映射
包括英文材料
SQL+
https://liaoxuefeng.com/books/sql/introduction/index.html
什么是SQL?简单地说,SQL就是访问和处理关系数据库的计算机标准语言。
https://sqlbolt.com/
Learn SQL with simple, interactive exercises.
https://www.youtube.com/watch?v=p3qvj9hO_Bo
In this video we will cover everything you need to know about SQL in only 60 minutes.
Python+
https://liaoxuefeng.com/books/python/introduction/index.html
中文,免费,零起点,完整示例,基于最新的Python 3版本。
https://www.learnpython.org/
a free interactive Python tutorial for people who want to learn Python, fast.
https://www.youtube.com/watch?v=K5KVEU3aaeQ
Master Python from scratch 🚀 No fluff—just clear, practical coding skills to kickstart your journey!
https://www.youtube.com/watch?v=rfscVS0vtbw
This course will give you a full introduction into all of the core concepts in python.
ETL+
https://www.ibm.com/think/topics/etl
ETL—meaning extract, transform, load—is a data integration process that combines, cleans and organizes data from multiple sources into a single, consistent data set for storage in a data warehouse, data lake or other target system.
https://www.youtube.com/watch?v=OW5OgsLpDCQ
It explains what ETL is and what it can do for you to improve your data analysis and productivity.
Hadoop+
https://www.runoob.com/w3cnote/hadoop-tutorial.html
Hadoop 为庞大的计算机集群提供可靠的、可伸缩的应用层计算和存储支持,它允许使用简单的编程模型跨计算机群集分布式处理大型数据集,并且支持在单台计算机到几千台计算机之间进行扩展。
[英文] Hadoop Tutorial
https://www.tutorialspoint.com/hadoop/index.htm
Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models.
Hive+
[英文] Hive Tutorial
https://www.tutorialspoint.com/hive/index.htm
Hive is a data warehouse infrastructure tool to process structured data in Hadoop. It resides on top of Hadoop to summarize Big Data, and makes querying and analyzing easy.
https://www.youtube.com/watch?v=D4HqQ8-Ja9Y
Impala+
[英文] Impala Tutorials
https://impala.apache.org/docs/build/html/topics/impala_tutorial.html
This section includes tutorial scenarios that demonstrate how to begin using Impala.
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
数据仓库+
https://www.youtube.com/watch?v=9GVqKuTVANE
From Zero to Data Warehouse Hero: A Full SQL Project Walkthrough and Real Industry Experience!
https://www.youtube.com/watch?v=k4tK2ttdSDg
相关职位
社招J34TP
1、负责字节跳动中台业务的数据仓库架构设计、建模和ETL开发; 2、参与数据治理工作,提升数据易用性及数据质量,与数据工具团队紧密合作; 3、理解并合理抽象业务需求,发挥数据价值,与业务团队紧密合作。
更新于 2021-06-02
社招5年以上A01882
1、主导财经业务相关主题的分布式数据仓库规划、设计、落地及运营; 2、主导财经数据资产公共层建设,从工具和效果上实现敏捷智能的目标; 3、深入了解业务,主动优化数据仓库实现数据治理与迭代闭环,不断提升数据质效。
更新于 2023-09-15
社招住宿业务AI &
1、负责离线和在线数据的采集、清洗和加载; 2、负责通过专项分析,输出专项分析报告,为业务决策和监控提供数据支持; 3、负责携程大量商户/用户数据的分析和提炼。
更新于 2025-03-31
社招3年以上技术
1. 深入理解滴滴海外业务模式、流程和系统架构,和相关产品技术、业务运营高效沟通,设计合理的数据仓库架构。 2. 充分利用滴滴现有的各种数据仓库及反作弊处理平台,设计、开发和维护高效、可扩展的大数据处理系统,以支持安全反作弊业务的数据驱动决策和业务发展。设计并优化离线/实时数仓模型(如 ODS、DWD、DWS、ADS 分层),支持实时报表、监控和风控等场景。 3. 对多源异构数据(日志、事件消息、API、爬虫数据等)进行清洗、解析和结构化处理,提取关键业务信息。解决数据清洗中的 脏数据、格式混乱、缺失值、重复数据 等问题,提升数据质量。与数据开发、算法、业务团队协作,理解需求并构建高效的数据处理流程。 4. 跟踪大数据领域的新技术、新工具,不断探索并引入以提升团队的技术能力和项目效率。优化流式计算任务的 资源利用率(如 Flink 任务调优、Kafka 分区策略调整)。 5. 国际化安全反作弊技术团队其他开发工作。
更新于 2025-06-16