
有赞大数据平台开发工程师
社招全职地点:杭州状态:招聘
任职要求
基本要求: 1. Java 能力:精通 Java 核心编程,熟练使用 Spring Boot 等框架,具备系统架构设计和代码评审能力。 2. 分布式计算:熟悉 Hadoop/Spark/Flink 原理,熟悉其运行机制和体系结构。 3. 数据存储:熟练使用 MySQL、Elasticsearch 等存储引擎,能设计高效存储方案。 4. 工程与协作:遵循代码规范,具备测试和文档能力;擅长跨部门协作,推动项目落地。 5. 综合能力:数据敏感,逻辑分析能力强,能协调多方,探索前沿技术解决业务问题。 成长建议: 1. 从设计架构和代码实现的角度深入理解Hadoop大数据生态某一个技术组件; 2. 了解大数据在各个行业的产品和解决方案。
工作职责
岗位职责: 1. 平台全生命周期管理:负责大数据平台的架构设计、核心模块研发与全链路维护。通过系统化监控、故障预警与应急响应机制,保障系统稳定运行。 2. 资源效能优化:深度分析平台资源使用状况,通过性能调优、成本控制与资源动态调度策略,实现集群资源利用率提升。同时推动数据治理体系建设,保障数据质量、安全性及合规性。 工作内容: 1. 平台迭代与稳定性保障:负责数据平台核心模块(如分布式调度系统、元数据资产管理、异构数据集成平台等)的持续迭代 2. AI 技术深度融合:参与算法平台与 AI 基础服务的研发,构建智能化数据处理流水线,提升业务研发效率。 3. 智能化数据治理:利用 NLP、大模型 等 AI 技术实现数据治理自动化,降低人工成本,提升数据价值。
包括英文材料
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
Spring Boot+
https://spring.io/guides/gs/spring-boot
his guide provides a sampling of how Spring Boot helps you accelerate application development.
https://www.youtube.com/watch?v=Nv2DERaMx-4&list=PLzUMQwCOrQTksiYqoumAQxuhPNa3HqasL
The author teaches you how to use Spring Boot from a complete beginner, to building a REST API with a real database, Dockerising it and deploying it to the cloud.
系统设计+
https://roadmap.sh/system-design
Everything you need to know about designing large scale systems.
https://www.youtube.com/watch?v=F2FmTdLtb_4
This complete system design tutorial covers scalability, reliability, data handling, and high-level architecture with clear explanations, real-world examples, and practical strategies.
Hadoop+
https://www.runoob.com/w3cnote/hadoop-tutorial.html
Hadoop 为庞大的计算机集群提供可靠的、可伸缩的应用层计算和存储支持,它允许使用简单的编程模型跨计算机群集分布式处理大型数据集,并且支持在单台计算机到几千台计算机之间进行扩展。
[英文] Hadoop Tutorial
https://www.tutorialspoint.com/hadoop/index.htm
Hadoop is an open-source framework that allows to store and process big data in a distributed environment across clusters of computers using simple programming models.
Spark+
[英文] Learning Spark Book
https://pages.databricks.com/rs/094-YMS-629/images/LearningSpark2.0.pdf
This new edition has been updated to reflect Apache Spark’s evolution through Spark 2.x and Spark 3.0, including its expanded ecosystem of built-in and external data sources, machine learning, and streaming technologies with which Spark is tightly integrated.
Flink+
https://nightlies.apache.org/flink/flink-docs-release-2.0/docs/learn-flink/overview/
This training presents an introduction to Apache Flink that includes just enough to get you started writing scalable streaming ETL, analytics, and event-driven applications, while leaving out a lot of (ultimately important) details.
https://www.youtube.com/watch?v=WajYe9iA2Uk&list=PLa7VYi0yPIH2GTo3vRtX8w9tgNTTyYSux
Today’s businesses are increasingly software-defined, and their business processes are being automated. Whether it’s orders and shipments, or downloads and clicks, business events can always be streamed. Flink can be used to manipulate, process, and react to these streaming events as they occur.
MySQL+
https://juejin.cn/post/7190306988939542585
这是一篇 MySQL 通关一篇过硬核经验学习路线,包括数据库相关知识,SQL语句的使用,数据库约束,设计等。
[英文] MySQL Tutorial
https://www.mysqltutorial.org/
your go-to resource for mastering MySQL in a fast, easy, and enjoyable way.
https://www.youtube.com/watch?v=5OdVJbNCSso
MySQL SQL tutorial for beginners
https://www.youtube.com/watch?v=7S_tz1z_5bA
This beginner-friendly course teaches you SQL from scratch.
ElasticSearch+
https://www.youtube.com/watch?v=a4HBKEda_F8
Learn about Elasticsearch with this comprehensive course designed for beginners, featuring both theoretical concepts and hands-on applications using Python (though applicable to any programming language). The course is structured in two parts: first covering essential Elasticsearch fundamentals including index management, document storage, text analysis, pipeline creation, search functionality, and advanced features like semantic search and embeddings; followed by a practical section where you'll build a real-world website using Elasticsearch as a search engine, working with the Astronomy Picture of the Day (APOD) dataset to implement features such as data cleaning pipelines, tokenization, pagination, and aggregations.
大数据+
https://www.youtube.com/watch?v=bAyrObl7TYE
https://www.youtube.com/watch?v=H4bf_uuMC-g
With all this talk of Big Data, we got Rebecca Tickle to explain just what makes data into Big Data.
相关职位

社招3年以上计算机网络技术类
1. 负责或参与大数据平台的架构设计以及基础组件的调优、改造、升级; 2. 结合客户需求,负责和参与大数据产品的设计与研发; 3. 负责数据平台新技术的研究及应用落地。
更新于 2024-10-16

社招3年以上
1、负责大数据平台开发工作,主要hadoop集群相关,能够解决业务的需求开发。 2、负责大数据平台的日常运维管理,保障数据平台的稳定性。 3、解决大数据平台的故障及性能问题,提升数据存储和计算效率。
更新于 2025-04-10
社招3年以上TEG公共技术
1.负责大数据开发平台的设计实现,为公司各业务提供高效稳定的数据研发平台能力; 2.参与到需求评审、技术方案设计、编码实现、代码CR、功能测试等研发全流程工作; 3.与产品经理、测试运维等相关团队紧密协作,推动平台能力的高效快速落地; 4.持续优化系统架构,沉淀平台级公共服务组件,促进平台研发迭代效率,进一步提升系统性能和稳定性。
更新于 2025-08-28