米哈游通用网页爬虫工程师
社招全职5年以上程序&技术类地点:上海状态:招聘
工作描述
任职要求 计算机科学、软件工程或相关专业,本科及以上学历。 至少5年以上后端/数据工程经验,3年以上分布式爬虫系统或大规模数据采集平台经验。 精通 Python/Go/Java 任一语言及其并发编程模型。 熟悉 Scrapy、Playwright、Selenium 等爬取框架及反爬机制。 具备分布式调度与大数据生态(Kafka、Spark、Airflow、Trino、S3/TOS/HDFS)经验。 理解网页结构化抽取(HTML DOM、XPath、正则、语言检…
登录查看完整工作描述
微信扫码,1秒登录
包括英文材料
学历+
Python+
https://liaoxuefeng.com/books/python/introduction/index.html
中文,免费,零起点,完整示例,基于最新的Python 3版本。
https://www.learnpython.org/
a free interactive Python tutorial for people who want to learn Python, fast.
https://www.youtube.com/watch?v=K5KVEU3aaeQ
Master Python from scratch 🚀 No fluff—just clear, practical coding skills to kickstart your journey!
https://www.youtube.com/watch?v=rfscVS0vtbw
This course will give you a full introduction into all of the core concepts in python.
Go+
https://www.youtube.com/watch?v=8uiZC0l4Ajw
学习Golang的完整教程!从开始到结束不到一个小时,包括如何在Go中构建API的完整演示。没有多余的内容,只有你需要知道的知识。
Java+
https://www.youtube.com/watch?v=eIrMbAQSU34
Master Java – a must-have language for software development, Android apps, and more! ☕️ This beginner-friendly course takes you from basics to real coding skills.
Playwright+
https://learn.microsoft.com/en-us/training/modules/build-with-playwright/
In this module, you'll learn how to use Playwright to test a sample web application.
https://playwright.dev/docs/intro
Playwright Test is an end-to-end test framework for modern web apps.
https://www.youtube.com/watch?v=hN1Zn6kLntk
Selenium+
https://www.youtube.com/watch?v=j7VZsCCnptM
Learn Selenium by building a web scraping bot in Python.
https://www.youtube.com/watch?v=mOAXEQevCAE&list=PLhW3qG5bs-L_s9HdC5zNshE5Ti8jABwlU
大数据+
https://www.youtube.com/watch?v=bAyrObl7TYE
https://www.youtube.com/watch?v=H4bf_uuMC-g
With all this talk of Big Data, we got Rebecca Tickle to explain just what makes data into Big Data.
Kafka+
https://developer.confluent.io/what-is-apache-kafka/
https://www.youtube.com/watch?v=CU44hKLMg7k
https://www.youtube.com/watch?v=j4bqyAMMb7o&list=PLa7VYi0yPIH0KbnJQcMv5N9iW8HkZHztH
In this Apache Kafka fundamentals course, we introduce you to the basic Apache Kafka elements and APIs, as well as the broader Kafka ecosystem.
还有更多 •••
相关职位
实习阿里巴巴2027
1、对大模型相关技术有项目实操及研究经历,安全领域项目优先; 2、具备优秀的研究能力,在 NDSS、IEEE S&P、CCS、Usenix Security 等顶会有工作发表优先; 3、在至少一个安全
更新于 2026-03-17北京|杭州
实习
1、教育背景:2027届硕士,计算机相关专业,具备扎实的专业基础; 2、编程能力:熟练掌握C++/Python开发,熟悉新语言特性,具有良好的代码规范和质量意识;熟练使用针对Linux系统及其应用的分
更新于 2026-06-23广州
校招程序&技术类
1、计算机科学、人工智能、电子工程等相关博士学历; 2、具备大模型(LLM 或多模态)训练经验,熟悉 Transformer 架构与大规模分布式训练框架(Megatron-LM, DeepSpeed,
上海|北京