英伟达AI Infrastructure Software Engineer — CosmosLab
任职要求
• 5+ years developing software infrastructure for large-scale AI or distributed systems. • Bachelor's degree or higher in Computer Science or a related technical field (or equivalent experience). • Strong debugging and triage skills across the stack — from AI application down to GPU/hardware behavior. • Proven track record building and scaling large-scale distributed systems, ideally distributed training or inference. • Hands-on experience with AI training and/or inference infrastructure — RL/post-training, training frameworks, or inference serving. • Proficiency in Python (plus scripting), and solid software engineering practices: testing, defensive programming, version control, and CI. • Excellent communication and collaboration skills; intellectual curiosity, problem-solving, and willingness. Ways to stand out from the crowd: • Experience bu…
工作职责
• Create and implement the training infrastructure spanning pre-training, SFT, and RL post-training for Physical AI world foundation models. The work involves the framework and a comprehensive control plane across clusters to coordinate workloads efficiently. • Develop and improve the pre-training and SFT pipelines — large-scale data loading, distributed training, and checkpointing — to achieve high throughput and scalability. • Develop and improve the inference and evaluation stack, including the inference engine, inference/generation pipelines (which also support RL rollout), and evaluation pipelines. Use methods like continuous batching and KV-cache management to achieve high throughput and low latency. • Build and improve the effective interaction and data flow among the RL system's roles (policy, rollout, reward, simulation) while investigating system-level optimization opportunities. • Integrate and orchestrate simulation and robotics environments as RL environments — driving the simulation↔rollout↔training loop at scale. • Build and refine the distributed training backend — sharding/parallelism, mixed precision, activation checkpointing, and memory/throughput optimization across many GPUs. • Improve the efficiency, scalability, and resiliency of training and RL workloads — focusing on fault tolerance, fast/elastic restart, and throughput optimization under preemption and hardware failure. • Define meaningful, actionable reliability and efficiency metrics to track and improve system reliability. • Root cause, triage, and resolve failures from the application level down to the framework, GPU, and network/hardware level.
• Design, develop, and improve scalable infrastructure to support the next generation of AI applications, including copilots and agentic tools. • Drive improvements in architecture, performance, and reliability, enabling teams to bring to bear LLMs and advanced agent frameworks at scale. • Collaborate across hardware, software, and research teams, mentoring and supporting peers while encouraging best engineering practices and a culture of technical excellence. • Stay informed of the latest advancements in AI infrastructure and contribute to continuous innovation across the organization.
Joining NVIDIA's DGX Cloud Team means contributing to the infrastructure that powers our innovative AI research. This team focuses on optimizing efficiency and resiliency of AI workloads, as well as developing scalable AI and Data infrastructure tools and services. Our objective is to deliver a stable, scalable environment for AI researchers, providing them with the necessary resources and scale to foster innovation. We are seeking an AI infrastructure software engineer to join our team. You'll be instrumental in designing, building, and maintaining AI infrastructure that enable large-scale AI training and inferencing. The responsibilities include implementing software and systems engineering practices to ensure high efficiency and availability of AI systems.As a senior DGX Cloud AI Infrastructure software engineer at NVIDIA, you will have the opportunity to work on innovative technologies that power the future of AI and data science, and be part of a dynamic and supportive team that values learning and growth. The role provides the autonomy to work on meaningful projects with the support and mentorship needed to succeed, and contributes to a culture of blameless postmortems, iterative improvement, and risk-taking. If you are seeking an exciting and rewarding career that makes a difference, we invite you to apply now! What you’ll be doing: • Develop infrastructure software and tools for large-scale AI, LLM, and GenAI infrastructure. • Develop and optimize tools to improve infrastructure efficiency and resiliency. • Root cause and analyze and triage failures from the application level to the hardware level • Enhance infrastructure and products underpinning NVIDIA's AI platforms. • Co-design and implement APIs for integration with NVIDIA's resiliency stacks. • Define meaningful and actionable reliability metrics to track and improve system and service reliability. • Skilled in problem-solving, root cause analysis, and optimization.
• Work with the DGX Cloud Lepton Marketplace team to establish integrations with NVIDIA Cloud Partners, enabling global developers to easily access GPU-optimized virtual machines. • You will craft and implement IaaS API integrations, collaborating with external engineering teams to ensure reliable, scalable, and consistent connectivity across diverse cloud environments. • Shape integration strategies, develop stateful workflow orchestration, and drive improvements in testing, observability, and automation to ensure high-quality, fault-tolerant solutions. • Be responsible for developing the two-sided marketplace, including the integration of compute providers and crafting discovery and bidding experiences to match supply with demand.
• Help design, develop, and improve scalable infrastructure to support the next generation of AI applications, including copilots and agentic tools. • Drive improvements in architecture, performance, and reliability, enabling teams to bring to bear LLMs and advanced agent frameworks at scale. • Stay informed of the latest advancements in AI infrastructure and contribute to continuous innovation.