Talent.com
この求人はお住まいの国からは応募できません。
Distributed Training & Inference Optimization Engineer (LLM) - GPU Optimization Department (GPUOD)

Distributed Training & Inference Optimization Engineer (LLM) - GPU Optimization Department (GPUOD)

RakutenTokyo, Japan
16日前
職務内容の詳細

Description

Business Overview

AI & Data Division (AIDD) spearheads data science & AI initiatives by leveraging data from Rakuten Group. We build a platform for large-scale field experimentations using cutting-edge technologies to provide critical insights that enable faster and better and faster contribution for our business. Our division boasts an international culture created by talented employees from around the world. Following the strategic vision “Rakuten as a data-driven membership company”, AIDD is expanding its data & AI related activities across multiple Rakuten Group companies.

Department Overview

GPU Optimization Department is responsible for the strategic management, optimization, and governance of Rakuten's company-wide AI infrastructure, ensuring high-performance, cost-efficient utilization of compute resources for machine learning workloads. We oversee a large-scale hybrid infrastructure spanning thousands of accelerators, including the latest Hopper and upcoming Blackwell architectures.

As a central enabler for AI innovation, we :

  • Optimize compute resource allocation across on-premises and multi-cloud environments, maximizing efficiency for training and inference workloads
  • Manage hybrid orchestration of diverse accelerator resources, ensuring seamless scalability and cost-effective deployment
  • Develop and enhance frameworks for large-scale distributed training, with special focus on LLMs and generative AI
  • Optimize inference performance through model optimization techniques and system-level acceleration
  • Collaborate with internal teams to deliver scalable, high-availability inference services tailored to business needs
  • Continuously evaluate next-generation hardware solutions, including specialized AI chips optimized for LLM workloads
  • By effectively managing both conventional and specialized compute resources across on-premises and cloud environments, our team ensures Rakuten's AI ecosystem remains at the forefront of performance, reliability, and cost-efficiency.

Position : Why We Hire

  • Work on cutting-edge LLM training & inference optimization at scale.
  • Directly impact Rakuten’s AI infrastructure by improving efficiency and reducing costs.
  • Collaborate with global AI / ML teams on high-impact challenges.
  • Opportunity to research and implement state-of-the-art GPU optimizations.
  • Position Details

    As a  GPU Training & Inference Optimization Engineer , you will focus on maximizing the performance, efficiency, and scalability of LLM training and inference workloads on Rakuten’s GPU clusters. You will deeply optimize training frameworks (e.g., PyTorch, DeepSpeed, FSDP) and inference engines (e.g., vLLM, TensorRT-LLM, Triton, SGLang), ensuring Rakuten’s AI models run at peak efficiency.

    This role requires strong expertise in GPU-accelerated ML frameworks, distributed training, and inference optimization, with a focus on reducing training time, improving GPU utilization, and minimizing inference latency.

    Key Responsibilities

  • Optimize LLM training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, FSDP) to maximize GPU utilization and reduce training time.
  • Profile and optimize distributed training bottlenecks (e.g., NCCL issues, CUDA kernel efficiency, communication overhead).
  • Implement and tune inference optimizations (e.g., quantization, dynamic batching, KV caching) for low-latency, high-throughput LLM serving (vLLM, TensorRT-LLM, Triton, SGLang).
  • Collaborate with infrastructure teams to improve GPU cluster scheduling, resource allocation, and fault tolerance for large-scale training jobs.
  • Develop benchmarking tools to measure and improve training throughput, memory efficiency, and inference latency.
  • Research and apply cutting-edge techniques (e.g., mixture-of-experts, speculative decoding) to optimize LLM performance.
  • Mandatory Qualifications :

  • 3+ years of hands-on experience in GPU-accelerated ML training & inference optimization, preferably for LLMs or large-scale deep learning models.
  • Deep expertise in PyTorch, DeepSpeed, FSDP, or Megatron-LM, with experience in distributed training optimizations.
  • Strong knowledge of LLM inference optimizations (e.g., quantization, pruning, KV caching, continuous batching).
  • Bachelor’s or higher degree in Computer Science, Engineering, or related field.
  • Desired Qualifications :

  • Proficiency in CUDA, Triton kernel, NVIDIA tools (Nsight, NCCL), and performance profiling (e.g., PyTorch Profiler, TensorBoard).
  • Experience with LLM-specific optimizations (e.g., FlashAttention, PagedAttention, LoRA, speculative decoding).
  • Familiarity with Kubernetes (K8s) for GPU workloads (e.g., KubeFlow, Volcano).
  • Contributions to open-source ML frameworks (e.g., PyTorch, DeepSpeed, vLLM).
  • Experience with inference serving frameworks (e.g., vLLM, TensorRT-LLM, Triton, Hugging Face TGI).
  • Languages :

    English (Overall - 3 - Advanced)

    この検索に対してジョブアラートを作成する

    Engineer • Tokyo, Japan

    関連性のある求人
    AI / LLM Prompt Engineer(AI / LLM プロンプトエンジニア)

    AI / LLM Prompt Engineer(AI / LLM プロンプトエンジニア)

    AnyMind GroupTokyo, Japan
    EC領域の独自データから、クライアントにとって価値のあるインサイトの提供と意思決定の支援を実現し、ビジネス加速と市場優位性構築に貢献することをミッションとしたチーム「AI App Studio」の一員として、プロンプトエンジニアを募集しています。プラットフォームを通じた顧客への提供価値向上、社内生産性の向上、革新的なAIアシスタントとソリューションを開発する挑戦的なプロジェクトに携わっていた...もっと見せる最終更新日: 16日前
    Sales Operations Manager

    Sales Operations Manager

    Candela Medical FranceTokyo, Japan
    The Sales Operations Manager is responsible for the development and delivery of sales enablement tools, programs, and processes, to support activities essential to sales force productivity.The Sale...もっと見せる最終更新日: 30日以上前
    Application Management Functional Lead, Global DD&T(Data, Digital & Technology), Japan

    Application Management Functional Lead, Global DD&T(Data, Digital & Technology), Japan

    TakedaTokyo, Japan
    Please note this job requires business level Japanese language command not only in speaking, business writing and reading. Takeda is unwavering in our commitment to patients.With the patient at the ...もっと見せる最終更新日: 15日前
    Delivery Solutions Architect

    Delivery Solutions Architect

    DatabricksTokyo, Japan
    At Databricks we are on a mission to empower our customers to solve the world's toughest data problems by utilising the Intelligence platform. As a Delivery Solutions Architect (DSA), you will play ...もっと見せる最終更新日: 30日以上前
    Solution Architect - Generative AI

    Solution Architect - Generative AI

    NVIDIATokyo, Japan
    NVIDIA’s Worldwide Field Operations (WWFO) team is looking for a Data Science focused Solution Architect with expertise in Machine Learning (ML), Deep Learning (DL) and Data Science platforms.Expos...もっと見せる最終更新日: 30日以上前
    Principal Cloud Architect – HPC / GPU & AI Platform Solutions

    Principal Cloud Architect – HPC / GPU & AI Platform Solutions

    OracleTOKYO, Japan
    As a Principal Cloud Architect, you will be at the forefront of designing and implementing next generation accelerated computing and AI solutions on Oracle Cloud Infrastructure (OCI).You will engag...もっと見せる最終更新日: 4日前
    Digital Experience Specialist, Japan

    Digital Experience Specialist, Japan

    COSTokyo, Japan
    As a Digital Experience Specialist, you contribute to digital profitable growth in your sales market, playing a critical role in securing the end to-end digital experience of our customers that dri...もっと見せる最終更新日: 30日以上前
    Product Sales Simulation & Learning and Value Chain Optimization

    Product Sales Simulation & Learning and Value Chain Optimization

    AVEVATokyo, Japan
    AVEVA is creating software trusted by over 90% of leading industrial companies.Product Sales Simulation & Learning and Value Chain Optimization. Reporting line VP Product Sales Simulation & Learning...もっと見せる最終更新日: 16日前
    MultiChannel Strategy Senior Manager / Director

    MultiChannel Strategy Senior Manager / Director

    Veeva SystemsTokyo, Japan
    In the Japanese pharmaceutical industry, changes such as physician workstyle reforms and MR optimization are reshaping how pharmaceutical companies provide information to healthcare professionals, ...もっと見せる最終更新日: 16日前
    Burberry Architecture Senior Project Manager

    Burberry Architecture Senior Project Manager

    BurberryTokyo, Japan
    At Burberry, we believe creativity opens spaces.Our purpose is to unlock the power of imagination to push boundaries and open new possibilities for our people, our customers and our communities.Thi...もっと見せる最終更新日: 11日前
    Cloud Support Engineer (Contract - Infrastructure / KR)

    Cloud Support Engineer (Contract - Infrastructure / KR)

    EpamTokyo, Japan
    EPAM Systems provides 24 / 7 support services for users of a major Cloud platform.The Support team is distributed across multiple locations. We are hiring Application Support Engineers to extend our m...もっと見せる最終更新日: 30日以上前
    【Internal Only】Junior Full-stack Engineer, QA Automation & AI Section (RMI Quality Assurance Platforms Dep)

    【Internal Only】Junior Full-stack Engineer, QA Automation & AI Section (RMI Quality Assurance Platforms Dep)

    Rakuten MobileTokyo, Japan
    The team is responsible for software development and maintenance based on specific use cases, ensuring end-to-end (E2E) quality assurance validation. Members contribute to both frontend and backend ...もっと見せる最終更新日: 15日前
    Founding Scientist

    Founding Scientist

    ProclinicalTokyo, Japan
    Looking to escape your long commute to work? Unlock your new remote working position with this exciting role!.Proclinical is seeking a visionary and highly skilled Founding Scientist to lead the de...もっと見せる最終更新日: 5日前
    Fund Operation Specialist, Transversal Fund Operations Department, SG Haussmann Japan

    Fund Operation Specialist, Transversal Fund Operations Department, SG Haussmann Japan

    Société Générale AssurancesTokyo, Japan
    Fund Operation Specialist , Transversal Fund Operations Department, SG Haussmann Japan.Asset Management Permanent contract Tokyo, Japan Hybrid Reference 25000CCO Start date 2025 / 07 / 03 Publication d...もっと見せる最終更新日: 30日以上前
    Data Consultant

    Data Consultant

    Amaris ConsultingTokyo, Japan
    Design, develop, and maintain robust data pipelines using Snowflake, dbt, and AWS services (Lambda and Glue).Ensure data is processed efficiently and accurately from various sources to data warehou...もっと見せる最終更新日: 16日前
    Manager, Finance

    Manager, Finance

    BASF Japan Ltd.Tokyo, Japan
    The main areas for the job scope include financial reporting compliance for group reporting and overseeing legal compliance including tax filing, IFRS / J-GAAP closing and internal control as well ...もっと見せる最終更新日: 30日以上前
    Data Analyst

    Data Analyst

    Isearch worldwideTokyo, Japan
    DATA ANALYTICS in a leading, progressive bank.Business analysis to solve strategic issues within the company].Analysis related to marketing and sales efficiency, campaign analysis, customer segment...もっと見せる最終更新日: 30日以上前
    Head of Client Funding, Reporting & Referrals Experience

    Head of Client Funding, Reporting & Referrals Experience

    Saxo BankTokyo, Japan
    Act as the liaison between global and local teams, promoting journey interests and ensuring alignment of local priorities with global capacity. Collaborate with Institutional & Other Local Journey O...もっと見せる最終更新日: 16日前