Data Engineer - Business Data Solution Group, Business Support Development Department (BSD)

Description

Business Overview

Rakuten Group is based in Japan and serves about 100 million customers domestically and 1 billion customers globally. We offer over 70 services in various fields such as e-commerce, payment services, financial services, telecommunications, media, and sports.

Department Overview

The Business Support Development Department is responsible for part of the RMS (ASP service) provided to stores by Rakuten Ichiba. Additionally, we develop and operate various applications that support the business staff who run Rakuten Ichiba. Our mission is to provide high-quality services using cutting-edge technology so that stores can efficiently expand their businesses.

The EC Business Data Analytics & Platform Section builds the data infrastructure that supports the massive transaction data of Rakuten Ichiba. We handle enormous data, including 50,000 stores, 300 million products, and an annual transaction volume exceeding 3 trillion yen, acting as a data hub to support business decision-making and data utilization. We also process Japan's largest traffic data during Rakuten Super SALE and other large events, striving to provide stable systems and continuously improve service quality.

Position : Why We Hire

We are looking for new members to expand our services for both B2B (for stores) and B2C (for consumers) on Rakuten Ichiba. We use Hadoop and Google Cloud Platform for data storage and efficiently store and process data using Python, Spark, and Kafka.

This recruitment is for a data engineer responsible for expanding and optimizing the data pipeline that handles the massive transaction data of Rakuten Ichiba. This position plays a crucial role in providing data that supports important decision-making in Rakuten Ichiba, directly contributing to business growth and service improvement.

We are seeking an experienced lead software engineer. The ideal candidate will have a strong background in big data technology, cloud platforms, and data migration, with the ability to lead a team, manage projects, and provide high-quality solutions that promote operational efficiency and cost reduction.

Position Details

• Building Large-Scale Batch Data Pipelines : Use data processing frameworks and technologies like Spark, Hive, Kafka, Spark-Streaming, Hadoop, and Google Cloud Platform to build large-scale batch data pipelines. Utilize data pipelines to provide important decision-making data for data checks, customer acquisition, and other key business performance indicators in line with Rakuten Ichiba's guidelines.
• Data Pipeline Management : Build and maintain data pipelines for a large-scale e-commerce platform.
• Automation : Automate data ingestion processes to reduce manual data processing.
• Cloud Migration : Migrate on-premises data infrastructure to Google Cloud Platform.
• Application Management : Manage applications on the platform that ingest and process large amounts of data.
• Solving Complex Data Storage and Retrieval Issues : Resolve and clearly explain complex data storage and retrieval issues in big data pipelines.
• Debugging and Maintenance in Production : Debug and maintain applications in production using logging, alerting, and monitoring tools.
• Effective Collaboration : Collaborate effectively with other teams and cross-team members.
• Team Leadership : Lead a team of software engineers to achieve project goals.

Work Environment

Development Environment

Python / Hadoop / Hive / Spark / Kafka / Google Cloud Platform

Mandatory Qualifications :

Desired Qualifications :

Others Information :

Additional information on Location

Osaka or Tokyo

Additional information on English Qualification

TOEIC score 800 or above

engineer #datascientist #researcher #commerce