Talent.com
Innodata
LLM Evaluation Specialist – Cultural and Linguistic Alignment - Japanese native speakingInnodata • , Saitama
LLM Evaluation Specialist – Cultural and Linguistic Alignment - Japanese native speaking

LLM Evaluation Specialist – Cultural and Linguistic Alignment - Japanese native speaking

Innodata • , Saitama
30+日前
職務内容の詳細

We are looking for linguistically and culturally aware professionals to support the evaluation and enhancement of multilingual prompt-response datasets for large language models (LLMs). This role involves rubric design, evaluation of translations and model outputs, prompt creation, and red teaming focused on identifying and surfacing cultural nuances and biases in LLM behaviour.

Key Responsibilities:

Rubric Definition & Prompt Evaluation

  • Update rubric definitions with Japanese examples to ensure cultural and linguistic relevance.
  • Identify the need for additional rubrics tailored to specific languages or regional contexts.
  • Review prompts translated from English into the target language and revise where translations appear unnatural or inaccurate.
  • Writing of thoughtful prompts which can test the cultural awareness of LLM models.
  • Rate prompt-response pairs using a standardized evaluation template based on rubrics and provide detailed justifications to base the findings.
  • Document problematic outputs and annotate them with clear explanations of rubric violations or cultural insensitivities.

Required Qualifications:

  • Native proficiency in Japanese and deep familiarity with cultural norms in the corresponding region.
  • Experience in LLM evaluation, content moderation, or linguistic QA preferred.
  • Strong attention to detail with the ability to identify subtle issues in language use, tone, and cultural references.
  • Comfortable working in spreadsheets and evaluation templates.
  • Master’s degree in relevant stream.

Preferred Qualifications:

  • Prior experience with prompt engineering or LLM testing.
  • Familiarity with tools such as Gemini, ChatGPT or similar LLM platforms.
  • Ability to clearly articulate reasoning behind rubric ratings or prompt edits.
この検索に対してジョブアラートを作成する

LLM Evaluation Specialist – Cultural and Linguistic Alignment - Japanese native speaking • , Saitama

類似の仕事

Business Development Manager (Japanese)

David Kennedy RecruitmentTokyo, Tokyo, Japan

David Kennedy Recruitment is working with a global fintech company that provides traders worldwide with access to a wide range of financial instruments, innovative trading tools, and professional s... もっと見せる

Project Manager (Field Operations & Automotive Projects)

DRS Holding AGMiyashiro, 11, JP

DRS Automotive Solutions Japan K.Japan as part of DRS Group’s global operations across Europe, North America, the Middle East, and Asia-Pacific.We are seeking a Project Manager / Event Operations M... もっと見せる

Game localization and creative writing (Simplified Chinese to Japanese)remote freelancer

TransPerfectTokyo, 13, JP

GDPR consultancy, and voice-over recording.With over 1000 well-known global game developers and publishers as our clients, we are looking for Game Localization and creative writing talents with nat... もっと見せる

Fluent Japanese Opportunity | Customer Support Role in Fintech | Hybrid KL

CbtalentsTokyo, Kantō, Japan
Quick Apply

Japanese & English Customer Support Representative.Hybrid (Onsite for day shifts, Work from Home for night shifts).Looking to grow your career in customer service while gaining exposure to the ... もっと見せる

Team Manager - Japan

Acacium GroupTokyo, Japan

Recruitment Team Manager - Life Sciences - Japan.Are you an experienced Life Sciences agency recruiter with leadership experience who is looking for a new challenge? Do you want to work in an entre... もっと見せる

Freelancer | Simplified Chinese into Japanese for MTPE and Proofreading in Technical

TransPerfectTokyo, 13, JP

Simplified Chinese into Japanese for MTPE and Proofreading in Technical.Join TransPerfect’s international technology brand team.We’re growing our long-term freelance pool for an exciting technology... もっと見せる

Technical Recruiter (Remote)

RM Staffing B.V.Saitama, 37, JP

Reboot Monkey is a global datacenter services provider headquartered in Haarlem, Netherlands, operating.We deliver colocation, IP transit, smart hands, remote hands, and managed datacenter services... もっと見せる

Sustainability Specialist

Preferred by NatureTokyo, Tokyo, Japan

Do you have a background in agriculture or forestry and a passion for sustainability certification? Are you fluent in Japanese and English and looking for a role where you can build lasting client ... もっと見せる

Korean & Japanese linguistic projects (Remote)

Sigma GroupTokyo, Tokyo, Japan

AI – Shaping the Future of Artificial Intelligence 🌍.Sigma is a leading global technology company specializing in data collection and annotation for Artificial Intelligence.With over 30 years of e... もっと見せる

AI Research Engineer (MultiModal Reinforcement Learning) 100% Remote Worldwide

Tether Operations LimitedTokyo, 13, JP

Join Tether and Shape the Future of Digital Finance.At Tether, we’re not just building products, we’re pioneering a global financial revolution.Our cutting-edge solutions empower businesses—from ex... もっと見せる

Renewable Energy Roles in Japan / 日本における再生可能エネルギーの役割

Bradman Carbon and Renewable Energy東京都, JP

Seeking candidates with Renewable Energy experience in Japan.Development or construction of Solar, BESS or Wind numerous roles available.Most roles based in Tokyo plus other locations stated.Seekin... もっと見せる

Technical Project Manager Data Center (Remote)

RM Staffing B.V.Saitama, 37, JP

Clients need a single point of contact who actually understands hardware, not just a relationship manager who has to relay every technical question.Projects span hardware deployment, structured cab... もっと見せる

Clinical Research Associate

ICONTokyo, Japan

CRA I - Japan, Tokyo Hybrid: Office/Remote.ICON is a global healthcare intelligence and clinical research organisation united by a mission to bring new medicines and treatments to patients faster.A... もっと見せる

Academic Advisor (Sales Consultant - Japan - Mandarin required)

Crimson EducationTaito City, Tokyo, Japan

Crimson Education is the world’s leading college admissions consulting firm, with over 1,700 Ivy League and 450 Oxbridge offers.We help ambitious students gain admission to the world’s top universi... もっと見せる

Sales Development Specialist Data Center Services (APAC timezone)

RM Staffing B.V.Saitama, 37, JP

Reboot Monkey is a global datacenter services provider headquartered in Haarlem, Netherlands, operating.We deliver colocation, IP transit, smart hands, remote hands, and managed datacenter services... もっと見せる

Key Account Manager – Automotive (Dealers & OEMs)

DRS Holding AGMiyashiro, 11, JP

DRS Automotive Solutions Japan K.Japan as part of its global operations across Europe, North America, the Middle East, and Asia-Pacific.We are seeking a Key Account Manager to manage and develop re... もっと見せる