AI Quality Analyst (Personalization) – Japanese

Location: Remote
Job Type: Contract
Duration: 3 months
Rate: $15/hour
Hours: 4–20 hours per week

About the Role

We are looking for a Japanese-speaking AI Quality Analyst to evaluate personalized AI interactions.

You will create realistic, multi-turn prompts based on your personal experiences and assess how effectively the AI uses relevant information to provide accurate, natural, and helpful responses.

This role combines AI evaluation, prompt design, analytical reasoning, Japanese language expertise, and attention to detail .

What You’ll Do

Create and test multi-turn prompts, typically 1–5 turns, using personal context.
Evaluate whether AI responses appropriately address the user's intent.
Assess the accuracy and relevance of personalization.
Identify incorrect assumptions, unsupported inferences, hallucinations, and forced connections.
Evaluate how naturally personal information is integrated into responses.
Compare two AI responses side-by-side and determine which is more helpful, relevant, natural, and enjoyable.
Write clear rationales explaining your evaluations, referencing specific conversation turns.
Review relevant model debug information and verify that available data sources were appropriately used.
Provide detailed annotations and constructive feedback.
Follow required data hygiene procedures, including deleting evaluation conversations when required.

Requirements

Strong Japanese reading and writing skills.
Excellent analytical and critical-thinking abilities.
Strong attention to detail.
Ability to evaluate nuanced or ambiguous AI responses.
Ability to identify subtle differences in naturalness, relevance, and quality.
Strong written communication skills.
Ability to create creative prompts and realistic conversational scenarios.
Ability to work independently in a remote environment.
Desktop or laptop with a reliable internet connection.

Preferred Experience

Experience in any of the following is a plus:

AI quality evaluation
Data annotation
Content moderation
Linguistic evaluation
Quality assurance
Research or analytical work
Computer Science, Linguistics, Law, Policy, Ethics, Journalism, or related fields

Personal Account Requirement

For authentic personalization testing, you must be willing to use your primary personal Google account rather than a testing account.

The project may require access to relevant personal data sources such as previous AI conversations, Gmail, Google Search, and YouTube activity.

Schedule

Minimum 4 hours per day
Up to 20 hours per week
4 hours of overlap with PST required
Global 24-hour operations environment

Selection Process

Shortlisted candidates receive a Job Interest Form.
Profiles are reviewed for relevant qualifications and availability.
Selected candidates receive an assessment, which must be completed within 24 hours .
Successful candidates proceed to pre-onboarding.

Ideal Candidate

You are a strong fit if you are a fluent Japanese writer with excellent analytical judgment, enjoy testing AI systems, can identify subtle quality issues, and can clearly explain why one AI response is better than another.

AI Quality Analyst (Personalization) - Japanese

Start with your email