This is a remote, hourly-paid contractor role focused on improving the safety, reliability, and reasoning quality of advanced AI systems. You will review AI-generated content, evaluate step-by-step reasoning, and provide expert feedback to ensure outputs are accurate, logical, and aligned with safety policies.
You will work across Japanese and English, applying high-level linguistic, cultural, and policy judgment. Your evaluations will directly contribute to improving the safety and robustness of large-scale AI models used globally.
This role may involve exposure to sensitive content, including material that is sexual, violent, or psychologically disturbing, as part of structured AI safety evaluation workflows.
Key Responsibilities
Review and evaluate AI-generated responses for accuracy, clarity, reasoning quality, and safety compliance
Identify logical, methodological, or conceptual errors in model outputs
Compare and rate multiple AI responses based on correctness, safety alignment, and policy adherence
Perform fact-checking and validation when required
Conduct red-teaming and adversarial testing to identify vulnerabilities, edge cases, and unsafe behaviors
Document findings with clear, structured reasoning and reproducible decision paths
Apply and localize safety policies consistently across Japanese and English content, including slang, cultural nuance, and coded language
Escalate ambiguous cases using defined safety frameworks and guidelines
Required Qualifications
Bachelor’s degree or higher in Linguistics, Communications, Psychology, Law, Policy, Security Studies, or a related field (or equivalent experience)
Native or near-native Japanese proficiency (reading and writing)
Minimum C1 English proficiency (reading and writing)
Extensive experience in Trust & Safety, content moderation, policy enforcement, risk operations, compliance, or investigations (senior level)
Proven LLM red-teaming or adversarial testing experience (required)
Strong ability to preserve meaning, intent, and severity across languages
Excellent analytical writing skills with structured reasoning and clear justification
Strong attention to detail and consistency in applying policy frameworks
Ability to work independently in a remote contractor environment
Experience handling sensitive content with professionalism and confidentiality
Preferred Qualifications
Experience in AI data annotation, model evaluation, or localization/translation
Familiarity with AI tools such as ChatGPT, Gemini, Perplexity, or similar systems
Experience working with structured safety taxonomies or policy classification systems
Background in editorial QA, investigative work, or analytical review roles
Seniority level
Mid-Senior level
Employment type
Full-time
Job function
Education and Training
Industries
Software Development
Referrals increase your chances of interviewing at YO IT Consulting by 2x