Handshake · Seattle
OtherHAI Delivery OpsFullTimeposted 10d ago
Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.
Handshake AI works directly with frontier AI lab researchers to create evaluations, publish benchmarks, and improve AI models through human expertise.
As an AI Model Policy Trainer, Generalist, you will turn complex customer policies into consistent, well-reasoned evaluations of AI model behavior.
You will read user requests, model responses, and relevant conversation history, then determine which policy category best applies. The most interesting cases will not have obvious answers. Two examples may look almost identical until a single word, contextual detail, or difference in intent changes the correct classification.
We are looking for people who enjoy splitting hairs in a healthy way. You form clear opinions, explain precisely why two cases should be treated differently, challenge interpretations respectfully, and change your mind when better evidence emerges. Productive disagreement helps the team find the most accurate and consistent interpretation.
Policies cannot anticipate every possible edge case. You will balance the policy’s text and intent with customer expectations, conversation context, precedent, and team calibration.
The subject matter will vary across projects, from distinguishing benign assistance from meaningful facilitation of harm to evaluating emotional reliance or nuanced sexual-safety boundaries. Success requires learning each customer’s framework on its own terms.
Learn customer policies, definitions, taxonomies, and evaluation rubrics, and adapt as projects and guidance change.
Evaluate user requests and AI model responses within the full relevant conversation context.
Distinguish between closely related labels, severity levels, and policy boundaries to select defensible classifications.
Write concise, evidence-based rationales citing relevant policy language and conversation details.
Identify policy gaps, contradictions, and emerging edge cases, and raise questions when guidance does not resolve a case.
Participate in calibration discussions with evaluators, project leads, policy teams, and researchers; challenge interpretations respectfully and update your judgment when evidence warrants it.
Apply customer policy consistently, maintain accuracy across repeated evaluations, and incorporate feedback into future work.
Help improve evaluation frameworks, examples, decision rules, and quality standards.
You enjoy precise distinctions and notice when one word, contextual detail, or change in intent affects the answer.
You can hold a strong opinion without becoming attached to being right and discuss disagreement respectfully.
You explain judgment calls clearly enough that another person can audit your reasoning.
You ask productive questions when a policy is ambiguous instead of guessing or forcing certainty.
You separate personal views from the customer’s standard while understanding both the letter and purpose of a policy.
You remain careful and consistent during repetitive work and apply feedback quickly.
You learn unfamiliar subjects quickly, work independently, and recognize when you need more context.
You treat sensitive information and difficult subject matter with maturity and sound judgment.
Strong candidates may come from quality assurance, research, editing, law, teaching, operations, trust and safety, content moderation, social science, policy, investigations, compliance, customer support, or other fields that require careful interpretation and defensible decision-making. We care more about how you reason than where you learned to reason.
Professional experience evaluating language models or working in AI evaluation, data annotation, RLHF, trust and safety, or content moderation.
Experience applying detailed rubrics, taxonomies, regulatory language, or quality frameworks.
Familiarity with calibration, quality audits, adjudication, or writing policy guidance and evaluation examples.
Comfort evaluating long conversations, incomplete context, and conflicting evidence, with an interest in AI safety.
This role involves regular and deliberate engagement with sensitive material. Depending on the project, evaluations may include sexual content, emotional distress, self-harm, suicide, violence, weapons, abuse, exploitation, discrimination, and other potentially disturbing subjects.
The work is conducted within structured evaluation frameworks and professional guidelines. Candidates must be able to engage with this material carefully, responsibly, and sustainably while maintaining sound judgment and consistent work quality.
Location: Onsite in Seattle, WA, Monday–Friday.
Compensation: $30-$85/hr. Placement within the range depends on experience.
Employer: TCWGlobal. This is a W-2 assignment supporting Handshake AI.
Employment: Full time, 40 hours per week, non-exempt and eligible for overtime pay.
Schedule: Monday–Friday, 8 a.m.–5 p.m. PT.
Benefits are provided through TCWGlobal. Eligible employees can enroll in medical coverage, including prescription and mental health benefits, dental and vision insurance, healthcare and dependent care flexible spending accounts, and pretax commuter benefits. A 401(k) retirement plan with an employer match is available to eligible participants.
Additional offerings include voluntary accident, critical illness, and term life insurance, wellness and pet-related reimbursements, charitable matching, and employee discounts. Eligibility requirements, waiting periods, employee contributions, and plan terms apply.
Full-time employees scheduled for at least 30 hours per week are eligible for health coverage beginning the first of the month following at least 30 days of employment. Retirement plan eligibility follows a separate schedule.
Paid time off: PTO accrues from the first day of the assignment at one hour per 40 hours worked, with no waiting period to use accrued PTO. PTO may be used for vacation, personal days, or illness, subject to the policy’s approval requirements. Accrual is capped at 80 hours per year and balances at 100 hours, unless otherwise required by law. Additional paid sick leave is provided in accordance with applicable Washington and Seattle requirements.
Paid holidays: The holiday policy lists 13 paid holidays. Eligible employees receive eight hours of holiday pay at their base hourly rate for observed holidays that fall on a scheduled workday, subject to the holiday policy.
See the TCWGlobal 2026 Benefits Guide for coverage options, costs, and enrollment details.
The process includes application review, a recruiter screen, a skills assessment, a hiring manager screen, onsite interviews, and a final round before the offer stage. Your recruiter will share the next steps as you move through the process.
TCWGlobal is an equal opportunity employer. Hiring decisions are based on qualifications and abilities, without discrimination based on characteristics protected by applicable law.
Reasonable accommodations are available during the application and interview process. If you need an accommodation, please let your recruiter know.