Okx
Annotator, AI Ops & Enablement
Apply Now
Date Posted
Department
Al Operations & Enablement
Location
Kuala Lumpur, Malaysia
At OKX, we believe that the future will be reshaped by crypto, and ultimately contribute to every individual's freedom. OKX is a leading crypto exchange, and the developer of OKX Wallet, giving millions access to crypto trading and decentralized crypto applications (dApps). OKX is also a trusted brand by hundreds of large institutions seeking access to crypto markets. We are safe and reliable, backed by our Proof of Reserves. Across our multiple offices globally, we are united by our core principles: We Before Me, Do the Right Thing, and Get Things Done. These shared values drive our culture, shape our processes, and foster a friendly, rewarding, and diverse environment for every OK-er. OKX is part of OKG, a group that brings the value of Blockchain to users around the world, through our leading products OKX, OKX Wallet, OKLink and more.
About the Team
The Annotation team sits within AI Operations & Enablement in Global Business Services (GBS). Its primary responsibility is to annotate and evaluate outputs generated by the bot. The team uses approved Knowledge Base content as an evaluation reference. Operations teams own Ops Readiness and maintain their respective Knowledge Bases, including content used by Annotation and AI Operations & Enablement. Ops Tech owns bot tools, configuration, service health, monitoring, and technical operations.
About the Opportunity
As Specialist, Annotation, you will be a senior individual contributor responsible for complex annotation, quality review, and reviewer calibration. You will help reviewers apply approved standards consistently, investigate recurring AI failures, and produce evidence that supports remediation.
What You Will Be Doing
-
Annotate and evaluate complex or ambiguous AI responses, including sensitive customer journeys, using approved rubrics and reference materials.
-
Conduct assigned peer reviews and quality audits. Identify labeling errors, give evidence-based feedback, and refer unresolved disagreements to the Manager.
-
Support calibration sessions and reviewer onboarding through worked examples, coaching, and documented interpretations. Recommend guideline changes for approval.
-
Demonstrates critical thinking on being able to question whether a response is truly correct rather than just plausible, check claims against source material, and reach well-reasoned conclusions on complex or ambiguous cases.
-
Analyze recurring failures in correctness, completeness, policy alignment, context handling, and escalation quality. Separate observed failures from suspected causes.
-
Prepare reproducible issue reports with conversation context, expected behavior, supporting references, and severity under the approved taxonomy. Support remediation follow-up.
-
Curate assigned reference cases and test datasets, covering material edge cases and preserving version history, labeling rationale, and traceability.
-
Perform launch, regression, and post-remediation evaluations. Compare results against agreed criteria and escalate critical defects promptly to the Manager.
-
Identify gaps or contradictions in approved reference content during evaluation. Document their effect on AI responses and refer findings to the relevant Operations owners for resolution.
-
Identify recurring workflow friction and suggest improvements to review tools, reporting, or sampling. Follow approved data-handling controls and validate any automated evaluation support through human review.
What We Look For In You
-
2+ years of relevant experience in annotation, AI evaluation, quality assurance, content review, or customer operations, with demonstrated handling of complex review cases.
-
Practical experience applying structured review criteria to conversational AI, LLM outputs, or comparable quality workflows; understanding of hallucinations and source-grounded responses.
-
Strong judgment in ambiguous cases, with the ability to explain decisions, identify patterns, and distinguish evidence from assumptions.
-
Continuous improvement mindset: learns from feedback and calibration, and proactively suggests practical ways to improve guidelines, workflows, and tools.
-
Experience supporting peer review, calibration, coaching, or structured issue reporting. Strong written English and attention to detail.
Nice to Haves
-
Experience with RAG evaluation, multilingual reviews, or fintech, crypto, and trading products.
-
Familiarity with annotation platforms, spreadsheets, SQL, or Python for analysis. Mandarin fluency is an advantage.
Perks & Benefits
- Competitive total compensation package
- Various team building programs and company events
- Wellness and meal allowances
- Comprehensive healthcare schemes for employees and dependants
- More that we love to tell you along the process!
Notice:
All official OKX vacancies are published on this website. While roles may appear on selected third-party platforms from time to time, information on other sites may be inaccurate or outdated. If in doubt, please apply directly through our official careers website.
Information collected and processed as part of the recruitment process of any job application you choose to submit is subject to OKX's Candidate Privacy Notice.
Interested in this job?
Apply for this position
Date Posted
Department
Al Operations & Enablement
Salary
N/A