Vacancy description
$49920 USD - $62400 USD / hour
India
AI Session Annotator in Australia, India is listed on Jobeax. Browse 30,000+ vacancies available.
Remote | Independent Contractor | $49,920–$62,400 annualized ($24–$30/hour)
Help improve the next generation of voice-and-camera AI assistants by evaluating how accurately they understand and respond to real-world visual information.
As an AI Session Annotator, you’ll review recorded interactions between users and an AI assistant. You’ll compare the assistant’s responses with what was actually visible in the video, evaluate its performance against a detailed rubric, and provide precise written feedback.
Your comments and evaluations will directly help identify issues such as visual hallucinations, inaccurate interpretations, poor timing, and inadequate safety warnings.
This is a judgment-focused role where accuracy, consistency, and quality of feedback matter more than review speed.
Review recorded sessions where users interact with an AI assistant through voice and camera.
Compare the assistant’s responses with the visual information actually available in the video.
Identify fabricated, hallucinated, or inaccurate visual details.
Evaluate each session for:
Visual interpretation
Accuracy
Relevance
Response timing
Appropriate use of spatial and directional language
Assess whether the assistant provides appropriate and timely warnings when responses could affect a user's health, physical safety, or financial security.
Write detailed comments supporting each score with specific evidence from the session.
Document conversations turn by turn using the provided intake form.
Apply the evaluation rubric consistently without introducing personal scoring criteria.
Native or native-level English proficiency , both spoken and written.
Excellent written English and the ability to communicate observations clearly and precisely.
Strong attention to detail and analytical judgment.
Ability to follow a detailed evaluation rubric consistently.
Ability to distinguish between what is actually visible in a video and what an AI assistant claims to see.
An Android smartphone with a working camera and microphone — mandatory for this project.
Experience in any of the following is a plus:
Data annotation
Linguistic quality assurance
Content moderation
AI model evaluation
Linguistics
Translation or interpreting
Conversational AI evaluation
An ear for regional English usage and the ability to recognize unnatural or non-native phrasing is also valuable.
Start Date: Immediate
High-Volume Period: August 26–31
Compensation: $24–$30/hour
Annualized Equivalent: $49,920–$62,400
Work Arrangement: Fully remote
Engagement: Independent contractor
Schedule: Flexible, based on project requirements
Annualized compensation is based on 2,080 hours per year for comparison purposes only. Actual earnings depend on the number of hours and projects completed.
Some sessions involve scenarios where an incorrect AI response could have meaningful real-world consequences. Examples may include identifying medication, determining whether food has spoiled, navigating a street-crossing situation, or interpreting bank card information.
There is no graphic or violent content , but some sessions involve safety-sensitive situations. You’ll be expected to evaluate whether the AI assistant recognized potential risks and provided an appropriate warning when necessary.
Submit your resume or relevant professional background.
If selected, you may be asked to complete a short calibration exercise using sample sessions.
The calibration focuses on scoring consistency and the quality of your written comments—not speed .
Help improve the accuracy and reliability of advanced multimodal AI systems.
Apply your language and analytical skills to real AI evaluation work.
Work remotely with a flexible schedule.
Contribute directly to identifying and correcting AI visual reasoning failures.
Develop hands-on experience in AI model evaluation and data quality.
You will be engaged as an independent contractor.
Work is fully remote and can be completed on your own schedule.
Projects may be extended, shortened, or concluded early depending on project needs and performance.
Your work will not require access to confidential or proprietary information belonging to any employer, client, or institution.
Payments are made weekly via Stripe or Wise based on services rendered.
H-1B and STEM OPT candidates cannot be supported at this time.
All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.
Similar vacancies
Recruitment Room
$49920 USD - $62400 USD / hour
AI Session Annotator Remote | Independent Contractor | $49,920–$62,400 annualized ($24–$30/hour) About the Role Help improve the next generation of voice-and-camera AI assistants by evaluating how accurately they understand and respond to real-world visual information. As an AI Session Annotator, you’ll review recorded ...
India
Mindrift - Data annotation
$8 USD / hour
... testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What this opportunity involves Annotation is what helps AI make sense of the world. As an annotator, you may be invited to take part in online projects such as rating AI-generated content, evaluating factual accuracy, ...
India, Delhi
Mindrift - Data annotation
$8 USD / hour
... testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What this opportunity involves Annotation is what helps AI make sense of the world. As an annotator, you may be invited to take part in online projects such as rating AI-generated content, evaluating factual accuracy, ...
India, Hyderabad
Mindrift - Data annotation
$8 USD / hour
... testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What this opportunity involves Annotation is what helps AI make sense of the world. As an annotator, you may be invited to take part in online projects such as rating AI-generated content, evaluating factual accuracy, ...
India, Mumbai
Mindrift - Data annotation
$8 USD / hour
... testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What this opportunity involves Annotation is what helps AI make sense of the world. As an annotator, you may be invited to take part in online projects such as rating AI-generated content, evaluating factual accuracy, ...
India, Delhi
Mindrift - Data annotation
$8 USD / hour
... testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What this opportunity involves Annotation is what helps AI make sense of the world. As an annotator, you may be invited to take part in online projects such as rating AI-generated content, evaluating factual accuracy, ...
India, Mumbai
Mindrift - Data annotation
$8 USD / hour
... testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What this opportunity involves Annotation is what helps AI make sense of the world. As an annotator, you may be invited to take part in online projects such as rating AI-generated content, evaluating factual accuracy, ...
India, Kolkata
$49920 USD - $62400 USD / hour
AI Session Annotator Remote | Independent Contractor | $49,920–$62,400 annualized ($24–$30/hour) About the Role Help improve the next generation of voice-and-camera AI assistants by evaluating how accurately they understand and respond to real-world visual information. As an AI Session Annotator, you’ll review recorded ...
India
Description :AuxoAI is hiring a Senior Applied AI Engineer to design and deploy production-grade AI agents capable of structured reasoning, planning, and https://jobeax.com/link/zur1j9KkFSPiSZ7e role focuses on building intelligent agent systems that combine LLM-based reasoning with classical planning, search algorithms, ...
India, Gurugram
Role Overview : Join a high-impact team building scalable AI systems that power conversational interfaces, speech recognition, and large-scale data transformation https://jobeax.com/link/Hn2b60846aifVrwk You'll Do : - Architect and operate end-to-end AI systems for conversational interfaces and speech recognition.- Drive ...
India, Bengaluru
r3 consultant
... the design and architecture of enterprise AI and Generative AI solutions.- Define end-to-end architecture for scalable AI applications, platforms, and reusable AI services.- Translate business requirements and use cases into scalable Data & AI technical solutions.- Design and implement solutions leveraging LLMs, RAG, AI ...
India, Pune
Axtria India Private Limited
... methodologies to assess faithfulness, relevance, and toxicity. Monitor model drift, performance, and reliability using frameworks such as RAGAS, TruLens, and DeepEval.- AI Safety & Governance : Implement strict guardrails (e.g., NeMo Guardrails, Llama Guard) to protect against prompt injection, data leakage, and other vulnerabilities, ...
India, Noida
Axtria India Private Limited
... methodologies to assess faithfulness, relevance, and toxicity. Monitor model drift, performance, and reliability using frameworks such as RAGAS, TruLens, and DeepEval.- AI Safety & Governance : Implement strict guardrails (e.g., NeMo Guardrails, Llama Guard) to protect against prompt injection, data leakage, and other vulnerabilities, ...
India, Hyderabad
Movate technology
... threats.- Drive implementation of Responsible AI principles, including explainability, transparency, fairness, governance, safety, and human oversight.- Ensure AI solutions comply with applicable healthcare and enterprise standards, including HIPAA, FHIR, and X12/EDI where relevant.- Design containerized AI workloads using ...
India, Chennai
... Python & Agentic AI Expertise : Strong hands-on experience developing Python applications using agentic AI frameworks such as LangChain and LangGraph.- Generative AI & LLM Expertise : Experience integrating and configuring LLMs for enterprise AI use cases, including conversational AI, intelligent automation, and AI agents.- ...
India, Hyderabad
... Python & Agentic AI Expertise : Strong hands-on experience developing Python applications using agentic AI frameworks such as LangChain and LangGraph.- Generative AI & LLM Expertise : Experience integrating and configuring LLMs for enterprise AI use cases, including conversational AI, intelligent automation, and AI agents.- ...
India, Delhi
AI EngineeringLocation : Sector 63, Noida | Employment Type: Full-Time | Work Mode: Work From OfficeTeam context : We are building a new AI engineering team focused on production-grade LLM integrations, AI-powered workflows, intelligent automation, and integration with an existing Java/Spring backend platform. These roles ...
India, Noida
... Python & Agentic AI Expertise : Strong hands-on experience developing Python applications using agentic AI frameworks such as LangChain and LangGraph.- Generative AI & LLM Expertise : Experience integrating and configuring LLMs for enterprise AI use cases, including conversational AI, intelligent automation, and AI agents.- ...
India, Chennai
... that AI-driven outputs align with specific industry requirements and regulatory standards.- Establish rigorous evaluation frameworks and monitoring protocols to maintain model accuracy, mitigate hallucinations, and ensure the ethical deployment of AI systems.- Mentor junior engineers and foster a culture of technical excellence ...
India, Bengaluru