Vacancy description
$49920 USD - $62400 USD / hour
India
AI Session Annotator in Australia, India is listed on Jobeax. Browse 30,000+ vacancies available.
Remote | Independent Contractor | $49,920–$62,400 annualized ($24–$30/hour)
Help improve the next generation of voice-and-camera AI assistants by evaluating how accurately they understand and respond to real-world visual information.
As an AI Session Annotator, you’ll review recorded interactions between users and an AI assistant. You’ll compare the assistant’s responses with what was actually visible in the video, evaluate its performance against a detailed rubric, and provide precise written feedback.
Your comments and evaluations will directly help identify issues such as visual hallucinations, inaccurate interpretations, poor timing, and inadequate safety warnings.
This is a judgment-focused role where accuracy, consistency, and quality of feedback matter more than review speed.
Review recorded sessions where users interact with an AI assistant through voice and camera.
Compare the assistant’s responses with the visual information actually available in the video.
Identify fabricated, hallucinated, or inaccurate visual details.
Evaluate each session for:
Visual interpretation
Accuracy
Relevance
Response timing
Appropriate use of spatial and directional language
Assess whether the assistant provides appropriate and timely warnings when responses could affect a user's health, physical safety, or financial security.
Write detailed comments supporting each score with specific evidence from the session.
Document conversations turn by turn using the provided intake form.
Apply the evaluation rubric consistently without introducing personal scoring criteria.
Native or native-level English proficiency , both spoken and written.
Excellent written English and the ability to communicate observations clearly and precisely.
Strong attention to detail and analytical judgment.
Ability to follow a detailed evaluation rubric consistently.
Ability to distinguish between what is actually visible in a video and what an AI assistant claims to see.
An Android smartphone with a working camera and microphone — mandatory for this project.
Experience in any of the following is a plus:
Data annotation
Linguistic quality assurance
Content moderation
AI model evaluation
Linguistics
Translation or interpreting
Conversational AI evaluation
An ear for regional English usage and the ability to recognize unnatural or non-native phrasing is also valuable.
Start Date: Immediate
High-Volume Period: August 26–31
Compensation: $24–$30/hour
Annualized Equivalent: $49,920–$62,400
Work Arrangement: Fully remote
Engagement: Independent contractor
Schedule: Flexible, based on project requirements
Annualized compensation is based on 2,080 hours per year for comparison purposes only. Actual earnings depend on the number of hours and projects completed.
Some sessions involve scenarios where an incorrect AI response could have meaningful real-world consequences. Examples may include identifying medication, determining whether food has spoiled, navigating a street-crossing situation, or interpreting bank card information.
There is no graphic or violent content , but some sessions involve safety-sensitive situations. You’ll be expected to evaluate whether the AI assistant recognized potential risks and provided an appropriate warning when necessary.
Submit your resume or relevant professional background.
If selected, you may be asked to complete a short calibration exercise using sample sessions.
The calibration focuses on scoring consistency and the quality of your written comments—not speed .
Help improve the accuracy and reliability of advanced multimodal AI systems.
Apply your language and analytical skills to real AI evaluation work.
Work remotely with a flexible schedule.
Contribute directly to identifying and correcting AI visual reasoning failures.
Develop hands-on experience in AI model evaluation and data quality.
You will be engaged as an independent contractor.
Work is fully remote and can be completed on your own schedule.
Projects may be extended, shortened, or concluded early depending on project needs and performance.
Your work will not require access to confidential or proprietary information belonging to any employer, client, or institution.
Payments are made weekly via Stripe or Wise based on services rendered.
H-1B and STEM OPT candidates cannot be supported at this time.
All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.
Similar vacancies
TVS Next
... SageMaker Familiarity with Big Data technologies including: Responsible AI AI Ethics Explainable AI (XAI) AI Governance Experience deploying scalable cloud-native AI services using containerized and microservices architectures. Experience working in Agile and Scrum delivery environments. Domain experience within Manufacturing, ...
India, Chennai
Millennium
... Strong working experience in Linux environments and with AI coding tools such as Cursor, Copilot, and Claude - Experience with workflow orchestration tools such as Airflow or Temporal for running agentic flows and batch jobs - Strong understanding of LLMs, RAG, prompt engineering, conversational AI platforms, and modern AI-powered ...
India, Bengaluru
EarnIn
... https://jobeax.com/link/s1WvHFV9ITSwCkyS, or equivalent) in Computer Science, Software Engineering, or a related field from an Indian university - Genuine curiosity about large language models, AI evaluation, and building reliable Gen AI systems - Familiarity with Python and comfort working with APIs, data pipelines, or scripting - Basic understanding of ...
India, Bengaluru
Aurigo
40k
... databases and retrieval optimization.- Agent frameworks such as LangChain, CrewAI, AutoGen, or equivalent.- Evaluation frameworks and metrics for AI systems.- AI observability, monitoring, and performance tuning.- Cloud platforms (AWS/Azure) and container orchestration (Kubernetes).- Python / .Net and relevant ML/AI libraries ...
India, Bengaluru
Cognizant Softvision
... generation of synthetic data. Design and develop backend services using Python or .NET to support OpenAI-powered solutions (or any other LLM solution) Develop and Maintaining AI Pipelines Work with custom datasets, utilizing techniques like chunking and embeddings, to train and fine-tune models. Integrate Azure cognitive services ...
India, Chennai
Axtria
... consent preference and do not load Google Tag Manager localStorage.data-track-share-click]').data('track-share-click') + ' Share' ); ); isIdInSamplePopulation(sessionID, samplePercentage)) // Load Gainsight Script (function (n, t, a, e) let i = 'aptrinsic'; https://jobeax.com/link/UsGHo7h5XS3GSXyf = a + 'a=' + e; Send custom ...
India, India
Useful BI Corporation
... engineering, AI orchestration frameworks, vector databases, and cloud-based AI deployments. The candidate will work closely with Data Engineering, Product, and AI teams to build scalable AI-driven solutions for enterprise use https://jobeax.com/link/sLvd3ppXDN3k9Qmi Responsibilities :- Design and develop GenAI and Agentic ...
India, Pune
talentmatics
... AI, Generative AI, LLMs and industry trends . Contribute to the development and adoption of AI/Generative AI best practices, standards and frameworks . Drive AI innovation and Agentic AI adoption within the organization. Ensure appropriate application of Responsible AI and AI ethics principles. Required Skills - 8–10 years ...
India, Mumbai
Path Mentors Staffing Solution
... AI / Generative AI Engineer with 6+ years of experience in building and deploying AI-powered applications. The role focuses on developing practical Generative AI solutions using LLMs, RAG, Agentic AI frameworks, and cloud-based AI https://jobeax.com/link/tlplQAnK5B36KMXq Responsibilities :- Develop and deploy AI-powered ...
India, Bengaluru
Accenture
Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. https://jobeax.com/link/6WcBICn0LM558SP0 ...
India, Bengaluru
Jobgether
... product delivery. Strong emphasis on professional growth, knowledge sharing, and expanding technical expertise. Participation in on-call operations as part of maintaining reliable production services. We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the ...
India, India
paytmpayments
... Campaign Manager – AI Operations who will own the end-to-end execution and success of AI-powered voice and Agentic AI campaigns. The role will involve creating AI voice agents, writing and optimizing prompts, designing conversational journeys, launching campaigns, monitoring performance, analyzing AI interactions, and continuously ...
India, Noida
Tredence Inc.
Role description Senior AI Engineer – Generative AI (GenAI) + Cloud (AWS/GCP/Azure) We are seeking a Senior AI Engineer with deep expertise in Generative AI (GenAI) and hands-on experience deploying solutions on cloud platforms (AWS, GCP, or Azure). This role focuses on building scalable, production-ready GenAI systems, ...
India, Bengaluru
CodeChavo
... and maintain CI/CD pipelines using Cloud Build and GitHub Actions, collaborating closely with DevOps engineers and SREs for robust deployment and monitoring.- Maintain comprehensive documentation including API references, runbooks, architecture diagrams, and conduct internal demos and training sessions.- Ensure all AI solutions ...
India, Pune
Infinite Computer Solutions India Pvt. Ltd.
Job Description Contact center platform experience is the core: NICE CXone administration and integration, plus Microsoft-stack build work including Copilot Studio, Dynamics 365 Contact Center, Azure Communication Services, Azure AI (Speech and Foundry), Microsoft Fabric, and Power BI.
India, India
Latent Bridge
... Responsible AI, Guardrails, OCR, Document Intelligence, Knowledge https://jobeax.com/link/aNVbdEPz4r9S0llT Join LatentBridge?Join an innovation-driven engineering team building enterprise-grade AI solutions for global clients using the latest advancements in Generative AI, Agentic AI, and Machine Learning. (ref:hirist.tech)
India, Bhopal
Tidyhire
... business requirements into clear technical designs and execution plans.- Review architecture, code quality, and engineering best practices; ensure scalable and maintainable solutions.- Drive adoption of AI capabilities (e.g., ML models, GenAI, intelligent automation) in engineering solutions where applicable.- Proactively ...
India, Hyderabad
English language Contract work Part time Freelance Remote job
India, Bombay
Remote job Contract work Freelance
India, Chennai
Impetus Technologies India Pvt. Ltd
... teams to gather requirements and convert them into technical specifications for GenAI solutions.- Stay current with emerging trends and advancements in generative AI, LLMs, and related technologies.- Build and integrate APIs and services to enable seamless interaction between GenAI models and existing systems.- Write clean, ...
India, Chennai