Back to list

Vacancy description

Research Associate - AI Evaluations in Bangalore, भारत

Karya

Bangalore, भारत

About Karya :

Why was Karya on the  cover of the Time Magazine ,  highlighted by Satya Nadella , and  chosen by Google as its partner for Project Vaani ?

In part, because Karya is on a mission to provide AI enabled earning and learning opportunities to economically underserved communities, thereby building a pathway out of poverty for them. Karya achieves this while also delivering high quality, timely, and price competitive data to its clients.

Karya's workers make at least 20 times the Indian minimum wage and through our one-of-a-kind digital work platform, we have delivered over 40 million digital tasks and are poised to positively impact over 100 thousand workers by the end of the year. In the coming years, our goal is to rapidly scale our impact by bringing economic opportunities to millions of underserved users in India.

About The Role

At Karya, we’re building AI evaluation systems that measure and understand whether AI systems actually work for the communities they’re meant to serve. We’re looking for a Research Associate to help design and run community-facing research that shapes how AI systems are evaluated. You’ll contribute to research workstreams end to end: shaping the research design, building toolkits, running fieldwork, and turning what you learn into evaluation frameworks and datasets. You’ll contribute to a range of evaluation projects across the team. These could include building a gender-bias benchmark for AI chatbots in domains like agriculture; conducting grounded research into what people in India actually use AI for, in which languages and in what contexts, to build evaluation categories from real use; or contributing to conceptual and methods work on what “community” and “participatory” should mean in AI evaluation, pressure-tested against fieldwork with Karya’s worker community. 

The role suits someone with roots in social science who has worked where technology, impact and AI meet, and who cares about gender equity and doing research with communities.

Concretely, this means:
  • Contribute to research design, including research questions, methods, site selection and sampling, working closely with project leads
  • Build and adapt research toolkits: discussion guides, oral consent scripts in local languages, training materials and feedback tools
  • Plan and conduct focus group discussions, interviews and pilots with communities, including participant recruitment and field logistics
  • Coordinate with field teams during data collection, keeping research consent-first and respectful of participants’ time and context
  • Clean, cull and analyse qualitative and quantitative data through a gender and intersectional lens
  • Synthesise insights into findings that strengthen evaluation frameworks, datasets and toolkits
  • Contribute to desk research, reports and external outputs
  • Support project management by tracking timelines, deliverables and decisions across internal teams (Ops, Tech) and external partners

Why This Role

You’ll be embedded with the Evaluations team within Karya’s tech team, where social science and engineering work as one practice. We believe you can’t judge whether an AI system works for people without understanding both the people and the system, and how social and technical structures shape each other. So researchers and engineers on our team work closely together, each bringing what the other can’t. Your research will shape what gets built: community insights become evaluation categories, fieldwork defines what a good answer looks like, and research design decides whose voices go into a dataset. Working alongside engineers, you’ll learn how AI systems are built, tested and deployed, and how they reach the communities we work with. If you’re a social scientist who wants to understand AI from the inside, this is a good place to do it. 

What We Are Looking For

Qualifications

  • At least 2–3 years of research experience, ideally at a tech, impact or AI-focused organisation
  • A degree in a social science discipline (e.g., sociology, development studies, gender studies, anthropology), or equivalent experience
  • Grounding in gender and intersectionality (caste, class, region) through study, research or practice
  • Experience contributing to research design and delivering research projects
  • Hands-on qualitative and field research experience, including running FGDs
  • Working data skills: SQL, quantitative analysis, and some coding (e.g., Python or R)
  • Fluency in spoken and written Hindi
  • Willingness to travel for fieldwork

Skills And Competencies

  • Applied AI/ML literacy: enough to engage meaningfully with model evaluation, human-in-the-loop systems and language technologies
  • Socio-technical systems thinking, meaning you understand technology and society as shaping each other rather than as separate concerns
  • Strong written and verbal communication in Hindi and English
  • Comfort working in ambiguous, early-stage research settings

Good to have

  • Proficiency in Indian languages beyond Hindi
  • Familiarity with agriculture or rural livelihoods
  • Exposure to participatory methods 

People matter at Karya and these are some of the perks and benefits we created for our team:
• Flexible vacation and leave policy
• Flexible work options
• Insurance as per industry standards
• Access to industry stalwarts and networking opportunities

Karya is an equal opportunity and affirmative action employer.

Qualified applicants will receive consideration without regard to their race, colour, religion, sex, sexual orientation, gender identity and disability.

Karya invites all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search to apply for a career opportunity, reach out at  [email protected] .

Similar vacancies

More like this role

  • Full-time

Rs 1 lakh p.a.

... team - Contribute to research design, including research questions, methods, site selection and sampling, working closely with project leads - Build and adapt research toolkits: discussion guides, oral consent scripts in local languages, training materials and feedback tools What you need - Research - SQL - Python - R - Machine ...

Bangalore, भारत

  • Hybrid
  • Full-time
... automated and hybrid evaluation based on reliability, cost and validity trade-offs - Trace failures in multi-stage AI pipelines to the responsible component - Research: prototype new evaluation methods, and turn results into publications, technical reports and public releases What you need - Research - Machine learning - Python ...

Bangalore, भारत

  • Full-time
... and rural development - Translate academic and technical evidence into clear decision ready outputs for internal and external stakeholders What you need - Research - Excel - Market risk Job details - Location: Delhi - Experience: 3–6 years - Education: Any graduate - Employment type: Full-time - Languages: English [{"name": ...

Delhi, भारत

Cadila Pharmaceuticals

Research Associate

  • Full-time
... Perform the forced degradation studies - Develop and validate the cleaning method - Perform partial method validation for new develop method What you need - Research - SOP writing Job details - Location: Dholka - Education: Any graduate - Employment type: Full-time - Languages: English [{"name": "full_time", "text": "Full ...

Dholka, भारत

  • Hybrid
  • Full-time
... Research Associate, you'll play a key role in supporting Consumer & Market Research teams across different global regions. Working within NIQ's Analytics & Research Center (ARC), you'll support innovation and internal research projects by gathering, validating, formatting, and maintaining research data used in reports, ...

Vadodara, भारत

  • Full-time
  • Data Science
  • Research
... model training and post-training approaches for translating complex domain knowledge into structured representations and enabling precise - Drive hands-on research in reinforcement learning, synthetic data generation, reasoning, and agentic AI, exploring new techniques to advance model capabilities - Own and lead end-to-end ...

Bangalore, भारत

Louis Dreyfus Company

Associate Research Analyst

... and enhance research methodologies for crop estimation and strategy, ensuring continuous review based on observations from crop tours - Regularly update and maintain databases and time-series data, contributing to the development and refinement of research methodologies and strategies - Collect and evaluate primary and ...

Bangalore, भारत

HSBC Global Services Limited

Senior Associate - Research

... helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to join our team in the role of Senior Associate - Research In this role, you will: - Timely and accurate completion of client interaction submissions. - Quality and reliability of Market insight reports ...

Bangalore, भारत

  • Full-time
... for Process improvement, Process development, Process yield factor, Effluent reduction etc - Ensure online documentation like LNB writing etc What you need - Research - Chemistry - Process improvement Job details - Location: Ahmedabad - Experience: 1–2 years - Education: https://jobeax.com/link/MPrGoo1PSJpZ7d0e or PhD - Employment ...

Ahmedabad, भारत

Aditya Birla Group

Research Associate I

  • Full-time
... technology center as a part of a collaborative team - Perform laboratory testing following all safety rules and with a good safety attitude What you need - Research - Reporting - Technical writing - Instrumentation - Lab testing Good to have - Chemistry Job details - Location: Taloje - Experience: 3–5 years - Education: ...

भारत

  • Data Science
  • Research
AI Research Scientist : We are seeking a highly skilled and innovative Research Scientist specializing in Artificial Intelligence to join our R&D team. This role is ideal for someone with deep expertise in implementing and customizing state-of-the-art AI research papers, optimizing large-scale models, and building robust ...

Bangalore, भारत