AI Benchmarking Spec.

hace 1 día

Barcelona, España Amazon A tiempo completo

The Seller AI team within the International Seller Services organization focuses on helping sellers with the right set of Gen‑AI/LLM powered tools and agentic solutions that can enable them to accelerate business growth on Amazon. Our primary focus lies in handling annotations for training, measuring, and improving artificial intelligence (AI) and large language models (LLMs), enabling Amazon to deliver a superior seller experience worldwide. The AI Benchmarking Associate supports the evaluation of AI systems by designing and executing benchmarking and audit activities to assess model quality, compliance, robustness, and fairness. The role combines elements of AI auditing, quality assurance, and traditional audit‑style documentation and stakeholder communication. By joining us, you will play a pivotal role in shaping the future of selling on Amazon for sellers worldwide.

Key Job Responsibilities

- Assist in planning and executing benchmarking exercises for AI models, including defining test plans, metrics, and acceptance criteria across accuracy, robustness, bias, and reliability.
- Support content accuracy, relevancy, and privacy checks by reviewing datasets, model outputs, and data handling practices, escalating potential regulatory risks.
- Validate data based on specific annotation guidelines, ensuring the accuracy and quality of the collected information.
- Prepare clear audit and benchmarking reports, including error ratings, root‑cause analysis, and recommendations, and contribute to presentations for senior stakeholders.
- Maintain organized audit documentation, evidence, and benchmarking datasets to support internal review.
- Work closely with team members and managers to drive process efficiencies and explore opportunities for automation.
- Strive to enhance the productivity and effectiveness of data generation by contributing to the development and continuous improvement of AI audit methodologies, checklists, and test frameworks as regulations and best practices evolve.

Location

We are open to hiring candidates to work out of: Bengaluru, Karnataka, IND.

Basic Qualifications

- Speak, write, and read fluently in Spanish.
- Minimum B2.2 or C1 certified in Spanish Language.
- Bachelor's degree or equivalent with 3+ years of equivalent experience.

Preferred Qualifications

- Masters Degree or C1 certified in Spanish Language.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability or other legally protected status.

#J-18808-Ljbffr

  • AI Evaluation Engineer: Scalable Benchmarking

    hace 2 días

    Barcelona, España Axiomatic_AI A tiempo completo

    A cutting-edge AI firm in Barcelona is seeking an experienced AI Engineer/Scientist specializing in Evaluation and Benchmarking. In this role, you'll design and maintain evaluation pipelines that ensure reliable measurement of AI performance. Essential qualifications include a Master’s degree and strong software engineering skills, with an emphasis on...

  • AI Evaluation Engineer: Scalable Benchmarking

    hace 2 días

    Barcelona, España Axiomatic_AI A tiempo completo

    A cutting-edge AI firm in Barcelona is seeking an experienced AI Engineer/Scientist specializing in Evaluation and Benchmarking. In this role, you'll design and maintain evaluation pipelines that ensure reliable measurement of AI performance. Essential qualifications include a Master’s degree and strong software engineering skills, with an emphasis on...

  • AI Engineer/Scientist

    hace 2 días

    Barcelona, España Axiomatic_AI A tiempo completo

    About Us Axiomatic AI is building a new class of AI systems designed to reason with the rigor of the scientific method. By combining deep learning with formal logic and physics-based modeling, we create verifiable, interpretable AI systems that collaborate with and support human researchers in high-stakes scientific and engineering workflows.Our mission,...

  • AI Engineer

    hace 5 días

    Barcelona, España Axiomatic Ai A tiempo completo

    AI Engineer - Agentic AI for Scientific Workflows About us: Axiomatic_AI is dedicated to accelerating R&D by developing the next generation of Automated Interpretable Reasoning, a verifiably truthful AI model built for reasoning in science and engineering, empowering engineers specifically in hardware design and Electronic Design Automation (EDA), with a...

  • Ai Benchmarking Manager

    hace 1 semana

    Barcelona, Barcelona, España Amazon A tiempo completo

    What you´ll do Drive AI adoption by creating and delivering upskilling opportunities, leveraging internal communities (VivaEngage), SharePoint updates, and targeted email communications to foster engagement and knowledge sharing.Support and grow the AI community within Henkel, encouraging collaboration and best practice exchange across global and local...

  • Lead AI Architect

    hace 4 días

    Barcelona, España ADP A tiempo completo

    ADP is hiring a Lead AI ArchitectDoes the thought of working with one of the most comprehensive employment data sets excite you?Are you empathetic to client needs and inspired by transformation and impacting the lives of millions of people every day?Are you looking to join a dynamic, inclusive team environment with a culture of collaboration and...

  • Lead AI Architect

    hace 2 días

    Barcelona, España ADP A tiempo completo

    ADP is hiring a Lead AI Architect Does the thought of working with one of the most comprehensive employment data sets excite you? Are you empathetic to client needs and inspired by transformation and impacting the lives of millions of people every day? Are you looking to join a dynamic, inclusive team environment with a culture of collaboration and...

  • Lead AI Architect

    hace 4 días

    barcelona, España ADP A tiempo completo

    ADP is hiring a Lead AI Architect Does the thought of working with one of the most comprehensive employment data sets excite you? Are you empathetic to client needs and inspired by transformation and impacting the lives of millions of people every day? Are you looking to join a dynamic, inclusive team environment with a culture of collaboration and...

  • Lead AI Architect

    hace 2 días

    Barcelona, España ADP A tiempo completo

    OverviewADP is hiring a Lead AI Architect. In this role, you will serve as the integration architect and hands-on Lead Architect for our Benchmarking Platform, an ecosystem of master data, job classification, compensation estimation, and additional services (e.g., skills, location search, job search, translation, requisition, etc). This role bridges data...

  • AI Research Scientist

    hace 5 días

    Barcelona, España Axiomatic Ai A tiempo completo

    AI Research Scientist - Formal Methods for Science About us: Axiomatic_AI is dedicated to accelerating R&D by developing the next generation of Automated Interpretable Reasoning, a verifiably truthful AI model built for reasoning in science and engineering, empowering engineers specifically in hardware design and Electronic Design Automation (EDA), with a...