Evaluate AI search responses for realistic failures involving relevance, source quality, freshness, language, completeness, and regional applicability. Develop search queries, test multiple scenarios, document model shortcomings, and write clear instructions and evaluation criteria. The role requires Chinese language expertise, strong research and source-verification skills, critical thinking, attention to detail, problem-solving, clear writing, and deep familiarity with local Australian cultural and regulatory context.
Overview
Are you passionate about languages, writing, and quality evaluation? We are launching an exciting language evaluation project and are looking for Chinese-speaking experts to join our team.
What you will do:
Identify realistic search-related failures where an AI model provides inadequate responses. Create search queries, test them, document what went wrong, and write instructions and evaluation criteria for correct answers.
Common issues: local relevance, source quality, outdated information, language mismatches, incomplete answers, and region-specific inapplicability. Testing multiple queries before finding failures is expected.
Project Details:
- Start Date: ASAP
- Duration: Ongoing Project.
- Commitment: 20 hours per week.
- Job Type: Freelance contract.
- Location: Remote, Australia
- Rate: $40 per hour.
Requirements:
- Research skills — ability to find and verify information from multiple sources; comfort navigating different websites, databases, and documentation
- Critical thinking — Can spot contradictions between sources, identify outdated information, and recognize when something doesn't apply to a specific locale/context
- Attention to detail — Can catch nuances in language, local variations (prices, regulations, requirements), and edge cases
- Problem-solving mindset — Comfortable iterating and testing when things don't work; doesn't get frustrated by trial- and-error
- Domain knowledge — Ideally someone with background in research, quality assurance, compliance, or similar work that requires precision and source verification
- Writing clarity — Can articulate instructions and rubrics clearly so others understand the logic behind their reasoning
Locale knowledge — Deep familiarity with their specific locale (local regulations, business practices, cultural nuances) is valuable for spotting region-specific failures
Similar Jobs
Fitness • Hardware • Healthtech • Sports • Wearables
Own and grow WHOOP’s retail partnerships across Australia and New Zealand. Manage account relationships, forecasts, purchase orders, inventory, assortment, reporting, and joint business plans. Lead retailer reviews, trade marketing programs, merchandising, promotions, digital activations, and associate training. Collaborate cross-functionally to resolve issues, support launches, expand the retail footprint, identify new partners, and drive sales and sell-through.
Top Skills:
Ai Tools
Cloud • Security • Software • Cybersecurity • Automation
Source, screen, and engage passive Account Executive and other sales candidates across the APJ region. Build diverse talent pipelines and benches, map competitive talent markets, partner with recruiters and hiring managers, assess candidate alignment, deliver a strong candidate experience, and maintain accurate recruiting data. Improve sourcing workflows and outreach methods while managing multiple searches in GitLab’s fully remote, asynchronous environment.
Top Skills:
GitlabGreenhouseLinkedInLinkedin Recruiter
Aerospace • Information Technology • Software • Cybersecurity • Design • Defense • Manufacturing
Performs aircraft maintenance engineering duties for Boeing’s MQ-28 Ghost Bat MRO program at RAAF Base Amberley. Requires Certificate IV in Aeroskills (Mechanical), knowledge of aircraft maintenance procedures and regulations, Australian citizenship for NV1 clearance eligibility, a C class driver’s licence, and strong communication skills. Military aircraft, CASA or DASA licensing, EHS knowledge, and maintenance management system experience are desirable. Travel and periods away from home may be required.
Top Skills:
Computer-Based Maintenance Management Systems (Cmms)
What you need to know about the Sydney Tech Scene
From opera to comedy shows, the Sydney Opera House hosts more than 1,600 performances a year, yet its entertainment sector isn't the only one taking center stage. The city's tech sector has earned a reputation as one of the fastest-growing in the region. More specifically, its IT sector stands out as the country's third-largest, growing at twice the rate of overall employment in the past decade as businesses continue to digitize their operations to stay competitive.
