Job description:
Data Skill Source is hiring a Quality Assurance (QA) Partner Data Evaluation Team Lead to run a structured data-quality workstream for a fast-growing B2B SaaS client in the intake-to-procurement space. This is a North America–based role leading a small, distributed team that evaluates external data partners against a rigorous, multi-tier framework before any data is integrated into production.
Your team's job is to answer one question with discipline: is this partner's data accurate, reliable, and complete enough to drive high-stakes procurement decisions? You'll own that evaluation process end to end—coverage baselining, API reliability testing, accuracy validation against real transaction data, and live agent evaluation—and report go/no-go signals to the client's AI Operations team.
If you're rigorous, comfortable with data and evaluation methodology, and want to own a function rather than execute tickets, this is for you.
What You'll Do
Evaluation Workstream Ownership
- Own the end-to-end partner data evaluation process across a four-tier framework: coverage baseline, API reliability, accuracy validation, and live agent evaluation.
- Maintain clear, documented pass/fail gates and ensure every partner is evaluated consistently against them.
- Produce defensible go/no-go recommendations with documented evidence for the client's AI Operations team.
Team Leadership
- Lead a distributed team of data evaluation specialists (Manila-based), coordinating across time zones with required ET overlap.
- Assign and prioritize evaluation work, review results for rigor, and maintain consistency across contributors.
- Onboard, coach, and level up team members; surface performance issues early.
Hands-On Evaluation
- Run coverage analysis against the client's top-vendor lists, measure API latency/error/schema reliability, and validate benchmark accuracy against structured line items and document-extracted pricing.
- Configure and assess partner data inside live agent workflows; classify outliers and distinguish extraction flaws from data errors.
- Establish ongoing QA cadences (spot-checks, quarterly re-runs) and flag partners that drift below thresholds.
What We're Looking ForCore Traits (Non-Negotiable)
- Rigor: You hold a hard line on pass/fail criteria and document your evidence; "it seems fine" isn't an answer.
- Ownership: You take a defined framework and run it as a reliable, repeatable operation.
- Player-Coach Balance: You can do the analysis yourself and lead a team doing it.
- Strong Communication: Direct, clear written and spoken English; you turn evaluation results into clear recommendations for stakeholders.
Technical Skills
- Experience in data quality, data analysis, QA, or evaluation work involving structured datasets and accuracy validation.
- Comfort with APIs (testing latency, error rates, schema consistency) and with quantitative comparison methods (deviation, outlier analysis, fuzzy matching).
- Strong spreadsheet/data-handling skills; comfortable working with large vendor and transaction datasets.
- Prior experience leading, mentoring, or coordinating a team—formal or informal.
Nice to Have
- Experience evaluating data vendors, benchmarking datasets, or AI/agent outputs.
- Background in B2B SaaS, enterprise software, or procurement/fintech domains.
Note: Formal titles matter less than demonstrated ability. We care far more about what you've built and led than where you've worked.
What We Offer
- Consistent, long-term contract work with a stable, high-growth B2B SaaS client.
- 30+ hours/week, fully remote (Canada or US).
- MacBook Pro provided.
- The opportunity to own a high-visibility data-quality function tied to a company-priority initiative.
About Data Skill Source
Data Skill Source partners with fast-growing SaaS companies to deliver managed services, systems support, and applied automation. We focus on practical, production-ready technology—not demos or experiments that never ship.
How to Apply
Please submit your resume and a short note describing:
- Data quality, evaluation, or QA workstreams you've owned or built.
- Your experience leading or coordinating a team.
- Any hands-on work validating data accuracy, testing APIs, or evaluating vendor/AI outputs.
- Links to portfolios, write-ups, or work samples (if available).
Job Types: Full-time, Contract Work Location: Remote (Canada or US)
Pay: From $30.00 per hour
Benefits:
- Flexible schedule
- Work from home
Work Location: Remote
