Every engineer uses AI now. Hire the ones who use it well.

Evaluator scores how well a candidate actually works with AI: reading its output, fixing it, prompting it, and overriding it when it is wrong. Scored next to the fundamentals that still decide whether someone ships.

10 free every monthNo card requiredSee what is tested
AI critiqueQuestion 14 of 17
20 points

An AI assistant produced this. It looks reasonable. It is not. Find every flaw and fix it.

async function fetchUserPosts(userId: string) {
  const res = await fetch(`/api/users/${userId}/posts`)
  const posts = res.json.parse()
  return posts.filter((p, i) => i <= posts.length)
}

What the candidate found

  • res.json.parse() is invented. The real call is await res.json().
  • i <= posts.length is off by one, and the filter does nothing useful anyway.
Critique score92/ 100

Five fundamentals, plus the one most tests skip.

Every assessment is generated for the role you are hiring for, in the stack you use. The questions change. The dimensions do not.

AI collaboration

How it is scored

Five sub-tests covering prompt quality, reading AI code, fixing it, critique, and live collaboration. The dimension a conventional coding test does not look at.

Prompt the candidate sent

Implement a debounced search hook for the Postgres-backed /api/search endpoint we already use in SearchBar.tsx. 300ms debounce. Cancel in-flight requests with the AbortController we use elsewhere. Return { data, error, loading }. Do not add a fetch library. Empty query returns early with no request.

What we scored it on

  • Context4 / 4
  • Constraints4 / 4
  • Edge cases4 / 4
  • Acceptance criteria0 / 4
  • Names the existing pattern4 / 4
  • Prompt quality16 / 20

Lost four points for never stating how the reviewer would know the hook was finished.

Code reading

How it is scored

Untangle real code and say what it actually does. Spot the subtle bug, reason about the architecture around it.

if (user?.perms?.includes('admin')
    || user?.role === 'admin') {
  return grant(user)   // which wins?
}

Code writing

How it is scored

Implement to spec. Complete partial code. Build features that meet the spec they were given.

Spec compliance7 / 8 assertions

Debugging

How it is scored

Find the bug in messy legacy code. Bad naming, deep nesting, hidden state, no tests to lean on.

- if (idx = list.length) return
+ if (idx === list.length) return
  one character, four hours

Communication

How it is scored

Write for the next human, not the compiler. Pull request descriptions. Explaining a refactor to a product manager.

“Swapped the N+1 in the digest job for a single join. Cuts the nightly run from 40 minutes to about 90 seconds. No API change.”

Tradeoffs

How it is scored

Justify the choice. Build or buy, SQL or NoSQL, ship it now or do it properly.

Ship the join

Fast now, harder to shard later

Denormalise

Slower to build, scales cleanly

Graded on the argument, not the answer.

You have been screening for the wrong thing.

Hiring in 2023

Did the candidate use ChatGPT? Block them, detect them, ban the tool.

Hiring in 2026

Of course they use AI. The question is whether they can read it, fix it, prompt it, and override it when it hallucinates.

Every shop now has Copilot, Cursor, Claude Code. The bottom quartile of every team takes the AI's first answer. The top quartile catches the hallucinated import, rewrites the over-engineered class, and ships something that works. We test for the top quartile.

Five tests for how someone works with AI.

No other platform does this. Most still treat AI as something to detect. We treat it as something to grade.

  1. 01

    Can they brief an AI like they brief a junior?

    We give them a feature spec. They write the prompt they would actually send. We score for context, constraints, edge cases, and acceptance criteria, not for verbosity.

    A strong candidate response

    Implement a debounced search hook for the Postgres-backed /api/search endpoint we already use in SearchBar.tsx. 300ms debounce. Cancel in-flight requests on new input, using the AbortController we use elsewhere. Return { data, error, loading }. Do not introduce a new fetch library, we use native fetch. Handle the empty query by returning early without a request.

    Gives contextStates constraintsNames the edge case
  2. 02

    Can they tell working from good?

    We show them AI-written code that runs. They explain what it does, flag the tells (an over-engineered class, a defensive try/catch swallowing real errors, a pattern nobody writes by hand), and say what they would change.

    class UserDataManager {
      private cache: Map<string, User | null>
      constructor() {
        this.cache = new Map()
      }
      async getUserById(id: string | null): Promise<User | null> {
        if (!id) return null
        try {
          if (this.cache.has(id)) return this.cache.get(id)!
          return await fetchUser(id)
        } catch (e) { return null }
      }
    }

    Candidate

    A class for what should be a function. It swallows errors silently, so the caller cannot tell a 500 from a missing user. And it never writes to the cache, so the cache never warms.

  3. 03

    Can they fix one bug without touching anything else?

    We plant exactly one realistic bug in an AI-written function. They find it and patch it minimally. A broad refactor that misses the actual problem loses points.

    The candidate's diff
      function paginate(items, page, size) {
    - const start = page * size
    + const start = (page - 1) * size
        return items.slice(start, start + size)
      }
    Correct. One line, and no collateral refactor.
  4. 04

    Can they catch every hallucination?

    We give them code with several planted flaws: invented APIs, off-by-ones, swallowed errors. We grade thoroughness. Did they catch all of them, or stop at the first one and call it good?

    Found by candidate, 3 of 3

    • lodash.deepFlatten does not exist. _.flattenDeep does.
    • catch (e) {} swallows the error. It should log or rethrow.
    • The loop runs in quadratic time. The outer pass should be a Set lookup.
  5. 05

    Watch them work with the assistant.

    On the final question the candidate gets an AI sidebar inside the editor. Every prompt they send, every suggestion they accept, every chunk they reject, and every keystroke on top is recorded. The transcript goes to you.

    function debouncedSearch(query: string) {
      // accepted from the assistant
      if (!query) return
      if (controller) controller.abort()
      // candidate edit: was 200, made it 300
      timeout = setTimeout(...)
    }

    Sidebar transcript

    Candidate: use AbortController for cancellation

    Assistant: <draft>

    Candidate: debounce is wrong, it should be 300ms not 200ms

    4 prompts, 2 accepted, 1 rejected, 38% edited by hand

From a job description to a scored candidate, in one sitting.

  1. 01

    Paste a job description.

    Or describe the role in one sentence. We pick up seniority, stack, and what the person will actually be doing.

  2. 02

    Get an assessment in about 100 seconds.

    A custom test across all six dimensions, calibrated to the role.

  3. 03

    Share a link. Get a scored report.

    Candidates take it async. You get per-question feedback, integrity flags, and for AI questions the full transcript.

We allow AI where it is expected. We catch it where it is not.

On the AI collaboration section the sidebar is right there, because we are scoring how they use it. Everywhere else, behavioural analysis, keystroke pacing, paste patterns, and model fingerprinting flag anyone trying to outsource the fundamentals.

Allowed

On AI questions

The sidebar is visible. Every prompt, accept, and edit is logged for the reviewer.

Flagged

Model fingerprint on a no-AI question

Uniform structure, hedging language, and suspiciously polished prose written under time pressure.

Flagged

Pure paste

A non-trivial answer arrived with zero keystrokes, pasted from somewhere off the page.

Flagged

Burst pattern

Long idle, then a 400 character-per-minute burst, then submit. The alt-tab fingerprint.

Flagged

Tab switches

Five or more focus changes during a single question.

Priced per assessment, not per seat.

Nothing about the scoring is held back for Pro. You upgrade for volume, to drop the Evaluator badge, and for priority support.

Free

$0forever

The same scoring as Pro, at lower volume. Not a trial: it does not expire.

  • 10 assessments every month
  • All six dimensions, including AI collaboration
  • Full integrity reports
  • PDF and CSV export
  • Assessments carry the Evaluator badge
Start free

Pro

Best value

$39per month

For teams hiring regularly.

  • 250 assessments every month
  • Everything in Free
  • Remove the Evaluator badge
  • Priority support
  • $0.50 per extra assessment
Start Pro

Pro annual

$33per month, billed yearly

The same as Pro, about 15 percent cheaper.

  • 3,000 assessments a year
  • Everything in Pro
  • Saves about $69 against monthly
Start annual

Stop hiring engineers who can ace a 2019 coding test.

Start hiring the ones who ship working software in 2026: with AI, around it, and despite it.