How I Keep Agent Decisions Out of the Black Box

The first problem was where to begin. A fresh graduate might have education and personal projects but no work history. An experienced professional might arrive with a resume containing job titles and dates but very little about ownership or impact. Someone else might have years of experience and no resume at all.

Those people should not go through the same questionnaire.

The wizard still has to start somewhere. If a resume or professional page is available, it can read that first. Otherwise it can begin with the person's background and goals. After the first input, the harder questions begin:

• What useful information is already present?
• What is still missing for this person?
• Which gap is worth asking about next?
• When does the profile contain enough detail to stop?

A fixed sequence cannot answer those questions. It repeats things the user has already provided and asks early-career users for evidence they cannot reasonably have. A prompt that says “keep asking until the profile is complete” has the opposite problem: the LLM invents its own definition of complete, then judges whether it has met that definition.

I had already dealt with this kind of black-box decision in AirbaseHQ.

AirbaseHQ uses an LLM to evaluate candidates, but the model does not get to decide what makes a strong candidate. The hiring team defines the criteria, their importance, and the evidence expected at each score. The model applies that standard to the candidate evidence, and application code validates and calculates the result.

The profile wizard uses the same idea in the other direction. It applies explicit criteria before the profile is finished. The missing criterion identifies the evidence to collect, and that evidence determines the next question.

This is how I want agents to make product decisions: against requirements I can inspect, using evidence I can trace, inside limits enforced by code.

The black box is the decision, not the model

An LLM call will always contain internal reasoning I cannot inspect completely. That does not mean the product decision around it also has to be opaque.

Consider this prompt:

Review this candidate and give them a score from 1 to 5.

The model must invent most of the evaluation process. It decides which qualities matter, how much each one matters, what counts as evidence, and how the evidence maps to a score.

The response can be polished and still be unusable as a hiring system. A different model may reward different qualities. A prompt edit can shift the standard without anyone noticing. When the hiring team disagrees with a score, there is no criterion-level decision to inspect.

Adding “be fair and consistent” does not give the system a definition of fairness or consistency.

The same problem appears in an agent prompt:

Ask the user questions until you have enough information.

What is enough? Which missing detail deserves another interruption? When should the agent accept that evidence is unavailable? The prompt gives the model a goal but no decision policy.

AirbaseHQ makes the hiring team own the standard

AirbaseHQ uses rubrics so the company, department, or hiring team can define the evaluation standard before the LLM sees a candidate.

A rubric contains criteria, weights, and anchored score definitions. If a team cares about system design, for example, it can describe the evidence expected for requirements analysis, trade-offs, reliability, and operational thinking. A candidate does not receive a high score simply because their resume contains the phrase “large-scale system.”

The model evaluates one criterion at a time and returns the evidence behind its judgment. Code validates that the criterion belongs to the rubric, checks the score range, and performs the weighted calculation.

This division matters:

The hiring team defines what good means.
The model maps unstructured evidence to that definition.
Code validates and calculates the result.

The model still makes a semantic judgment. It may misunderstand an answer, so the result is not automatically correct. But it cannot quietly replace the hiring team's standard with its own and return only a final number.

AirbaseHQ also keeps “not observed” separate from a low score. If an interview never covered a criterion, assigning a low score would blame the candidate for a gap in the interview. If the topic was covered and the evidence was weak, that is a real observed result. The system needs to tell those situations apart.

That distinction is what allowed the rubric to work in the other direction.

The profile wizard uses missing criteria to choose the next question

For a completed candidate evaluation, the rubric helps answer:

How well does the evidence meet each criterion?

For an incomplete career profile, I ask:

Which required criterion still lacks evidence?

The rubric has different expectations for people at different career stages. For someone with professional experience, useful evidence includes ownership, scope, decisions, constraints, and outcomes. Someone starting a career may demonstrate ability through education, personal projects, or coursework. The agent should not interrogate both people with the same checklist.

The assessor reads the resume, relevant links, and answers collected so far. It returns the areas already supported, the important gaps, and a focused topic for the next question.

Suppose a resume says:

Senior Engineer, Acme — worked on checkout reliability.

The profile already contains the company, title, and general project. It does not show what the person owned or what the work achieved. The assessor can make that gap explicit, allowing the interviewer to ask:

What part of checkout reliability did you personally own, and what changed after your work?

An answer such as this resolves several criteria at once:

I led the retry redesign for card payments. Duplicate charges fell by 30%, and support tickets about failed checkout dropped over the next quarter.

The next assessment should mark ownership and outcome as covered. The agent can spend its remaining question budget on another gap or stop if the profile now has enough evidence.

If the person says no outcome was measured, the agent should record that answer and move on. The criterion is no longer unobserved. Asking the same question in three different ways would not create the missing measurement.

This gives each question a traceable reason:

criterion -> current evidence -> missing detail -> next question

Without that chain, the question is simply whatever the model decided to ask.

How the rubric changes the interview

Without the rubric, the next question is whatever the model thinks might be useful. With the rubric, every round begins by comparing what we know with the same profile requirements.

The decision is:

evidence = resume, professional page, and answers so far

while question budget remains:
    coverage = compare evidence with profile criteria

    if all required criteria have enough evidence:
        stop

    gap = choose the most important missing criterion
    ask a focused question about that gap
    add the answer to the evidence

This gives the wizard a consistent basis for three decisions: what the user has already covered, what is worth asking next, and whether another question is needed. A fresh graduate and an experienced professional can take different paths while both are assessed against criteria appropriate to their situation.

The question budget remains a hard limit in code. It can end the interview even when evidence is still missing, but it cannot turn an incomplete profile into a complete one. The remaining gaps stay visible instead of being hidden behind a confident result.

Runtime criteria and agent evals solve different problems

The assessor resembles an LLM-as-a-judge, but it runs inside the live agent. It judges the result so far, and its judgment affects the next action.

Conventional agent evals usually happen outside the run. A harness executes test cases, captures traces and final results, and applies graders for task success, tool use, trajectory quality, safety, latency, or cost. Teams compare those results across prompts, models, and agent versions. Anthropic describes the structure in its guide to agent evals.

The closest common agent pattern is the evaluator-optimizer loop described in Building Effective Agents. An evaluator critiques an artifact and sends feedback into another iteration. The profile wizard uses the evaluation to collect evidence rather than rewrite the same artifact.

Both layers are necessary. The runtime rubric gives one run an explicit basis for its decisions. An offline eval suite tells me whether those decisions remain reliable across different inputs and repeated trials.

Other agent decisions can use the same structure

This approach is useful when an agent must make a semantic decision that is important to the product and difficult to express as a Boolean condition.

A research agent can have explicit requirements for claim coverage, source quality, recency, and conflicting evidence. Instead of running another broad search, it can identify the unsupported claim and search for evidence relevant to that criterion.

A requirements agent can check a request for goals, constraints, dependencies, and acceptance conditions. If the deployment constraint is missing, the next action is a specific clarification rather than a generic “tell me more.”

An incident agent can evaluate the evidence available for likely causes. A missing log may justify a diagnostic tool call. A log that disproves a cause is negative evidence and should move the investigation elsewhere.

A security or compliance agent can check whether required controls have been observed. “No key-rotation policy was inspected” and “the policy confirms keys never rotate” should lead to different actions.

A support agent can define the evidence required before escalation, then gather only what the next human needs instead of forcing every customer through the same script.

The implementation process is consistent across these examples:

1. Let the domain owner define the criteria and acceptable evidence.
2. Require the model to judge each criterion against evidence it can cite.
3. Preserve missing, negative, and sufficient evidence as different states.
4. Map unresolved criteria to a limited set of permitted actions.
5. Keep budgets, validation, calculations, and escalation rules in code.

I would not use a model assessor where a schema or deterministic check can make the decision. A fixed rule is easier to test and cheaper to run. The rubric pattern earns its complexity when understanding the evidence requires semantic judgment.

The decision maker needs its own evals

Making a decision inspectable does not prove that the decision is right. An assessor can cite a criterion, quote the resume, and recommend a plausible question while misunderstanding the person's experience.

LLM judges also have documented biases. The G-Eval paper found stronger alignment with human ratings on its generation tasks while noting possible preference for model-written text. Other work has measured position bias across judge models.

For the profile wizard, the useful eval cases are incomplete profiles. A human reviewer can label which criteria have enough evidence, which gap matters most, and which follow-up questions would be acceptable. I can then compare the assessor's decision with that reference.

I care about failures that affect the person using the extension: stopping with an important gap, repeating a question, asking for evidence the user already said does not exist, or spending a limited round on a low-value detail. I also need to compare latency, cost, interview length, and variance across repeated trials.

That labelled dataset is still missing from my current implementation. Until I compare the rubric-driven interview with a fixed questionnaire on the same profiles, I cannot claim a measured improvement. I can show that the decision has explicit requirements, evidence, and boundaries. The eval set must show how often the assessor applies them correctly.

AirbaseHQ taught me to keep the definition of a strong candidate outside the LLM. The profile wizard applies the same principle to agent behaviour. The agent does not get an instruction to “do the right thing” and unlimited freedom to interpret it. It gets criteria, evidence, permitted actions, and a code-enforced boundary around the decision it is making.