Skip to content

Next evaluationfor solversof agents

NextEval Solver 1.0

A derivative-free optimizer in which a language-model agent chooses numerical-method campaigns in a shared decision space, with a textbook autopilot as the fixed-policy fallback.

Release: coming soon

What it is

NextEval Solver minimizes a scalar objective on Rn from function values alone: solve(fun, x0, options=None). A language-model agent selects numerical methods, reads evidence, continues or switches methods, and decides when to finish. A trusted environment owns objective access, validation, function-evaluation accounting, the immutable history, and method state.

Shared decision space. The solver has one toolset. The agent and the autopilot act on the same tools with the same parameters: bounded campaigns of numerical methods, single sampling actions, reads that perform no function evaluations, and the agent's own analysis code. Every actual objective call goes through one transaction and one ledger.

Textbook autopilot. The autopilot is a fixed policy over that space and calls no model: trust-region slices of 2n+1 evaluations with textbook defaults (initial radius 1.0, 2n+1 interpolation points), a reset at the initial radius after a floor, a stall, or two method calls that spent nothing, and finalize after six such calls in a row. By default it is the fallback: when the agent session raises, reports an error or a timeout, or stops without finalize while budget remains, the autopilot spends the rest from the current ledger in the same run.

The solver treats fun as an opaque oracle. It does not inspect source code, problem names, known optima, or verifiers, and it does not support bounds, constraints or derivatives. Dimensions and evaluation budgets are configurable; the default budget is 200 function evaluations.

WHO DECIDES SHARED DECISION SPACE TRUSTED ENVIRONMENT Agent session A language model chooses the next tool call, reads the result, and decides when to finish. built-in runner / Codex adapter / Claude Code adapter Textbook autopilot A fixed policy, no model. Completes the run when the agent session fails or stops without finalize. fallback, same tools Reads situation, state, history, diagnostics, model preview: no evaluations Method campaigns trust region, line search, poll sweep, coordinate search, Newton jump, geometry rebuild, global surrogate Single actions one catalog step or explicit points Own analysis the agent's code on a read-only history finalize Validation and budget schema, preconditions, evaluation ceiling Objective fun(x) called only here, one transaction per call Evaluation ledger every call recorded in order, failures included Method state and result returns the best valid recorded point calls calls actions results Solid teal: the agent's calls. Dashed gold: the fallback takes over the remaining budget on the same ledger. WHO DECIDES Agent session A language model chooses each tool call. built-in runner or an adapter Textbook autopilot A fixed policy, no model. fallback, same tools calls SHARED DECISION SPACE Reads state, history, diagnostics: no evaluations Method campaigns trust region, line search, poll sweep, coordinate search, Newton jump, geometry, global surrogate Single actions one catalog step or explicit points Own analysis, finalize the agent's code on a read-only history actions results TRUSTED ENVIRONMENT Validation and budget schema, preconditions, evaluation ceiling Objective fun(x) called only here, one transaction per call Evaluation ledger every call recorded in order Method state and result returns the best valid recorded point
The agent and the autopilot differ only in who decides. Both reach the objective through the same tools, and the environment performs and meters every function evaluation. When the agent session fails or stops without finalize, the autopilot takes over the remaining budget on the same ledger.

Quick start

User Guide

Release: coming soonThe package and its repository are not public yet. The install command and the repository link appear here with the release.

Configure a model provider (the User Guide lists the environment variables), then minimize a function from Python:

import numpy as np
from nexteval import solve

result = solve(
    lambda x: float(x @ x),
    np.array([2.0, -1.0, 0.5]),
    options={"model": "env", "maxfev": 80, "seed": 7, "trace": "off"},
)
print(result.fun, result.nfev, result.status)
print(result.runner_report["fallback"])  # whether the autopilot completed the run

maxfev is the ceiling on actual objective calls, including the initial point and failed calls. A missing model configuration is an error before the objective is evaluated.

The tool surface

One runner owns the whole solve and talks to the environment only through these tools. There is no tool for recording: the environment appends every ledger and trace record itself.

Tools of the shared decision space
GroupToolsWhat they doFunction evaluations
Readsget_situation, get_state, read_history, read_diagnostic, preview_model, read_manual, search_knowledgeBudget, incumbent, progress, model credibility, history and diagnostics; a model fit and its predicted step; tool manuals.None
Memoryread_memory, memory_write, memory_deleteShort notes the agent keeps for the rest of the solve.None
Method campaignsrun_trust_region, run_line_search, run_poll_sweep, run_coordinate_search, newton_jump, run_geometry_rebuild, run_global_surrogateA bounded run of one method inside the environment, reported step by step: trust region with minimum-Frobenius-norm quadratic models, line search, positive-basis polling, coordinate search, Newton steps, geometry repair, a global polynomial surrogate.One per step, capped per call and by the remaining budget
Own analysisrun_analysisThe agent's own Python on a read-only copy of every evaluation; the candidate points it returns are evaluated in order.None for the analysis; one per candidate
Single actionsexecute_actionOne step from the catalog, such as model_step, geometry_repair, direction_step, poll_set or explicit_points, for a hand-chosen move.One point or one batch
Episode controlfinalizeEnds the task deliberately. The result is the best valid recorded point; a method's own stop never ends the task.None

A runner connects a model to the tools for one complete solve. The three runners share one prompt builder, one tool catalog and one session lifecycle.

DefaultNativeToolRunner

Built-in runner

solve and minimize use it, wrapped in FallbackRunner. It talks to a tool-calling model through the provider clients for Anthropic-compatible, OpenAI-compatible and Azure OpenAI endpoints, which need only the standard library.

AdapterCodexTaskRunner

Codex adapter

Runs a whole solve in one Codex app-server thread. Codex keeps the conversation and its context; the solver keeps the objective, ledger, method state and tools. It needs a separately installed Codex CLI.

AdapterClaudeCodeTaskRunner

Claude Code adapter

Runs a whole solve through the Claude Agent SDK with the same tools. It is an optional extra of the package and uses the existing Claude Code authentication unless an endpoint is configured.

A runner passed as minimize(..., runner=...) is used exactly as given. Wrap it in FallbackRunner to have the autopilot complete a run whose session fails or stops without finalize.

Configuration

All options

The defaults of solve() and minimize() are one fixed configuration.

Default configuration of NextEval Solver 1.0
ComponentDefaultMeaning
Toolsetnexteval.toolset/1The shared decision space, the only toolset: the agent and the autopilot act on the same tools with the same parameters.
Prompttask-shared/1Lists the tools and their costs, documents the autopilot as one reference strategy, and prescribes no plan.
RunnerFallbackRunner(NativeToolRunner())The built-in runner with the autopilot as fallback. The agent session ends 90 seconds before the wall-clock limit so the fallback has time left.
Fallback policyTextbook autopilotTrust-region slices of 2n+1 evaluations with the textbook defaults; reported as textbook in the runner report.
Tool surfacetask_tool_surface="full"Every task tool; run_analysis predefines the helpers fit_poly and suggest.
Budgetmaxfev=200Actual objective calls, including the initial point and failed calls; each method call is capped at 40 evaluations by max_evals_per_method_call.
Observation noiseobjective_noise="unknown"What you declare about the objective's observations; it selects the evaluation cache and the statement every agent receives.
TraceOffThe in-memory evaluation ledger is always kept; persistent trace files are written only on request.

API reference

Python API
solve(fun, x0, options=None)
The entry point for an opaque scalar objective, with the default configuration. Reference
minimize(fun, x0, args=(), *, model, toolset, runner, options)
The composition entry point: supply a model client or a runner. Reference
DFOResult
The best valid recorded point, the evaluation count, the termination status and the runner report. Reference
NextEval Bench
How coding agents fare on derivative-free problems against COBYQA is measured separately, in NextEval Bench.