Skip to content

NextEval Solver 1.0

Python API

The library interface of the derivative-free solver: two entry points, the options, the result object, the provider clients and the runners. It is not a model provider API and not a hosted service.

Release: coming soon

Import

Release: coming soonThe package is not public yet; this page describes the interface of the release. The import name is nexteval.

from nexteval import solve, minimize, SolverOptions, DFOResult
from nexteval import AnthropicCompatClient, OpenAICompatClient, AzureOpenAICompatClient, FakeTextModel
from nexteval import NativeToolRunner, CodexTaskRunner, ClaudeCodeTaskRunner, FallbackRunner, AutopilotRunner

Use only this root surface. Installation, provider setup and complete examples are in the User Guide.

solve

solve(fun, x0, options=None) -> DFOResult

The entry point for an opaque scalar objective. It runs one whole session with the default configuration: the built-in runner wrapped in FallbackRunner, on the shared decision space.

fun
A callable that accepts a one-dimensional NumPy array and returns a scalar. The solver minimizes it as an opaque oracle.
x0
A nonempty, finite, one-dimensional starting point.
options
A mapping or a SolverOptions object; see Options. A model configuration is required, and a missing one is an error before the objective is evaluated.

minimize

minimize(
    fun, x0, args=(), *,
    model=None, toolset=None, runner=None, options=None,
) -> DFOResult

The composition entry point. Without runner it runs the same default as solve.

args
Extra positional arguments passed to fun after x.
model
A provider client or specification for the built-in runner. Do not pass it together with an external runner.
toolset
"nexteval.toolset/1", the shared decision space. It is the default and the only toolset.
runner
An object with run(task, gateway) that owns the whole task. It is used exactly as given; wrap it in FallbackRunner for the default wiring. Conflicting selections fail before the objective is evaluated.

Options

SolverOptions is the configuration object accepted by both entry points; a plain mapping with the same names works as well. Unknown names are rejected. The values below are defaults, not limits.

Budget and stopping

Budget and stopping options
OptionDefaultDescription
maxfev200Maximum number of objective-function calls, including the initial point and failed calls.
ftargetNoneStop successfully once the best finite value is at or below this target.
max_evals_per_method_call40Evaluation cap of one numerical method call.
task_max_tool_calls200Maximum tool calls in one session, reads included.
max_read_result_charsNoneMaximum cumulative characters of read results in one session; None resolves to max(200000, 2000 * maxfev). Exhaustion refuses reads, never actions.
wall_time_limit_sNoneCooperative wall-clock limit for one session, in seconds.
model_call_limitNoneStrict limit on model requests of the built-in runner.
model_token_limitNoneStrict token limit when the provider client reports usage.
invalid_eval_limitNoneInvalid objective calls (non-finite values, exceptions) that end the run; by default half of maxfev, at least 10.

Numerical

Numerical options
OptionDefaultDescription
rhobeg"auto"Initial search radius; "auto" is 1.0, the textbook default.
rhoend1e-06Small-radius threshold of the numerical methods. Reaching it stops that method, not the task.
rhoend_restarts0Number of restarts at the initial radius that the environment itself performs after a method reaches rhoend.
seed0Seed of the solver's internal randomness. It does not make a model provider deterministic.
initial_recovery_evals8Extra evaluations allowed when the initial point is invalid.
max_step_multiplier100.0Upper bound on proposed step sizes relative to the current scale.
radius_policy"auto"Radius update rules; "auto" is the ratio test with a hold for explicit hypothesis points, the only policy.

Observations and caching

Observation and caching options
OptionDefaultDescription
objective_noise"unknown"What you declare about observation noise: "unknown", "none" or "present". It selects the automatic evaluation cache and the statement every agent receives.
cache_evaluationsNoneAnswer a repeated point from the ledger. By default on for "none" and off otherwise; True is refused with "present".
cache_invalid_evaluationsFalseAlso cache invalid or non-finite outcomes.

Model, runner and tools

Model, runner and tool options
OptionDefaultDescription
modelNone"env", a provider specification, or a model client, for the built-in runner.
runner_optionsNoneSettings of the built-in runner, such as max_tokens, temperature, max_turns, compaction and retries. Refused together with a supplied runner.
user_instructionsNoneOptional user guidance delivered with the task. It does not replace the system instructions or relax any limit. Do not put credentials here.
toolsetNoneResolves to "nexteval.toolset/1", the only toolset.
task_tool_surface"full"Every task tool; the only surface.
task_action_surface"generic"Action surface exposed to the agent, such as "generic".
param_coercion"lossless"Lossless tool-parameter coercion mode ("lossless" or "strict").
analysis_helpersTruePredefine fit_poly and suggest for run_analysis code.
analysis_block_scipyFalseMake SciPy unavailable to run_analysis code even when it is installed. The agent's instructions do not change.
memory_max_entries16Maximum notes the agent may keep for one task; memory_max_chars (4000) and memory_max_entry_chars (500) bound their size.
include_tool_readinessFalseInclude tool-readiness diagnostics in state observations.

Tracing and output

Tracing and output options
OptionDefaultDescription
traceNone"off", "summary", "trajectory", "debug" or "develop". Persistent tracing is opt-in.
run_dir"runs"Directory for trace files when tracing is enabled.
return_evaluation_historyFalseInclude the evaluation ledger in the result as history.
log_level"off"Printed diagnostic level; separate from trace capture.

DFOResult

A dictionary with attribute access, similar to SciPy's OptimizeResult.

x / fun
The best valid recorded point and its objective value.
nfev
Actual objective calls recorded in the ledger; equal to ledger_length. Values answered from the cache consume nothing; exceptions and non-finite values stay recorded.
status / message
The numeric termination status and its explanation.
success
The completion flag. It does not, by itself, establish a valid trial for a comparison.
completion_quality
Complete, early stop, or degraded protocol. Read it together with comparable_under_budget before pooling results.
runner_report
The session's completion and transport report. Its fallback entry says whether the autopilot completed the run, why, and what it spent.
model_request_count
gateway_call_count
method_step_count
Counters with distinct scopes: model requests through the built-in meter, tool calls, and steps of numerical methods. A tool call is not one model request and not one evaluation.
history
The evaluation ledger, when return_evaluation_history=True: one record per objective call, in call order.
observation_policy
The requested and effective noise declaration and cache setting.
trace_path
The trace directory when persistent tracing is enabled.

Provider clients

The clients implement the transport to a tool-calling model with the standard library only. Live behaviour must be checked against the provider and model you use; connectivity alone does not show that tool calls work.

AnthropicCompatClient
Messages route with tool use; configured by ANTHROPIC_BASE_URL, ANTHROPIC_MODEL and ANTHROPIC_AUTH_TOKEN. Selected by model="env".
OpenAICompatClient
Chat Completions with tool calls and a Bearer token; configured by NEXTEVAL_OPENAI_BASE_URL, NEXTEVAL_OPENAI_MODEL and NEXTEVAL_OPENAI_API_KEY.
AzureOpenAICompatClient
Azure's deployment URL and api-key header; configured by the AZURE_OPENAI_* variables.
FakeTextModel
Replays scripted replies. For tests and examples only.
ProviderConfigurationError
ProviderRequestError
Raised for a missing or invalid provider configuration, and for a failed provider request.

Runners

NativeToolRunner
The built-in runner, also exported as ToolCallingTaskRunner. Configure it through runner_options.
CodexTaskRunner
The Codex adapter: one Codex app-server thread for the whole solve. Main arguments: model (required), effort, max_turns, timeout_s, base_url, env_from, model_provider and cli_path.
ClaudeCodeTaskRunner
The Claude Code adapter over the Claude Agent SDK, an optional extra. Main arguments: model, effort, max_turns, timeout_s, base_url and env_from.
FallbackRunner(agent)
Wraps a runner so that the textbook autopilot spends the remaining budget, in the same run, when the agent session raises, reports an error or a timeout, or returns without finalize.
AutopilotRunner
The autopilot on its own: the fixed textbook policy, with no model. Pass it as minimize(..., runner=AutopilotRunner()) to run the model-free reference; report it as a reference, not as an agent result.

A runner may define preflight(settings), which is called before the initial evaluation so that an invalid configuration costs no objective call.

Harness helpers

For other agent harnesses, the package exports the shared task inputs; all of them import without an optional SDK.

build_task_messages
The system prompt and the task message that every runner sends.
portable_tool_specs
The tool definitions in a harness-neutral form.
check_tool_registration
tool_schema_fingerprint
A check that a harness registered exactly these tools, and the digest of their schemas.

Results across packaged benchmark tasks are a separate matter: see NextEval Bench.