solprop.calibration.controller#

The calibration controller: children, rounds, a deadline and publication (SEP 46).

The controller never loads the native extension (section 6.2). It starts one child per candidate process setting (solprop.calibration.child), each pinned to the allocation being calibrated, plus one child on the built-in rules, and times the calibration problems’ windows in interleaved rounds: in each round every child once, in a fresh order, and the built-in child twice for the A/A noise. The selection rule (solprop.calibration.selection) admits a candidate only if it wins every round by more than the noise; the winner is then validated against the built-in child on fresh rounds, window by window. Everything runs under one monotonic deadline per level (section 6.9): a child that overruns is terminated, a stage that cannot finish its rounds is skipped and reported, and nothing partial is published. Publication writes one complete generation, keeping the other allocation classes’ rows, and swaps current (section 6.8).

Module Attributes

`LevelName`	A measuring level's name.
`LEVELS`	The levels: about 30 s, two minutes and seven minutes.
`ORDER`	Levels by depth: a higher level replaces a lower one for the same allocation.
`SINGLE_CPU`	Why an allocation of one CPU is never measured.
`POLICY`	The plan policy's effort for each group (`fftw_seed_effort`).

Functions

`builtin_wait`(budget, top_tier)	Return the built-in rule's wait for a pool of budget threads (ADR-071).
`calibrate`([level, budget, environ, cpus, ...])	Calibrate this machine's native backend for the process's allocation.
`candidates`(level, allocation, budget[, top_tier])	Return a level's candidate process settings for an allocation.
`nested`(topology, cpus)	Return the nested allocations a calibration of cpus also measures.

Classes

`CalibrationReport`(level, status, ...[, ...])	What a calibration measured, chose and published.
`Candidate`([threads, wait])	One candidate's process settings.
`Level`(name, budget, rounds, window, ...[, ...])	One calibration level's budget and depth.

class solprop.calibration.controller.CalibrationReport(level, status, fingerprint, allocation, chosen=<factory>, builtin=<factory>, evidence=<factory>, reason='', seconds=0.0, path=None)[source]#

Bases: object

What a calibration measured, chose and published.

Attributes:

levelstr: The level run.
statusstr: "published", "inconclusive" (nothing published: a busy or noisy machine, a failed child, or too little time), or "held" (another process is calibrating this allocation).
fingerprintstr: The machine and build’s store key.
allocationMapping[str, object]: The allocation class calibrated.
chosenMapping[str, object]: Each setting chosen (the process’s width and wait), the built-in values where none replaced them.
builtinMapping[str, object]: The built-in rules’ values.
evidenceMapping[str, object]: The noise, every candidate’s verdict, the validation and the probes.
reasonstr: Why the status is what it is.
secondsfloat: The wall time taken.
pathpathlib.Path or None: The published generation, if any.

allocation: Mapping[str, object]#

builtin: Mapping[str, object]#

chosen: Mapping[str, object]#

describe()[source]#

Return the report as lines for a terminal.

Returns:

str: The status, the reason, each setting beside its built-in value, and the profile written.

evidence: Mapping[str, object]#

fingerprint: str#

level: str#

path: Path | None = None#

reason: str = ''#

seconds: float = 0.0#

status: Literal['published', 'inconclusive', 'held']#

class solprop.calibration.controller.Candidate(threads=None, wait=None)[source]#

Bases: object

One candidate’s process settings.

Attributes:

threadsint or None: The pool width, or None for the built-in rule.
waitstr or None: "spin" or "passive", or None for the built-in rule.

environment(base)[source]#

Return a child’s environment, which never calibrates.

base without the user’s thread, wait and wisdom settings, with this candidate’s, and with FFT plans at estimate, so that no stage outside the FFT stage spends its time planning (SEP 46 section 6.3). NumPy’s and SciPy’s BLAS threads go idle at once (solprop._process.child_environment), so that they do not spin on the CPUs being timed.

Parameters:

baseMapping[str, str]: The controller’s environment.

Returns:

dict of str to str: The child’s.

property name: str[source]#

"built-in", or the wait and width, such as "spin/6".

Returns:

str: The candidate’s name in reports and scores.

threads: int | None = None#

wait: str | None = None#

solprop.calibration.controller.LEVELS: dict[str, Level] = {'quick': Level(name='quick', budget=30.0, rounds=3, window=0.08, problems=('gnlse', 'averaged', 'line'), widths=0, grain=4, fulls=False, reserve=0.2, planning=0.0), 'standard': Level(name='standard', budget=120.0, rounds=5, window=0.08, problems=('gnlse', 'capillary', 'averaged', 'radial', 'line', 'gnlse_large', 'capillary_large', 'radial_large', 'line_large'), widths=2, grain=10, fulls=False, reserve=0.15, planning=0.5), 'thorough': Level(name='thorough', budget=420.0, rounds=7, window=0.3, problems=('gnlse', 'capillary', 'averaged', 'radial', 'line', 'gnlse_large', 'capillary_large', 'radial_large', 'line_large'), widths=2, grain=40, fulls=True, reserve=0.2, planning=0.5)}#

The levels: about 30 s, two minutes and seven minutes.

SEP 46 section 6.9 set 20 s, 90 s and 5 min. With those, the FFT stage’s timing in the step ran out of rounds at the standard and thorough levels on an Intel Xeon Gold 6240, 16 cores of an AMD EPYC 7763 and an Apple M5 Max, and the standard level’s capture ran out of its share (solprop issue #595); the levels now give it the time.

class solprop.calibration.controller.Level(name, budget, rounds, window, problems, widths, grain, fulls=False, reserve=0.2, planning=0.5)[source]#

Bases: object

One calibration level’s budget and depth.

Attributes:

namestr: "quick", "standard" or "thorough".
budgetfloat: Its deadline, in seconds, for all of its work.
roundsint: Interleaved rounds per comparison, and the fewest that decide.
windowfloat: The seconds each timed window aims at.
problemstuple of str: The calibration problems timed.
widthsint: The team widths tried besides the built-in rule’s.
grainint: The most grain candidates timed.
fullsbool: Whether the grain moves the classes’ full sizes as well as their thresholds.
reservefloat: The share of the budget kept for validation and publication.
planningfloat: The share of the time left after the waiting and team that the grain stage leaves to FFT planning, which follows it; none at the quick level, which keeps the plan policy.

budget: float#

fulls: bool = False#

grain: int#

name: LevelName#

planning: float = 0.5#

problems: tuple[str, ...]#

reserve: float = 0.2#

rounds: int#

widths: int#

window: float#

type solprop.calibration.controller.LevelName = Literal['quick', 'standard', 'thorough']#: A measuring level’s name.

solprop.calibration.controller.ORDER: dict[str, int] = {'quick': 0, 'standard': 1, 'thorough': 2}#: Levels by depth: a higher level replaces a lower one for the same allocation.

solprop.calibration.controller.POLICY: dict[str, str] = {'planes': 'measure', 'rows': 'measure', 'threaded_rows': 'estimate'}#: The plan policy’s effort for each group (fftw_seed_effort).

solprop.calibration.controller.SINGLE_CPU = 'one CPU: no team, waiting or grain to choose (the built-in table applies)'#

Why an allocation of one CPU is never measured.

A standard calibration of one core of an AMD EPYC 7763 ran out of its 90 s before three rounds of its problems in series, and could not have chosen anything (SEP 46 Part II).

solprop.calibration.controller.builtin_wait(budget, top_tier)[source]#

Return the built-in rule’s wait for a pool of budget threads (ADR-071).

Idle threads spin unless the team is wider than the chip’s fastest kind of core.

Parameters:

budgetint: The pool’s width.
top_tierint or None: The fastest kind’s cores on a chip with several, else None.

Returns:

str: "spin" or "passive".

Examples

>>> builtin_wait(6, 6), builtin_wait(18, 6), builtin_wait(36, None)
('spin', 'passive', 'spin')

solprop.calibration.controller.calibrate(level='standard', budget=None, *, environ=None, cpus=None, progress=None)[source]#

Calibrate this machine’s native backend for the process’s allocation.

At the standard and thorough levels the nested allocations it can pin children to (one socket, one NUMA node, one L3 group) are calibrated too, while the deadline allows; each gets its own row (SEP 46 section 6.4).

Parameters:

levelstr, optional: "quick" (about 30 s), "standard" (about two minutes) or "thorough" (about seven minutes).
budgetfloat or None, optional: The deadline in seconds, the level’s when None.
environMapping[str, str] or None, optional: The environment; the process’s when None.
cpussequence of int or None, optional: The allocation’s CPUs, the process’s affinity mask when None.
progresscallable or None, optional: Called with a line of text as each stage starts.

Returns:

CalibrationReport: The allocation’s own report; the nested allocations’ are in its evidence (nested).

Raises:

CalibrationError: For an unknown level or a budget that is not positive.

solprop.calibration.controller.candidates(level, allocation, budget, top_tier=None)[source]#

Return a level’s candidate process settings for an allocation.

The other wait at the built-in width, then narrower widths at their built-in wait (the fastest kind’s cores on a chip with several, one socket’s on several sockets, one L3 group’s, half the cores), then the other wait at the first of them; at most MAX_CANDIDATES.

Parameters:

levelLevel: The level, whose widths bounds the narrower widths tried.
allocationAllocationClass: The allocation being calibrated.
budgetint: The built-in rule’s width for it (thread_budget).
top_tierint or None, optional: The chip’s fastest kind’s cores, where it has several.

Returns:

list of Candidate: The candidates, none equal to the built-in rule.

solprop.calibration.controller.nested(topology, cpus)[source]#

Return the nested allocations a calibration of cpus also measures.

One socket’s CPUs, one NUMA node’s and one L3 group’s, where cpus spans more than one (SEP 46 section 6.4): one calibration of a whole node then covers most smaller jobs on it. Equal subsets are listed once, at most two.

Parameters:

topologysolprop_native._topology.Topology: The machine’s.
cpussequence of int: The allocation’s CPUs.

Returns:

list of tuple of int: The first group’s CPUs for each kind of group spanned.