The Scheduler Boundary — Architecture

The Scheduler Boundary — Architecture

This document explains why the scheduler boundary is shaped the way it is. The operational specification is CONTRACT.md; this is the reasoning beneath it.

1. The one idea

Forge HPC owns the domain. A scheduler merely provides observations about reality. The scheduler never defines the domain; the domain defines what a scheduler must produce.

Every other decision in this package follows from that inversion of the usual relationship. In most systems, the scheduler is the source of truth and the platform bends to its vocabulary — its state names, its field formats, its command output. The platform ends up speaking a particular scheduler’s dialect, and the second scheduler forces either a rewrite or a second dialect. Forge refuses this at the boundary: the domain vocabulary is fixed first (domain), and a scheduler earns the right to be observed only by producing that vocabulary. The direction of definition runs from Forge to the scheduler, never the reverse.

This is FORGE-001’s founding move — unify meaning, not systems — applied to schedulers: the twelve systems dissolve not because they were merged but because they are all made to speak one set of nouns.

2. Why the boundary is in the domain plane

The scheduler contract lives in Forge HPC (the domain plane), not in Forge Core and not in an instance. FORGE-101 Part 15 places it exactly here, and the placement is load-bearing:

• It is not universal, so it is not Core. A telescope, a sequencing centre, or a digital archive has no scheduler; Core must be able to serve all of them and therefore must not know that schedulers exist (FORGE-101 Part 14, rule 2). The word “scheduler” appearing in Core would be a vocabulary-firewall violation.
• It is not one deployment’s, so it is not an instance. Every batch system on Earth shares the concept of a scheduler that admits, runs, and accounts for work. That shared expertise is precisely what a domain is (FORGE-101 Part 2.2). One instance’s endpoints and credentials are instance facts; the shape of “what a scheduler is” is domain expertise.

So the abstraction over schedulers is itself the proof that the domain plane must exist: it is knowledge that is neither universal nor local, and it needs a home that is neither Core nor instance.

3. Why the contract returns domain objects exclusively

Every operation returns Forge HPC domain objects and nothing else. This is what makes the contract truthful and reusable:

• Truthful. A domain object is self-validating (it cannot exist in an invalid state) and provenance-honest (timestamps are aware, states are the universal set, unknowns are None rather than invented). A scheduler that produces domain objects cannot smuggle an unvalidated or scheduler-shaped fact across the boundary.
• Reusable. Because the output vocabulary is fixed, everything downstream — projections, analytics, publication — is a pure function of domain objects and never of a scheduler’s dialect. A second scheduler changes what is behind the boundary and nothing in front of it. This is the implementation independence FORGE-100 Part 13 requires, demonstrated at the hardest layer.

If a scheduler reports something the domain cannot express, the honest responses are to map it to the nearest domain concept, to report unknown, or — for a genuinely universal new concept — to amend the domain. The forbidden response is to widen the contract with a scheduler-specific field.

4. Why adapters are separate

The contract knows nothing of SSH, REST, files, sockets, databases, or message queues. Those are how a particular source system is reached, and they live in adapters, behind a separate seam (adapters.py):

• ObservationSource — acquires raw SourceObservations over a window. It owns every mechanism of acquisition; the interface knows only that a bounded set of raw observations can be asked for.
• ObservationTranslator — turns one raw observation into validated domain objects, or raises TranslationError rather than guessing.

The split matters for three reasons:

1. The contract stays pure. A SchedulerContract is a source-neutral view of reality; mixing transport into it would couple every consumer to a connection that can fail at read time, defeating the static, observe-only thesis.
2. Adapter logic is domain; adapter binding is instance (FORGE-101 Part 8.4). The knowledge of what a source system’s record means is reusable across every deployment of that system and belongs to the domain; the endpoint and credential that reach this deployment’s copy belong to the instance. Declaring the translator interface here, and leaving both the concrete transport and the binding outside, is that split made structural.
3. Reality is untrusted. An adapter is where reality’s messy edge meets the platform’s validated interior. SourceObservation records raw payloads with provenance and never edits them (evidence is not edited); the translator is the single checkpoint where raw becomes valid, and it fails loudly when it cannot — “beautifully provenanced garbage is still garbage.”

5. Why capabilities and negotiation, rather than one fat interface

Real schedulers differ in what they can observe: one exposes accounting, another does not; one supports incremental change streams, another only full snapshots. Two designs could absorb this — a fat interface where every scheduler implements every operation (returning empty for what it lacks), or a capability model where a scheduler declares what it offers. Forge chooses capabilities because:

• Emptiness must not be a lie. An empty result from a fat interface is ambiguous: did the scheduler observe nothing, or can it not observe this at all? The capability model makes “cannot” a declared fact and “nothing” an honest empty tuple, and UnsupportedOperation makes asking for the undeclared a loud failure rather than a silent zero.
• Consumers negotiate on evidence. A consumer states its requirements and learns before observing whether they can be met — no surprise at read time.
• Capabilities are the extension seam. Growth is a new capability + operation
  • contract minor bump, additive by construction, leaving every existing scheduler valid.

6. Why compliance is mechanical

FORGE-103’s meta-standard is that a rule a machine can check must be checked by a machine. Whether an implementation honours the contract — declares what it implements, implements what it declares, reports a coherent version, builds a deterministic manifest — is entirely checkable statically, so verify_contract_compliance checks it, deterministically, without invoking a single observation. Review is thereby freed for what only humans can judge: is the mapping from a real source to domain objects honest.

7. Determinism, because everything downstream is a projection

The contract is designed so that equal reality produces equal output: manifests are deterministic, observation collections canonicalize their own ordering and reject duplicate identities, and negotiation is a pure function. This is not fastidiousness — it is what lets everything above the boundary be a pure, byte-reproducible projection (FORGE-100 Part 2.5). One home per object holds inside every observation, not only inside the truth store that will eventually keep it, because a duplicate admitted at the boundary is a duplicate the whole pipeline must then be trusted to remove.

8. What this boundary deliberately excludes

• No scheduler. No Slurm, no PBS, no Kubernetes, no batch system. This package is the shape; implementations are elsewhere and later.
• No command vocabulary, no source formats, no environment variables, no scheduler configuration. All of it is adapter-internal.
• No transport. SSH, REST, files, sockets, databases, message queues — all adapter-internal.
• No instance knowledge. No endpoints, credentials, hostnames, or policies.
• No runtime execution, no I/O, no network, no analytics, no prediction. This is architecture: interfaces, capability definitions, negotiation, validation, and typed failure. Everything that runs comes later, behind these contracts.