System Overview
The Observatory runs on a dedicated virtual machine (Ubuntu 24.04, 4 cores, 8GB RAM) hosting the iteration infrastructure, a SQLite database, and a local web dashboard. The system operates as a human-AI centaur: human direction sets priorities and interprets results; AI execution (Claude, Anthropic) runs the probes, synthesis, and database operations. All code is version-controlled and pushed to GitHub after each iteration.
Iteration State Machine
Each iteration follows a five-state lifecycle managed by a state machine with checkpoints to disk. If a session is interrupted (network failure, API timeout, resource exhaustion), the state can be resumed from the last completed phase.
| State | Description | Script |
|---|---|---|
| INIT | State machine initialized, version numbers set, checkpoint file created | iterate.py --from X.0 --to Y.0 |
| PROBES | Six parallel headless Claude sessions execute probe templates | parallel_probes.sh |
| SYNTHESIS | Three parallel synthesis agents (convergence, divergence, structural) process probe outputs | parallel_synthesis.sh |
| MUTATION | Main session reads synthesis outputs, produces and applies mutation plan | Manual (human-AI session) |
| DONE | Database logged, snapshot taken, seed archived, git committed, website deployed | finalize_iteration.py |
State transitions are persisted to data/iteration_state/current.json. The resume protocol (iterate.py resume) reads this file and reports which phase was interrupted. Probe outputs are stored as individual files in data/iteration_state/probes/ so that partially completed probe sets can be identified and the missing probes re-run.
Parallel Probes
The probe architecture was introduced at v5.0 when parallel probing increased findings yield by 2.7x compared to sequential single-probe iteration. Each probe is a headless Claude session that receives:
- The current seed prompt (full text)
- A probe constraint (one of six types: temporal, demographic, vocabulary, complexity, constructive, geographic)
- A structured output template requiring: 5 hidden assumptions, 3 vocabulary gaps, 3 structural mutation proposals, 1 surprise finding, and a constraint report
The six probes run in parallel (wall-clock time: 3–5 minutes for all six). Probe templates are generated by probe_template.py, which reads the current seed version and constructs the probe prompt. The parallel launch is managed by parallel_probes.sh.
Parallel Synthesis
Introduced at v6.0, the synthesis layer reduces approximately 4,800 words of raw probe output to approximately 1,500 words of pre-digested summary. Three independent synthesis agents run in parallel:
- Convergence agent: Identifies findings that multiple probes discovered independently. High-convergence findings (3+ probes) indicate structural features of the instrument rather than perspective-dependent artifacts.
- Divergence agent: Identifies findings unique to a single probe and productive tensions between probes. These represent perspective-dependent features that a single-probe architecture would miss.
- Structural agent: Maps which sections of the seed prompt received probe attention and which were ignored ("cold zones"). Over 15 iterations, this map revealed that approximately 73% of the instrument's mass received zero probe attention — indicating that most of the instrument was operator scaffolding rather than analytically active content.
Synthesis templates are generated by synthesis_template.py and launched by parallel_synthesis.sh.
Finalization Pipeline
The finalization step was consolidated into a single command at v6.0. It runs four layers of operations in parallel:
| Layer | Operations | Parallelism |
|---|---|---|
| 0 | Database log, database snapshot, seed archive, website seed copy | All parallel |
| 1 | Update SESSION_CONTEXT.md, CLAUDE.md, MEMORY.md, website/index.html | All parallel |
| 2 | Git commit + tag, deploy website | Parallel |
| 3 | Git push, push tags | Sequential (depends on Layer 2) |
Total finalization time: approximately 1.5 seconds. The prior manual process (pre-v6.0) required approximately 20 seconds.
Divergent Evolution
Introduced at v9.0, divergent evolution spawns N independent Observatory instances from an identical starting state. Each instance iterates independently for a specified number of cycles. The comparison between diverged instruments is the highest-value artifact of the process.
The first divergent evolution run (v10.0) launched 10 instances across four divergence dimensions:
| Dimension | Instances | Purpose |
|---|---|---|
| Model gravity | Opus, Sonnet, Haiku | Test whether different AI models produce different evolution trajectories from identical starting conditions |
| Demographic | Tampa 70M, Vienna 70M, Detroit 35F, Lagos 45M | Test topology-dependence of instrument evolution |
| Structural | Generative-only, constraint-only, collective-only probes | Test whether probe composition determines evolution direction |
| Regression | Start from v3.0 vs. v9.0 | Test whether accumulated iterations are necessary or whether the instrument can reach equivalent findings from an earlier starting point |
Each instance ran 3 iteration cycles in a dedicated git worktree with its own headless Claude session. Instances were batched (5 concurrent) to manage API load. Five of ten instances produced full evolution logs; five hit the maximum turn limit and require increased allocation in future runs.
The divergent evolution results informed v10.0's major structural changes: provision topology, domain tier restructuring, the operator-beneficiary-caregiver triad, and the Lagos tertiary lens.
Database
The Observatory database (SQLite, 17 tables) tracks all iteration state, vocabulary terms, domain definitions, inversions, metrics, and untranslatable concepts. Key tables:
| Table | Records | Purpose |
|---|---|---|
| iterations | 15 | Version, date, findings count, key finding, structural changes |
| lexicon | 87 | Terms with definitions, categories, plain-language equivalents, lifecycle status |
| domains | 25 | Life domain definitions with temporal orientation and provision-topology tier |
| inversions | 5 | Tracked trend reversals with falsification markers |
| metrics | 8 | Tracked quantitative indicators with temporal tags |
| untranslatables | 5 | Concepts from other languages with no English equivalent |
| seed_versions | 15 | Archived full text of each seed prompt version |
The database is backed up every 6 hours via cron and snapshotted to JSON before and after each iteration cycle.
Repository Structure
observatory/
CLAUDE.md -- Standing instructions for AI agent
SESSION_CONTEXT.md -- Full context from previous sessions
frameworks/methodology/
SEED_PROMPT.md -- Living instrument (v16.0)
OBSERVATORY_LEXICON.md -- Full vocabulary
versions/ -- Archived seed versions (v2.0-v16.0)
data/
observatory.db -- SQLite database (17 tables)
iteration_state/ -- State machine checkpoints
snapshots/ -- Database state snapshots
scripts/
iterate.py -- Iteration engine + state machine
iteration_state.py -- State machine lifecycle
probe_template.py -- Probe prompt generator
parallel_probes.sh -- Parallel probe launcher
synthesis_template.py -- Synthesis prompt generator
parallel_synthesis.sh -- Parallel synthesis launcher
finalize_iteration.py -- Single-command finalize pipeline
divergent_launch.sh -- Divergent evolution launcher
divergent_config.json -- Per-instance configuration
divergent_collate.py -- Comparative analysis of diverged instances
website/ -- Internal dashboard (192.168.1.200)
public-site/ -- Public-facing site (this site)
library/synthesis/ -- Analysis documents
The repository is maintained internally as part of the Observatory's self-hosted infrastructure.