What makes a security
experiment repeatable?
BlackIce studies how to represent a security exercise as an explicit, replayable world: its files, services, commands and remembered relationships. Change the scenario, inspect the evidence, and separate what the simulator knows from what it merely returns.
Research setting IBM Original PhantomBridge / BlackIce work · Current software reconstruction
The research question
Preserve the experiment,
not just its transcript.
A transcript tells us what an operator typed and what appeared next. It often leaves out the state that made the response possible: which files existed, which services were known, and which answers came from a prepared scenario.
BlackIce makes those dependencies explicit. A virtual filesystem, a service registry and response rules form a small executable model. A separate memory graph records successful transitions and ranks related techniques. This creates a setting for studying repeatability, state coverage and inspectable suggestions.
The current implementation reconstructs a prior project from surviving records. Its tests evaluate this implementation; the original source tree was not recovered.
01 / Controlled replay
Same setup. Same script. Inspect the difference.
Run two fresh worlds through a four-command script. Then change one initial file in World B and repeat. The table follows the actual commands; the comparison includes both execution records and serialized state.
World A / reference
Initial note: baseline
SHA-256: available after running
World B / comparison
Initial note: baseline
SHA-256: available after running
| Command | World A | World B | Match |
|---|---|---|---|
whoami
|
Run the comparison to populate measured outputs. | ||
Ready to compare two executions.
What is actually compared?
Sorted JSON includes the command records and the world’s own enumerable state: filesystem entries, service observations, response templates, identity, environment, users, groups, processes and configuration. Regular expressions are recorded by pattern and flags. Code and prototype methods are held fixed, not serialized. Equality is tested on the strings themselves; the SHA-256 digests are displayed as compact references.
A matching pair establishes agreement for this fixture and script. It does not prove universal determinism, real-system fidelity, a security boundary, or equivalence with the Python snapshot format.
02 / State & response provenance
A command is an observation of a model.
Explore the small world below. Files and command results live in this browser’s memory. A command such as nmap renders seeded service records; it does not scan a network.
Each result identifies its response source. Built-in commands operate on modeled state; templates supply an authored answer; unsupported commands return an explicit unknown result.
The fixture contains a note and seeded service observations. No command has run yet.
Response source and exit code will appear here.
03 / Relationships as evidence
What changes when the memory observes a sequence?
The memory represents techniques using sparse identifiers and a directed graph. Run a simulated discovery sequence, then inspect how remembered transitions change the ranking. A higher score means a stronger match under this model’s rules; it is not a probability of success.
-
T1046Discover services Not yet observed -
T1135Enumerate shares Not yet observed -
T1087List accounts Not yet observed
Before observation, only the built-in graph contributes transition evidence.
| Candidate | Identifier similarity | Path evidence | Walk score | Combined rank score |
|---|---|---|---|---|
| Enable JavaScript to inspect the calculated ranking. | ||||
Read the scoring rule and its assumptions
score = 0.45 × cosine + 0.25 × path + 0.30 × walk
Cosine compares sparse technique, CAPEC and tactic identifiers. Path evidence sums weighted simple paths up to four edges and caps the result. The random walk restarts at the query, with continuation factor 0.85 and at most ten iterations. These are implementation choices, not learned optimal weights.
Built-in graph edges have weight 0.4. Observed successful transitions add a count-normalized, recency-weighted contribution up to 0.6. The demo uses logical execution ticks, not elapsed wall-clock time. Its embedded knowledge slice contains eleven techniques. The graph and technique mappings support this demonstration; their presence does not validate a recommended action.
Method / From notes to a scenario
Extract, normalize, specify, replay.
Recover commands
Identify executable lines in fenced transcript blocks. Narrative context remains evidence to interpret, rather than automatically executable behavior.
Separate identity from structure
Replace scenario-specific hosts, users and certificate-authority names with placeholders while preserving command syntax.
Make missing state explicit
Seed files and service observations. Register response templates where a command is not modeled. This step determines much of a scenario’s fidelity.
Keep an inspectable record
Execute the script, record response sources, and compare states. The browser demonstrations begin with prepared fixtures; they do not ingest arbitrary transcripts.
Evidence / Checked 15 October 2026
What the current record establishes.
| Check | Result | Interpretation |
|---|---|---|
| World engine · two parity suites | 137 / 137 passed | Agreement on supplied command cases, including response precedence, pipelines, redirection and path handling. |
| Memory, ranking and session parity | 67 / 67 passed | Agreement with saved reference values; some floating-point comparisons use a 10⁻⁹ tolerance. |
| Paired replay and changed-input control | Run above | Directly inspect execution/state equality for a declared fixture, then check that a changed note is detected. |
The reference captures are supplied project artifacts. These runs do not independently regenerate the Python captures or evaluate training outcomes, suggestion quality on held-out scenarios, or fidelity to a live operating system. The wider Python and Rust test counts from the source landing page are not reported here as newly verified results.
Historical record and reconstruction
The recovery repository distinguishes explicit historical facts (H1), historical behavior (H2), newly reconstructed implementation (R1), and unresolved details (U). The browser engines are implementations of the current reconstruction. Their passing tests do not recover missing original source or settle unresolved historical behavior.
The original PhantomBridge work supplies the IBM research setting. This page presents a current research companion, with the historical setting kept separate from the new code and validation record.
Next experiments
Measure fidelity as well as repeatability.
Scenario coverage
Withhold commands from a transcript, then test whether the world can reproduce their outcomes. Track template hits, modeled responses and unknowns separately.
Memory ablations
Compare the combined ranking with identifier similarity alone, graph paths alone and observed transitions alone on held-out sequences.
Representation limits
Identify distinct real states that the same simulated state would merge. Check whether that loss of distinction changes an exercise’s outcome.
These are proposed evaluations, not completed results.