# OCaml RDMA/OpenZFS/GPU/SST Simulation Package

This package implements a deterministic OCaml simulation model for two related systems:

- **PCS** — Project Coherent Storage / Coherence-CE style inference storage scaling.
- **USM** — Universal Storage Mesh scaling with metadata federation, queueing, bulk data movement, and backpressure.

The simulator extends the earlier control-plane model into modular event/state models for:

- RoCE-v2 / RDMA packet batches, including MTU fragmentation, RDMA opcodes, queue pairs, PSN ranges, wire bytes, serialization latency, utilization, ECN marks, CNP feedback, PFC pause time, retransmits, and backlog.
- OpenZFS transaction groups, including Open, Quiescing, Syncing, and Synced/Commit phases, dirty-data thresholding, timeout-triggered TXG close, sync duration, writeback lag, flush overhead, DPU/RDMA path limits, and sync-thread/storage-node scaling.
- GPU kernel phases, including prefill attention/GEMM, decode attention/KV, GPU-direct RDMA waits, collective-like SEND traffic, and USM checksum/compress/vector-index kernels. Duration is modeled as max(compute time, HBM time) plus launch overhead, RDMA wait, and GPU queue time.
- SST-style components and events, including `WorkloadGen`, `Scheduler`, `MetadataRouter`, `GpuExecutor`, `RdmaNic`, `RoceSwitch`, `ZfsPool`, `BackpressureController`, and `ScalingController`.
- Vertical, horizontal, and combined scaling decisions under RoCE-v2 latency/bandwidth constraints.

The package includes source, scenarios, tests, generated sample outputs, and PlantUML diagrams rendered to both PNG and SVG.

## Requirements

Validated in this sandbox with:

```text
OCaml 5.3.0
Dune 3.17.2
Yojson OCaml library
PlantUML
Graphviz
Python 3.13
```

On Debian/Ubuntu-like systems, the equivalent packages are typically:

```sh
apt-get install ocaml-nox ocaml-dune libyojson-ocaml-dev plantuml graphviz python3
```

## Build

```sh
make build
```

This creates:

```text
bin/storage-sim
```

## Run PCS scenario

```sh
./bin/storage-sim pcs \
  --scenario scenarios/pcs/roce_gpu_txg_scale_both.json \
  --out out/pcs-scale-both
```

## Run USM scenario

```sh
./bin/storage-sim usm \
  --scenario scenarios/usm/mesh_roce_txg_scale_both.json \
  --out out/usm-scale-both
```

## Render diagrams

```sh
make render-diagrams
```

PlantUML sources are in:

```text
docs/diagrams/puml
```

Rendered diagrams are in:

```text
docs/diagrams/png
docs/diagrams/svg
```

## Run the complete repeatable test suite

```sh
make test
```

The test target:

1. Builds the OCaml binary.
2. Renders all PlantUML diagrams to PNG and SVG.
3. Runs PCS and USM scale-both scenarios.
4. Verifies all expected output files exist and contain model evidence.
5. Runs each scenario twice and checks deterministic output hashes.

## Output files

Each run emits:

```text
summary.json              Run summary and final topology.
timeline.csv              Per-tick cross-model metrics.
rdma_packets.csv          RoCE-v2/RDMA packet-batch model.
openzfs_txg.csv           OpenZFS transaction group lifecycle events.
gpu_kernels.csv           GPU kernel phase model.
sst_components.csv        SST-style component inventory.
sst_events.csv            SST-style event stream.
scaling_decisions.csv     Vertical/horizontal/both scaling decisions.
```

## Scenarios included

```text
scenarios/pcs/roce_gpu_txg_scale_both.json
scenarios/pcs/roce_vertical_only.json
scenarios/pcs/roce_horizontal_only.json
scenarios/usm/mesh_roce_txg_scale_both.json
scenarios/usm/metadata_backpressure_vertical.json
```

## Notes on SST

The simulator emits SST-style component and event streams in CSV so the same model can be moved into SST-Core. It does not require SST-Core to run. A C++ SST component skeleton is included under `docs/sst/pcs_usm_component_skeleton.cc` as integration collateral; it is intentionally not compiled by `make test` because SST-Core is not part of this package.
