rusty_marvin/docs/ARCHITECTURE_SUPERSONIC_EVALUATOR.md

# Supersonic LUT Evaluator — Architecture Plan

## Overview

The **Supersonic evaluator** is a port of a Java "Supersonic" poker hand evaluator that uses a pre-computed trie structure to enable 7-card hand evaluation in **exactly 7 array lookups** with no runtime enumeration. This replaces the current Cactus Kev + HashMap approach with a pure array-based lookup that trades memory (~124 MB) for deterministic O(7) performance.

### Two-Phase Design

1. **Offline Table Generator** — A one-time process that builds the `hand_ranks` trie by enumerating all possible card combinations and ranking them via the existing Cactus Kev 5-card evaluator. The result is serialized to disk.

2. **Runtime Lookup Evaluator** — Loads the pre-computed trie from disk and evaluates any 5/6/7-card hand with a simple chain of array lookups.

---

## Architecture Diagram

```mermaid
flowchart TB
    subgraph Offline [Offline: Table Generator]
        CK[Cactus Kev 5-card Evaluator]
        GEN[StateTableGenerator]
        DISK[HandRanks.dat - 124 MB on disk]
        
        GEN -->|uses| CK
        GEN -->|writes| DISK
    end

    subgraph Runtime [Runtime: Supersonic Evaluator]
        LOAD[Load HandRanks.dat]
        LUT[hand_ranks: Vec u32 - 32.5M entries]
        EVAL[evaluate_7_card]
        
        LOAD --> LUT
        EVAL -->|7 array lookups| LUT
    end

    subgraph Fallback [Fallback: Cactus Kev + Cache]
        CK2[CactusKevEvaluator]
        CACHE[HashMap u64 to u32 cache]
        ENUM[C of 7,5 = 21 enumeration]
        
        ENUM -->|uses| CK2
        ENUM -->|populates| CACHE
    end

    DISK -->|loaded by| LOAD
    
    style Offline fill:#e8f5e9
    style Runtime fill:#e3f2fd
    style Fallback fill:#fff3e0
```

### Evaluation Flow

```mermaid
flowchart LR
    CARDS[7 cards] --> ENCODE[Encode: rank * 4 + suit]
    ENCODE --> LOOP[Chain lookup loop]
    
    LOOP --> P1[p = hand_ranks 53 + card0 + 1]
    P1 --> P2[p = hand_ranks p + card1 + 1]
    P2 --> P3[p = hand_ranks p + card2 + 1]
    P3 --> P4[p = hand_ranks p + card3 + 1]
    P4 --> P5[p = hand_ranks p + card4 + 1]
    P5 --> P6[p = hand_ranks p + card5 + 1]
    P6 --> P7[p = hand_ranks p + card6 + 1]
    P7 --> RESULT[p = final hand rank]
    
    style LOOP fill:#fce4ec
    style RESULT fill:#c8e6c9
```

---

## Data Structures

### `hand_ranks: Vec<u32>` — The Main Trie

- **Size**: 32,487,834 entries (~124 MB in memory)
- **Layout**: A flat array where each entry is either:
  - A **pointer** (index into the same array) for intermediate nodes
  - A **hand rank value** for leaf nodes (7-card terminal)
  - A **pointer to a 5-card sub-table** for 5/6-card evaluation

### Card Encoding

Cards are encoded as `rank * 4 + suit`, yielding indices 0–51. This is compatible with the existing [`PackedCard`](holdem_core/src/card.rs) representation where:
- `rank()` returns 0–12 (Two through Ace)
- `suit()` returns 0–3 (clubs, diamonds, hearts, spades)

The conversion function already exists in [`tables.rs`](holdem_core/src/evaluator/tables.rs:181):
```rust
fn packed_card_to_index(pc: PackedCard) -> usize {
    pc.rank() as usize * 4 + pc.suit() as usize
}
```

### Hand Rank Encoding

Each hand rank value is packed as a 16-bit integer:
```
hand_rank = (category << 12) | sub_rank
```

Where:
- `category` (bits 12–15): 0 = high card, 1 = pair, ..., 9 = royal flush
- `sub_rank` (bits 0–11): relative rank within the category

This must be mapped to the existing [`HandValue`](holdem_core/src/evaluator/evaluator.rs:108) struct which uses `(rank << 20) | value` packing.

---

## Table Generation

### Algorithm Overview

The generator builds the trie in two passes:

#### Pass 1: Generate and Sort Keys

For each hand size (5, 6, 7 cards), generate all unique rank combinations (ignoring suit), normalize them by sorting, and create a sorted list of unique keys. This uses:

- **Suit stripping**: Remove suit information to get rank-only combinations
- **Sorting**: Bose-Nelson sorting network for deterministic ordering of ranks within each hand
- **Deduplication**: Only store unique rank combinations

#### Pass 2: Populate Hand Ranks

Walk the sorted keys and populate the `hand_ranks` array:

1. Start at index 53 (indices 1–52 are reserved for card offsets from root)
2. For each unique rank combination, allocate trie nodes
3. At leaf nodes, evaluate the corresponding 5-card hand using the existing [`CactusKevEvaluator`](holdem_core/src/evaluator/cactus_kev.rs) and store the hand rank
4. For 6-card and 7-card nodes, propagate the best 5-card sub-hand rank upward

### Key Normalization

The Bose-Nelson sorting network provides a branch-free, deterministic way to sort up to 7 elements. This ensures that the same set of ranks always produces the same trie path regardless of input order.

### Dependency on Cactus Kev

The generator uses the existing [`CactusKevEvaluator`](holdem_core/src/evaluator/cactus_kev.rs) to evaluate 5-card hands during table population. This means:
- The Cactus Kev module remains a build-time dependency
- It does **not** need to be loaded at runtime when using the Supersonic evaluator
- The [`plans/cactus-kev-evaluator.md`](plans/cactus-kev-evaluator.md) plan remains relevant

### Serialization

The `hand_ranks` array is serialized to disk as a flat binary file:
- **Format**: Raw `u32` little-endian array
- **Size**: 32,487,834 × 4 bytes ≈ 124 MB
- **Checksum**: CRC32 or SHA-256 for integrity verification
- **Location**: Configurable, default alongside the existing jump table file

---

## Runtime Evaluation

### Chain Lookup Algorithm

```rust
fn evaluate_7_card(&self, cards: &[PackedCard; 7]) -> HandValue {
    let mut p: usize = 53; // root node offset
    for card in cards {
        let idx = packed_card_to_index(*card);
        p = self.hand_ranks[p + idx + 1] as usize;
    }
    // p is now the final hand rank value
    decode_hand_rank(p as u32)
}
```

### Evaluation by Hand Size

| Hand Size | Lookups | Final Step |
|-----------|---------|------------|
| 5 cards | 5 chain lookups | `hand_ranks[p]` — extra dereference |
| 6 cards | 6 chain lookups | `hand_ranks[p]` — extra dereference |
| 7 cards | 7 chain lookups | `p` is the final value — no extra step |

The extra dereference for 5/6-card hands exists because the trie stores intermediate pointers at depth 5 and 6, and the actual hand rank is one more lookup away.

### Hand Rank Decoding

The raw `u32` from the trie must be decoded into the existing [`HandValue`](holdem_core/src/evaluator/evaluator.rs:108) format:

```rust
fn decode_hand_rank(raw: u32) -> HandValue {
    let category = (raw >> 12) as u8;  // top 4 bits
    let sub_rank = raw & 0xFFF;        // bottom 12 bits
    let rank = HandRank::from_u8(category).unwrap_or(HandRank::HighCard);
    HandValue::new(rank, sub_rank)
}
```

---

## Integration Plan

### New Files

| File | Purpose |
|------|---------|
| `holdem_core/src/evaluator/supersonic.rs` | Runtime Supersonic evaluator |
| `holdem_core/src/evaluator/supersonic_generator.rs` | Offline table generator (behind `feature = "generator"` flag) |

### Module Changes

#### [`mod.rs`](holdem_core/src/evaluator/mod.rs)

Add new sub-modules:
```rust
#[cfg(feature = "generator")]
pub mod supersonic_generator;
pub mod supersonic;
```

#### [`tables.rs`](holdem_core/src/evaluator/tables.rs) — `JumpTable`

Add a `supersonic` field and modify evaluation methods:

```rust
pub struct JumpTable {
    // ... existing fields ...
    
    /// Supersonic LUT evaluator (loaded from disk, ~124 MB).
    /// When present, used as the primary evaluation path.
    supersonic: Option<SupersonicEvaluator>,
}
```

The evaluation priority becomes:
1. **Supersonic** (if loaded) — 7 array lookups
2. **Cactus Kev + cache** (always available) — 21 enumeration + HashMap
3. **Algorithmic** (fallback) — rule-based evaluation

### Bug Fixes Required

#### Fix 1: Evaluator Guard Condition

**Current** (broken — `data` is always empty in the Cactus Kev architecture):
```rust
// In evaluator.rs lines 697, 715
if self.tables.size > 0 && !self.tables.data.is_empty() {
```

**Fixed**:
```rust
if self.tables.is_built() {
```

The `is_built()` method should check that the Cactus Kev tables are populated (which they always are after `build()`), not whether `data` is non-empty.

#### Fix 2: 7-Card Caching with Interior Mutability

**Problem**: [`evaluate_7_card`](holdem_core/src/evaluator/tables.rs:399) takes `&self` but the HashMap cache requires `&mut self` for insertion. Currently, cache misses just recompute every time.

**Fix**: Use `RefCell<HashMap<u64, u32>>` for interior mutability:
```rust
seven_card_cache: RefCell<HashMap<u64, u32>>,
```

Or, since the `Evaluator` wraps `JumpTable` in `Arc`, use `RwLock` or accept that the Cactus Kev path is always uncached when the Supersonic LUT is available.

---

## Performance Comparison

| Approach | 7-card eval | Memory | Startup | Dependencies |
|----------|------------|--------|---------|-------------|
| **Algorithmic** | O[C of 7,5 x eval] | 0 | Instant | None |
| **Cactus Kev + Cache** | O[21] HashMap lookups on miss | ~50 KB tables + cache growth | ~ms | Build CK tables |
| **Supersonic LUT** | O[7] array lookups | ~124 MB | Load from disk | Pre-generated .dat file |

### Detailed Comparison

| Metric | Algorithmic | Cactus Kev | Supersonic |
|--------|------------|------------|------------|
| 5-card eval | ~2 µs | ~50 ns | ~35 ns |
| 7-card eval first | ~42 µs | ~1 µs | ~50 ns |
| 7-card eval cached | N/A | ~50 ns | ~50 ns |
| Memory usage | 0 | ~50 KB + growing cache | ~124 MB fixed |
| Startup time | 0 | ~1 ms | ~100 ms disk load |
| Deterministic | Yes | No — HashMap ordering | Yes — array indexing |
| Cache-friendly | N/A | No — HashMap | Yes — sequential array |

### When to Use Which

- **Supersonic**: Production simulations, mass hand evaluation, any scenario where 124 MB is acceptable
- **Cactus Kev**: Development, testing, memory-constrained environments
- **Algorithmic**: Correctness oracle, zero-dependency fallback

---

## Phased Implementation Plan

### Phase 1: Fix Immediate Bugs

- [ ] Add `is_built()` method to [`JumpTable`](holdem_core/src/evaluator/tables.rs:118) that checks CK table population
- [ ] Fix guard condition in [`Evaluator::evaluate_5_card`](holdem_core/src/evaluator/evaluator.rs:672), [`evaluate_6_card`](holdem_core/src/evaluator/evaluator.rs:697), [`evaluate_7_card`](holdem_core/src/evaluator/evaluator.rs:715) to use `is_built()`
- [ ] Fix 7-card caching: wrap `seven_card_cache` in `RefCell<HashMap<u64, u32>>`
- [ ] Verify all existing tests still pass after bug fixes

### Phase 2: Implement Supersonic Generator

- [ ] Create `supersonic_generator.rs` behind `feature = "generator"` flag
- [ ] Implement key normalization (suit stripping, Bose-Nelson sort)
- [ ] Implement two-pass trie population algorithm
- [ ] Use [`CactusKevEvaluator`](holdem_core/src/evaluator/cactus_kev.rs) for 5-card hand ranking during generation
- [ ] Implement serialization to binary file with checksum
- [ ] Add CLI command or example to run the generator
- [ ] Validate generated table against all C(52,7) = 133,784,560 combinations

### Phase 3: Implement Supersonic Runtime Evaluator

- [ ] Create `supersonic.rs` with `SupersonicEvaluator` struct
- [ ] Implement file loading with checksum verification
- [ ] Implement chain lookup for 5/6/7-card evaluation
- [ ] Implement hand rank decoding (Supersonic format → existing `HandValue`)
- [ ] Add fallback to Cactus Kev if LUT file is missing or corrupt

### Phase 4: Wire Up as Default Evaluator

- [ ] Add `Option<SupersonicEvaluator>` to [`JumpTable`](holdem_core/src/evaluator/tables.rs:118)
- [ ] Modify evaluation methods to try Supersonic first, fall back to Cactus Kev
- [ ] Add configuration option to enable/disable Supersonic loading
- [ ] Update [`Evaluator`](holdem_core/src/evaluator/evaluator.rs:591) to attempt loading Supersonic LUT at startup

### Phase 5: Update Tests and Benchmarks

- [ ] Add correctness test: compare Supersonic results against AlgorithmicEvaluator for all hand types
- [ ] Add property test: Supersonic agrees with Cactus Kev for random hands
- [ ] Add benchmark: Supersonic vs Cactus Kev vs Algorithmic throughput
- [ ] Update [`benchmarks.rs`](holdem_core/src/evaluator/benchmarks.rs) with Supersonic benchmark suite
- [ ] Update [`ACHIEVEMENTS.md`](docs/ACHIEVEMENTS.md) when complete

---

## File Impact Summary

| File | Action | Phase |
|------|--------|-------|
| `evaluator/tables.rs` | Add `is_built()`, fix guard, add `Option<SupersonicEvaluator>` | 1, 4 |
| `evaluator/evaluator.rs` | Fix guard conditions | 1 |
| `evaluator/supersonic.rs` | **Create** — runtime evaluator | 3 |
| `evaluator/supersonic_generator.rs` | **Create** — offline generator | 2 |
| `evaluator/mod.rs` | Add new sub-modules | 3 |
| `Cargo.toml` | Add `generator` feature flag | 2 |
| `evaluator/benchmarks.rs` | Add Supersonic benchmarks | 5 |
| `evaluator/property_tests.rs` | Add Supersonic property tests | 5 |

---

## Risks and Mitigations

| Risk | Impact | Mitigation |
|------|--------|------------|
| 124 MB memory footprint | High for embedded/constrained environments | Make Supersonic optional; Cactus Kev remains default |
| Generator produces incorrect table | Silent correctness bugs | Validate against all 133M 7-card combinations |
| Binary file format portability | Cross-platform issues | Use explicit little-endian encoding + version header |
| Startup latency for disk load | ~100ms delay on cold start | Load asynchronously; use Cactus Kev until ready |
| `RefCell` panic in multithreaded context | Runtime crash | Use `RwLock` instead, or make cache per-thread |