top_drives/docs/ARCHITECTURE.md

# Architecture Documentation

## System Overview

The Top Drives Records Analyzer consists of a main analysis binary and a shared library working together to provide comprehensive car performance analysis for the Top Drives racing game.

## Component Architecture

### 1. Shared Library (`src/lib.rs`)

**Common types and utilities** shared across the project.

#### Key Responsibilities
- **Data Types**: Defines `Car`, `BestCarRecord`, `AnalysisContext`, `CarCounts`, `MyCarsData`, `AppError`
- **Track Name Translation**: Converts track codes to human-readable names via `translate_track_name()`
- **Track Type Translation**: Converts condition codes to surface descriptions via `translate_track_type()`
- **File Download**: Async file download utility via `get_file()`
- **User Data Loading**: Parses `my_cars.json` via `load_my_cars_data()`
- **Car Comparison**: Sorting comparator via `compare_cars()`
- **Count Aggregation**: Per-car win count computation via `compute_car_counts()`
- **CSV Escaping**: Safe CSV field formatting via `escape_csv_field()`

#### Error Handling
All public functions return `Result<T, AppError>` where `AppError` is a unified error type:
- `AppError::Io` — File system errors
- `AppError::Network` — HTTP/download errors
- `AppError::Parse` — JSON parsing errors with file context
- `AppError::MissingFile` — Required data file not found
- `AppError::Poisoned` — Mutex poisoning from panicked threads

### 2. Records Binary (`src/records.rs`)

**Primary analysis engine** that processes pre-downloaded JSON timing data.

#### Key Responsibilities
- **Data Fetching**: Downloads car and track databases from GitHub repository
- **File Processing**: Parses JSON timing files for each car configuration
- **Performance Analysis**: Compares timing data across RQ ranges and track conditions
- **Collection Management**: Identifies owned cars and optimization opportunities
- **CSV Output**: Writes analysis results to CSV files with proper escaping

#### Key Constants
- `MAX_RQ: u8 = 113` — Maximum RQ level used for range analysis
- `MIN_VALID_TIME: f64 = 1.2` — Minimum valid track time threshold
- `TIME_TOLERANCE: f64 = 0.005` — Tolerance for tie detection (5ms)
- `TEST_BOWL_FACTOR: f64 = 2000.0` — Multiplier for Test Bowl time inversion
- `CONCURRENCY_LIMIT: usize = 20` — Maximum concurrent tasks

#### Data Flow

```
GitHub Repository
       ↓
cars_repo.json + tracks_repo.json
       ↓
times/*.json (1000+ car files)
       ↓
JSON Parsing → BestCarRecord Analysis
       ↓
User Collection (my_cars.json)
       ↓
Output: CSV files / Terminal Results
```

## Data Structures

### Core Types

#### TripletKey
```rust
type TripletKey = (String, u8, String);
//   ↓           ↓        ↓
// Track ID    RQ Level  Surface Condition
```

**Purpose**: Composite key for uniquely identifying track conditions
**Usage**: HashMap key for storing best car records

#### Car
```rust
#[derive(Debug, Clone, Deserialize)]
struct Car {
    rid: String,        // Repository ID
    brand: String,      // Manufacturer
    only_name: String,  // Model name
    year: u16,          // Model year
    rq: u8,             // Rarity level
    prize: bool,        // Prize car status
    guid: String,       // Game GUID
}
```

#### BestCarRecord
```rust
struct BestCarRecord {
    car_rid: String,   // Car identifier
    brand: String,     // Manufacturer
    only_name: String, // Model name
    year: u16,         // Year
    rq: u8,            // RQ level
    config: String,    // Tune config (111, 233, etc.)
    time: f64,         // Track time (seconds)
    owned: bool,       // User ownership status
    prize: bool,       // Prize car status
    guid: String,      // Game identifier
}
```

## State Management

### AnalysisContext
```rust
struct AnalysisContext {
    best_cars: Arc<Mutex<HashMap<TripletKey, Vec<BestCarRecord>>>>,
}
```

**Thread Safety**: Protected by `Mutex` for concurrent access during parsing
**Lifetime**: Created in `main()`, passed through the call chain
**Access**: `lock_best_cars()` method returns `Result` with proper error handling

### CarCounts
```rust
struct CarCounts {
    by_rid: HashMap<String, (BestCarRecord, usize)>,
    guid_to_rid: HashMap<String, String>,
    car_map: HashMap<String, Car>,
    car_list: Vec<Car>,
}
```

**Purpose**: Aggregated per-car win counts computed once from `AnalysisContext`
**Usage**: Shared across all output functions to avoid redundant computation

## Algorithm Details

### RQ Range Analysis

The system analyzes car performance across multiple RQ brackets:

1. **Base RQ**: Car's natural rarity level
2. **RQ Range**: Tests from car's RQ up to `MAX_RQ` (113)
3. **Cross-RQ Competition**: Allows cars to compete above their base level

**Example**: A RQ 85 car might perform well at RQ 90-95 levels against higher-rated cars

### Time Validation

Multiple validation layers ensure data quality:

1. **Minimum Time**: Times below `MIN_VALID_TIME` (1.2s) are discarded
2. **Downvote Filter**: Community-downvoted times are excluded
3. **Tie Detection**: Times within `TIME_TOLERANCE` (5ms) are considered equal
4. **Special Cases**: Test Bowl track uses inverted time calculation (`TEST_BOWL_FACTOR / time`)

### Concurrent Processing

#### Bounded Concurrency Pattern
- **Task Spawning**: One blocking task per car file via `tokio::task::spawn_blocking`
- **Semaphore**: Limits concurrent operations to `CONCURRENCY_LIMIT` (20)
- **Error Isolation**: Individual car failures don't stop batch processing

#### Synchronization
- **Join Handles**: Waits for all parsing tasks to complete
- **Mutex Protection**: Safe concurrent access to shared `AnalysisContext`

## File Organization

### Input Files
```
├── cars_repo.json          # Car database from repository
├── tracks_repo.json        # Track definitions
├── track_names.json        # Track code → name mappings (compiled in)
├── my_cars.json            # User's car collection (game export)
└── times/                  # Individual car timing files
    ├── Ferrari_F40_1987.json
    ├── McLaren_F1_1993.json
    └── ...
```

### Output Files
```
├── best.csv              # All cars best on at least one track
├── very_best.csv         # Cars best on ≥ min_count tracks
├── details.csv           # Detailed timing data for all combinations
├── trash.csv             # Owned cars safe to remove
├── wishlist.csv          # Ideal wishlist (very_best unowned, capped)
├── add.csv               # Cars to add to wishlist
└── remove.csv            # Cars to remove from wishlist
```

## Performance Characteristics

### Records Module
- **Startup Time**: Fast (uses cached JSON files)
- **Memory Usage**: Moderate (loads all timing data into memory)
- **Accuracy**: High (comprehensive dataset)
- **Update Frequency**: Manual refresh required
- **Concurrency**: Bounded at 20 simultaneous operations via semaphore

## Error Handling Strategy

### Unified Error Type
All errors are represented as `AppError`:
- **I/O Errors**: File read/write failures
- **Network Errors**: Download failures with descriptive messages
- **Parse Errors**: JSON parsing failures with file name context
- **Missing Files**: Specific variant for absent data files
- **Poisoned Mutex**: Thread panic detection

### Graceful Degradation
- **Missing User Data**: `load_my_cars_or_warn()` returns empty data with a warning
- **Network Errors**: Fetch errors logged but don't crash
- **Parse Failures**: Individual car file failures emit warnings and skip

### User Feedback
- **Progress Indication**: Shows fetching/download progress
- **Error Messages**: Clear descriptions via `AppError` Display impl
- **Warning Messages**: Non-fatal issues reported via stderr

## Extension Points

### Adding New Tracks
1. Update `tracks_repo.json` with new track definitions
2. Add track name mapping in `track_names.json`
3. Add condition mappings in `translate_track_type()` if needed

### Adding New Analysis Types
1. Create new output function(s) using `write_sorted_csv`
2. Add command variant to the `Commands` enum
3. Add command handler in `main()`
4. Update help text and documentation

### Adding New Output Formats
1. Create new output function(s) alongside existing CSV writers
2. Reuse `CarCounts` for data access
3. Add CLI flag for format selection if needed

## Security Considerations

### Network Access
- **HTTPS Only**: All external requests use HTTPS
- **Rate Limiting**: Respects server limits with controlled concurrency

### Local File Access
- **Read Only**: No modification of game files
- **User Data**: Only reads exported collection data
- **No Persistence**: Analysis results are output-only

## Future Enhancements

### Potential Improvements
1. **Caching Layer**: Cache parsed results between runs
2. **Database Backend**: Replace in-memory HashMap with persistent storage
3. **Configuration Files**: External configuration for analysis parameters
4. **GUI Interface**: Web-based interface for non-technical users
5. **Export Formats**: Multiple output formats (JSON, XML, etc.)
6. **Module Split**: Break `records.rs` into a module tree for better organization
7. **Concurrent HashMap**: Use `DashMap` for finer-grained locking

### Scalability Considerations
1. **Pagination**: Handle large datasets with pagination
2. **Incremental Updates**: Only fetch changed data
3. **Compression**: Compress large timing files
4. **Streaming**: Process data without loading everything into memory