top_drives/docs/ARCHITECTURE.md

Architecture Documentation

System Overview

The Top Drives Records Analyzer is a Rust CLI tool for analyzing car performance data in the Top Drives racing game. It consists of a shared library (src/lib.rs), a binary entry point (src/records.rs), and three internal module trees: commands/, parsing/, and output/.

Component Architecture

1. Shared Library (src/lib.rs)

Thin re-export module that declares 6 focused sub-modules and re-exports all public types for backward compatibility. External code can use top_drives::Car without knowing the internal module layout.

Sub-modules

Module	File	Key Exports
types	src/types.rs	AppError, BestCarRecord, Car, TripletKey, normalize_to_final_build
context	src/context.rs	AnalysisContext, CarCounts, compute_car_counts
user_data	src/user_data.rs	MyCarsData, load_my_cars_data
translate	src/translate.rs	translate_track_name, translate_track_type
http	src/http.rs	get_file
csv_util	src/csv_util.rs	compare_cars, escape_csv_field
test_utils	src/test_utils.rs	make_record, make_car, empty_my_data (test helpers)

Error Handling

All public functions return Result<T, AppError> where AppError is a unified error type:

• AppError::Io — File system errors
• AppError::Network — HTTP/download errors
• AppError::Parse — JSON parsing errors with file context
• AppError::MissingFile — Required data file not found
• AppError::Internal — Runtime errors (semaphore closed, task join failures)

2. Records Binary (src/records.rs)

CLI entry point that defines command-line arguments and dispatches to command handlers.

Key Responsibilities

• CLI Definition: clap derive-based argument parsing with global path options
• Command Dispatch: Routes to the appropriate command handler in commands/
• Error Reporting: Catches all errors and prints to stderr with exit code 1

Global CLI Options

• --my-cars — Path to user's car collection export (default: my_cars.json)
• --cars-db — Path to car database file (default: cars_repo.json)
• --times-dir — Directory for per-car timing JSON files (default: times)
• --tracks-db — Path to tracks database file (default: tracks_repo.json)
• --output-dir — Directory for output CSV files (default: .)

3. Commands Module (src/commands/)

Command handlers that orchestrate analysis and output generation.

Module	Command	Description
fetch.rs	fetch	Downloads car DB, tracks DB, and per-car timing files
best.rs	best	Generates best.csv and very_best.csv
trash.rs	trash	Generates trash.csv (owned cars not best on any track)
wishlist.rs	wishlist	Generates wishlist.csv, add.csv, remove.csv
details.rs	details	Generates details.csv with all timing data
track.rs	track	Prints best car for a specific track/condition/RQ
improve.rs	improve	Generates improve.csv (owned cars not yet at final build)
all.rs	all	Generates all output files in a single analysis pass

Shared Helpers (src/commands/mod.rs)

• run_analysis(owned, cars_db_path, times_dir) — Common analysis setup: loads car DB, parses timing files, computes counts
• load_my_cars_or_warn(path) — Loads user data with graceful degradation (returns empty data if file missing)

4. Parsing Module (src/parsing/)

Core parsing logic for car timing data and car database loading.

Key Constants

• MAX_RQ: u8 = 113 — Maximum RQ level used for range analysis
• MIN_VALID_TIME: f64 = 1.2 — Minimum valid track time threshold
• TIME_TOLERANCE: f64 = 0.005 — Tolerance for tie detection (5ms)
• TEST_BOWL_FACTOR: f64 = 2000.0 — Multiplier for Test Bowl time inversion
• CONCURRENCY_LIMIT: usize = 20 — Maximum concurrent tasks

Key Functions

• parse_car_times() — Synchronous per-car timing file parser (runs inside spawn_blocking)
• update_best_car() — Updates DashMap with new performance records across RQ range
• parse_all_times() — Orchestrates concurrent parsing with semaphore + spawn_blocking
• fetch_times() — Downloads per-car timing files with bounded concurrency
• get_car_list() (car_list.rs) — Loads car database via serde deserialization

5. Output Module (src/output/)

CSV output utilities and sorting helpers.

• atomic_write() — Writes to a temporary file first, then renames on success (no partial files)
• write_sorted_csv() — Generic sorted CSV writer (index-based sorting to avoid cloning, uses atomic_write)
• write_best_csv(), write_very_best_csv() — Best car CSV output
• write_trash_csv() — Trash car CSV output
• write_details_csv() — Comprehensive timing data CSV output
• write_wishlist_files() — Wishlist, add, and remove CSV output
• write_improve_csv() — Improve car CSV output (owned cars not yet at final build)
• print_best_car() — Stdout output for track-specific query
• compute_ideal_wishlist() — Ideal wishlist computation
• compare_records() — Sorting comparator for BestCarRecord (by RQ, normalized config, brand, model)

Data Flow

GitHub Repository / API
        ↓
cars_repo.json + tracks_repo.json  ←  fetch command
        ↓
times/*.json (1000+ car files)     ←  fetch command
        ↓
parse_all_times() → parse_car_times() × N cars (concurrent via spawn_blocking)
        ↓
AnalysisContext (DashMap<TripletKey, Vec<BestCarRecord>>)
        ↓
compute_car_counts() → CarCounts
        ↓
User Collection (my_cars.json) → owned/wishlist sets
        ↓
Output: CSV files / Terminal Results

Data Structures

Core Types

TripletKey

type TripletKey = (String, u8, String);
//   ↓           ↓        ↓
// Track ID    RQ Level  Surface Condition

Purpose: Composite key for uniquely identifying track conditions Usage: DashMap key for storing best car records

Car

#[derive(Debug, Clone, Deserialize)]
struct Car {
    rid: String,        // Repository ID
    brand: String,      // Manufacturer
    only_name: String,  // Model name (serde rename: "onlyName")
    year: u16,          // Model year
    rq: u8,             // Rarity level
    prize: bool,        // Prize car status (serde default)
    guid: String,       // Game GUID
}

BestCarRecord

struct BestCarRecord {
    car_rid: String,           // Car identifier
    brand: String,             // Manufacturer
    only_name: String,         // Model name
    year: u16,                 // Year
    rq: u8,                    // RQ level
    config: String,            // Original tune config from JSON (111, 221, 233, etc.)
    normalized_config: String, // Normalized final build (323, 233, or 332)
    time: f64,                 // Track time (seconds)
    owned: bool,               // User ownership status
    prize: bool,               // Prize car status
    guid: String,              // Game identifier
}

State Management

AnalysisContext

struct AnalysisContext {
    best_cars: Arc<DashMap<TripletKey, Vec<BestCarRecord>>>,
}

Thread Safety: Uses DashMap for sharded internal locking, providing high-concurrency access without a global mutex. Lifetime: Created in command handlers, passed through the call chain. Access: Direct DashMap methods (insert, get, iter, entry).

CarCounts

struct CarCounts {
    by_rid: HashMap<(String, String), (BestCarRecord, usize)>,
    guid_to_rid: HashMap<String, HashSet<(String, String)>>,
    car_map: HashMap<String, Car>,
    car_list: Vec<Car>,
    owned_configs: HashMap<String, String>,
}

Purpose: Aggregated per-car win counts computed once from AnalysisContext Usage: Shared across all output functions to avoid redundant computation Key grouping: by_rid is keyed by (car_rid, normalized_config) so different tune configs of the same car are tracked separately.

Algorithm Details

RQ Range Analysis

The system analyzes car performance across multiple RQ brackets:

1. Base RQ: Car's natural rarity level
2. RQ Range: Tests from car's RQ up to MAX_RQ (113)
3. Cross-RQ Competition: Allows cars to compete above their base level

Example: A RQ 85 car might perform well at RQ 90-95 levels against higher-rated cars

Time Validation

Multiple validation layers ensure data quality:

1. Minimum Time: Times below MIN_VALID_TIME (1.2s) are discarded
2. Downvote Filter: Community-downvoted times are excluded
3. Tie Detection: Times within TIME_TOLERANCE (5ms) are considered equal
4. Special Cases: Test Bowl track uses inverted time calculation (TEST_BOWL_FACTOR / time)
5. Track Name Splitting: Uses rsplit_once("_a") to correctly handle track names containing _a (e.g., airplaneHangars)

Concurrent Processing

Bounded Concurrency Pattern

• Semaphore First: Semaphore permit acquired in async context (avoids deadlocking the blocking thread pool)
• Blocking Work: Synchronous parsing runs inside tokio::task::spawn_blocking while holding the permit
• Concurrency Limit: Maximum CONCURRENCY_LIMIT (20) simultaneous operations

Synchronization

• DashMap: Sharded concurrent HashMap — no global lock needed for parallel parsing
• Join Handles: Waits for all parsing tasks to complete
• Error Collection: Parse errors are counted and reported; all-cars-failure propagates as an error

File Organization

Input Files

├── cars_repo.json          # Car database from repository
├── tracks_repo.json        # Track definitions
├── track_names.json        # Track code → name mappings (compiled into binary)
├── my_cars.json            # User's car collection (game export)
└── times/                  # Individual car timing files
    ├── Ferrari_F40_1987.json
    ├── McLaren_F1_1993.json
    └── ...

Output Files

├── best.csv              # All cars best on at least one track
├── very_best.csv         # Cars best on ≥ min_count tracks
├── details.csv           # Detailed timing data for all combinations
├── trash.csv             # Owned cars safe to remove
├── wishlist.csv          # Ideal wishlist (very_best unowned, capped)
├── add.csv               # Cars to add to wishlist
├── remove.csv            # Cars to remove from wishlist
└── improve.csv           # Owned cars not yet at final build config

Performance Characteristics

• Startup Time: Fast (uses cached JSON files)
• Memory Usage: Moderate (loads all timing data into memory)
• Accuracy: High (comprehensive dataset)
• Update Frequency: Manual refresh required
• Concurrency: Bounded at 20 simultaneous operations via semaphore
• Lock Contention: Minimal (DashMap sharded locking replaces global Mutex)
• Allocation Optimization: update_best_car avoids cloning on the last RQ iteration

Error Handling Strategy

Unified Error Type

All errors are represented as AppError:

• I/O Errors: File read/write failures
• Network Errors: Download failures with descriptive messages
• Parse Errors: JSON parsing failures with file name context
• Missing Files: Specific variant for absent data files
• Internal Errors: Runtime failures (semaphore closed, task join errors)

Graceful Degradation

• Missing User Data: load_my_cars_or_warn() returns empty data with a warning
• Missing Timing Files: Individual car file failures emit warnings and skip
• Network Errors: Fetch errors logged; first error propagated
• All Parse Failure: If every car fails to parse, the error is propagated

User Feedback

• Progress Indication: Shows fetching/download progress
• Error Messages: Clear descriptions via AppError Display impl
• Warning Messages: Non-fatal issues reported via stderr

Extension Points

Adding New Tracks

1. Update tracks_repo.json with new track definitions
2. Add track name mapping in track_names.json
3. Add condition mappings in translate_track_type() if needed

Adding New Commands

1. Create new command handler in src/commands/
2. Add command variant to the Commands enum in src/records.rs
3. Add dispatch case in the run() function
4. Update documentation

Adding New Output Formats

1. Create new output function(s) in src/output/mod.rs
2. Reuse CarCounts and AnalysisContext for data access
3. Add CLI flag for format selection if needed

Security Considerations

Network Access

• HTTPS Only: All external requests use HTTPS
• Rate Limiting: Respects server limits with controlled concurrency
• Atomic Writes: Downloads write to .tmp files first, then rename on success

Local File Access

• Read Only: No modification of game files
• User Data: Only reads exported collection data
• No Persistence: Analysis results are output-only

Future Enhancements

Potential Improvements

1. Caching Layer: Cache parsed results between runs
2. Database Backend: Replace in-memory HashMap with persistent storage
3. Configuration Files: External configuration for analysis parameters
4. GUI Interface: Web-based interface for non-technical users
5. Export Formats: Multiple output formats (JSON, XML, etc.)

Scalability Considerations

1. Pagination: Handle large datasets with pagination
2. Incremental Updates: Only fetch changed data
3. Compression: Compress large timing files
4. Streaming: Process data without loading everything into memory