refactor: modular architecture with unified error handling and DashMap concurrency
- Break monolithic records.rs into commands/, output/, parsing/ modules - Add lib.rs with shared types (AppError, Car, BestCarRecord, etc.) - Replace Arc<Mutex<HashMap>> with DashMap for concurrent access - Add unified AppError enum with thiserror - Add configurable CLI paths (--my-cars, --cars-db, --times-dir, --tracks-db) - Add serde deserialization for Car struct - Add CSV escaping for fields with commas/quotes/newlines - Fix track name splitting: split_once -> rsplit_once for _a handling - Add integration tests, proptest property-based tests, and unit tests - Add rustfmt.toml configuration - Update documentation (API, ARCHITECTURE, DATA_STRUCTURES, README)
This commit is contained in:
parent
9d57b18a4e
commit
a9a46ee551
24 changed files with 3541 additions and 1470 deletions
|
|
@ -6,7 +6,29 @@ The Top Drives Records Analyzer consists of a main analysis binary and a shared
|
|||
|
||||
## Component Architecture
|
||||
|
||||
### 1. Records Module (`src/records.rs`)
|
||||
### 1. Shared Library (`src/lib.rs`)
|
||||
|
||||
**Common types and utilities** shared across the project.
|
||||
|
||||
#### Key Responsibilities
|
||||
- **Data Types**: Defines `Car`, `BestCarRecord`, `AnalysisContext`, `CarCounts`, `MyCarsData`, `AppError`
|
||||
- **Track Name Translation**: Converts track codes to human-readable names via `translate_track_name()`
|
||||
- **Track Type Translation**: Converts condition codes to surface descriptions via `translate_track_type()`
|
||||
- **File Download**: Async file download utility via `get_file()`
|
||||
- **User Data Loading**: Parses `my_cars.json` via `load_my_cars_data()`
|
||||
- **Car Comparison**: Sorting comparator via `compare_cars()`
|
||||
- **Count Aggregation**: Per-car win count computation via `compute_car_counts()`
|
||||
- **CSV Escaping**: Safe CSV field formatting via `escape_csv_field()`
|
||||
|
||||
#### Error Handling
|
||||
All public functions return `Result<T, AppError>` where `AppError` is a unified error type:
|
||||
- `AppError::Io` — File system errors
|
||||
- `AppError::Network` — HTTP/download errors
|
||||
- `AppError::Parse` — JSON parsing errors with file context
|
||||
- `AppError::MissingFile` — Required data file not found
|
||||
- `AppError::Poisoned` — Mutex poisoning from panicked threads
|
||||
|
||||
### 2. Records Binary (`src/records.rs`)
|
||||
|
||||
**Primary analysis engine** that processes pre-downloaded JSON timing data.
|
||||
|
||||
|
|
@ -15,9 +37,14 @@ The Top Drives Records Analyzer consists of a main analysis binary and a shared
|
|||
- **File Processing**: Parses JSON timing files for each car configuration
|
||||
- **Performance Analysis**: Compares timing data across RQ ranges and track conditions
|
||||
- **Collection Management**: Identifies owned cars and optimization opportunities
|
||||
- **CSV Output**: Writes analysis results to CSV files with proper escaping
|
||||
|
||||
#### Key Constants
|
||||
- `MAX_RQ: u8 = 113` — Maximum RQ level used for range analysis (replaces former magic number 115)
|
||||
- `MAX_RQ: u8 = 113` — Maximum RQ level used for range analysis
|
||||
- `MIN_VALID_TIME: f64 = 1.2` — Minimum valid track time threshold
|
||||
- `TIME_TOLERANCE: f64 = 0.005` — Tolerance for tie detection (5ms)
|
||||
- `TEST_BOWL_FACTOR: f64 = 2000.0` — Multiplier for Test Bowl time inversion
|
||||
- `CONCURRENCY_LIMIT: usize = 20` — Maximum concurrent tasks
|
||||
|
||||
#### Data Flow
|
||||
|
||||
|
|
@ -32,21 +59,9 @@ JSON Parsing → BestCarRecord Analysis
|
|||
↓
|
||||
User Collection (my_cars.json)
|
||||
↓
|
||||
Output: CSV/Terminal Results
|
||||
Output: CSV files / Terminal Results
|
||||
```
|
||||
|
||||
### 2. Shared Library (`src/lib.rs`)
|
||||
|
||||
**Common utilities** shared across the project.
|
||||
|
||||
#### Key Responsibilities
|
||||
- **Track Name Translation**: Converts track codes to human-readable names via `translate_track_name()`
|
||||
- **Track Type Translation**: Converts condition codes to surface descriptions via `translate_track_type()`
|
||||
|
||||
#### Configuration
|
||||
- Track name mappings are loaded from `track_names.json` at compile time using `include_str!`
|
||||
- This external config file maps track codes (e.g., `"laguna"`) to human-readable names (e.g., `"Laguna Seca"`)
|
||||
|
||||
## Data Structures
|
||||
|
||||
### Core Types
|
||||
|
|
@ -63,14 +78,15 @@ type TripletKey = (String, u8, String);
|
|||
|
||||
#### Car
|
||||
```rust
|
||||
#[derive(Debug, Clone, Deserialize)]
|
||||
struct Car {
|
||||
rid: String, // Repository ID
|
||||
brand: String, // Manufacturer
|
||||
only_name: String, // Model name
|
||||
year: u16, // Model year
|
||||
rq: u8, // Rarity level
|
||||
prize: bool, // Prize car status
|
||||
guid: String, // Game GUID
|
||||
year: u16, // Model year
|
||||
rq: u8, // Rarity level
|
||||
prize: bool, // Prize car status
|
||||
guid: String, // Game GUID
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -80,25 +96,41 @@ struct BestCarRecord {
|
|||
car_rid: String, // Car identifier
|
||||
brand: String, // Manufacturer
|
||||
only_name: String, // Model name
|
||||
year: u16, // Year
|
||||
rq: u8, // RQ level
|
||||
year: u16, // Year
|
||||
rq: u8, // RQ level
|
||||
config: String, // Tune config (111, 233, etc.)
|
||||
time: f64, // Track time (seconds)
|
||||
owned: bool, // User ownership status
|
||||
guid: String, // Game identifier
|
||||
time: f64, // Track time (seconds)
|
||||
owned: bool, // User ownership status
|
||||
prize: bool, // Prize car status
|
||||
guid: String, // Game identifier
|
||||
}
|
||||
```
|
||||
|
||||
## Global State Management
|
||||
## State Management
|
||||
|
||||
### BEST_CARS Cache
|
||||
### AnalysisContext
|
||||
```rust
|
||||
static BEST_CARS: Lazy<Mutex<HashMap<TripletKey, Vec<BestCarRecord>>>>
|
||||
struct AnalysisContext {
|
||||
best_cars: Arc<Mutex<HashMap<TripletKey, Vec<BestCarRecord>>>>,
|
||||
}
|
||||
```
|
||||
|
||||
**Thread Safety**: Protected by `Mutex` for concurrent access during parsing
|
||||
**Initialization**: Uses `once_cell::Lazy` for one-time setup
|
||||
**Structure**: HashMap with composite keys for efficient lookups
|
||||
**Lifetime**: Created in `main()`, passed through the call chain
|
||||
**Access**: `lock_best_cars()` method returns `Result` with proper error handling
|
||||
|
||||
### CarCounts
|
||||
```rust
|
||||
struct CarCounts {
|
||||
by_rid: HashMap<String, (BestCarRecord, usize)>,
|
||||
guid_to_rid: HashMap<String, String>,
|
||||
car_map: HashMap<String, Car>,
|
||||
car_list: Vec<Car>,
|
||||
}
|
||||
```
|
||||
|
||||
**Purpose**: Aggregated per-car win counts computed once from `AnalysisContext`
|
||||
**Usage**: Shared across all output functions to avoid redundant computation
|
||||
|
||||
## Algorithm Details
|
||||
|
||||
|
|
@ -116,20 +148,21 @@ The system analyzes car performance across multiple RQ brackets:
|
|||
|
||||
Multiple validation layers ensure data quality:
|
||||
|
||||
1. **Minimum Time**: Times below 1.2 seconds are discarded
|
||||
1. **Minimum Time**: Times below `MIN_VALID_TIME` (1.2s) are discarded
|
||||
2. **Downvote Filter**: Community-downvoted times are excluded
|
||||
3. **Special Cases**: Test Bowl track uses inverted time calculation (2000.0 / time)
|
||||
3. **Tie Detection**: Times within `TIME_TOLERANCE` (5ms) are considered equal
|
||||
4. **Special Cases**: Test Bowl track uses inverted time calculation (`TEST_BOWL_FACTOR / time`)
|
||||
|
||||
### Concurrent Processing
|
||||
|
||||
#### Bounded Concurrency Pattern
|
||||
- **Task Spawning**: One async task per car file
|
||||
- **Semaphore**: Limits concurrent operations to 20 (bounded concurrency)
|
||||
- **Task Spawning**: One blocking task per car file via `tokio::task::spawn_blocking`
|
||||
- **Semaphore**: Limits concurrent operations to `CONCURRENCY_LIMIT` (20)
|
||||
- **Error Isolation**: Individual car failures don't stop batch processing
|
||||
|
||||
#### Synchronization
|
||||
- **Join Handles**: Waits for all parsing tasks to complete
|
||||
- **Mutex Protection**: Safe concurrent access to shared BEST_CARS cache
|
||||
- **Mutex Protection**: Safe concurrent access to shared `AnalysisContext`
|
||||
|
||||
## File Organization
|
||||
|
||||
|
|
@ -138,8 +171,8 @@ Multiple validation layers ensure data quality:
|
|||
├── cars_repo.json # Car database from repository
|
||||
├── tracks_repo.json # Track definitions
|
||||
├── track_names.json # Track code → name mappings (compiled in)
|
||||
├── my_cars.json # User's car collection (game export)
|
||||
└── times/ # Individual car timing files
|
||||
├── my_cars.json # User's car collection (game export)
|
||||
└── times/ # Individual car timing files
|
||||
├── Ferrari_F40_1987.json
|
||||
├── McLaren_F1_1993.json
|
||||
└── ...
|
||||
|
|
@ -147,14 +180,13 @@ Multiple validation layers ensure data quality:
|
|||
|
||||
### Output Files
|
||||
```
|
||||
├── best.csv # Best cars analysis results
|
||||
├── details.csv # Detailed timing data
|
||||
├── add.csv # Cars to add to wishlist
|
||||
├── remove.csv # Cars to remove from wishlist
|
||||
├── best.csv # All cars best on at least one track
|
||||
├── very_best.csv # Cars best on ≥ min_count tracks
|
||||
├── details.csv # Detailed timing data for all combinations
|
||||
├── trash.csv # Owned cars safe to remove
|
||||
├── sorted.csv # Best cars sorted by count
|
||||
├── very_best.csv # Best cars with count ≥ 10
|
||||
└── wishlist.csv # Missing cars already on wishlist
|
||||
├── wishlist.csv # Ideal wishlist (very_best unowned, capped)
|
||||
├── add.csv # Cars to add to wishlist
|
||||
└── remove.csv # Cars to remove from wishlist
|
||||
```
|
||||
|
||||
## Performance Characteristics
|
||||
|
|
@ -168,33 +200,47 @@ Multiple validation layers ensure data quality:
|
|||
|
||||
## Error Handling Strategy
|
||||
|
||||
### Unified Error Type
|
||||
All errors are represented as `AppError`:
|
||||
- **I/O Errors**: File read/write failures
|
||||
- **Network Errors**: Download failures with descriptive messages
|
||||
- **Parse Errors**: JSON parsing failures with file name context
|
||||
- **Missing Files**: Specific variant for absent data files
|
||||
- **Poisoned Mutex**: Thread panic detection
|
||||
|
||||
### Graceful Degradation
|
||||
- **Missing Files**: Continue with available data
|
||||
- **Network Errors**: Log errors but don't crash
|
||||
- **Parse Failures**: Use default values or skip invalid entries
|
||||
- **Missing User Data**: `load_my_cars_or_warn()` returns empty data with a warning
|
||||
- **Network Errors**: Fetch errors logged but don't crash
|
||||
- **Parse Failures**: Individual car file failures emit warnings and skip
|
||||
|
||||
### User Feedback
|
||||
- **Progress Indication**: Shows fetching/download progress
|
||||
- **Error Messages**: Clear descriptions of what went wrong
|
||||
- **Error Messages**: Clear descriptions via `AppError` Display impl
|
||||
- **Warning Messages**: Non-fatal issues reported via stderr
|
||||
|
||||
## Extension Points
|
||||
|
||||
### Adding New Tracks
|
||||
1. Update `tracks_repo.json` with new track definitions
|
||||
2. Add track name mapping in `track_names.json`
|
||||
3. Add condition mappings in `translate_track_type()`
|
||||
3. Add condition mappings in `translate_track_type()` if needed
|
||||
|
||||
### Adding New Analysis Types
|
||||
1. Create new output functions (similar to `print_best_cars()`)
|
||||
2. Add command parsing in main function
|
||||
3. Update help text and documentation
|
||||
1. Create new output function(s) using `write_sorted_csv`
|
||||
2. Add command variant to the `Commands` enum
|
||||
3. Add command handler in `main()`
|
||||
4. Update help text and documentation
|
||||
|
||||
### Adding New Output Formats
|
||||
1. Create new output function(s) alongside existing CSV writers
|
||||
2. Reuse `CarCounts` for data access
|
||||
3. Add CLI flag for format selection if needed
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Network Access
|
||||
- **HTTPS Only**: All external requests use HTTPS
|
||||
- **Certificate Validation**: Accepts invalid certificates for compatibility
|
||||
- **Rate Limiting**: Respects server limits with controlled concurrency (semaphore limit of 20)
|
||||
- **Rate Limiting**: Respects server limits with controlled concurrency
|
||||
|
||||
### Local File Access
|
||||
- **Read Only**: No modification of game files
|
||||
|
|
@ -209,6 +255,8 @@ Multiple validation layers ensure data quality:
|
|||
3. **Configuration Files**: External configuration for analysis parameters
|
||||
4. **GUI Interface**: Web-based interface for non-technical users
|
||||
5. **Export Formats**: Multiple output formats (JSON, XML, etc.)
|
||||
6. **Module Split**: Break `records.rs` into a module tree for better organization
|
||||
7. **Concurrent HashMap**: Use `DashMap` for finer-grained locking
|
||||
|
||||
### Scalability Considerations
|
||||
1. **Pagination**: Handle large datasets with pagination
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue