top_drives/docs/DEVELOPMENT_SETUP.md

# Development Setup Guide

## Prerequisites

### System Requirements

**Operating System**: Windows 11, macOS 10.15+, or Linux (Ubuntu 18.04+, CentOS 7+)

**Architecture**: x86_64 (Intel/AMD) or ARM64 (Apple Silicon)

**Disk Space**: ~500MB free space (includes Rust toolchain and project data)

**Memory**: 2GB RAM minimum, 4GB recommended

**Network**: Broadband internet connection for downloading dependencies and data

### Required Software

1. **Rust Toolchain** (Latest Stable)
2. **Git** (for version control)
3. **Text Editor/IDE** (VS Code recommended)

## Installation Steps

### Step 1: Install Rust

**Windows:**
```powershell
# Download and run rustup installer from https://rustup.rs/
# Or use Winget:
winget install Rustlang.Rustup
```

**macOS:**
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```

**Linux (Ubuntu/Debian):**
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
sudo apt install build-essential  # Required for compilation
```

**Verify Installation:**
```bash
rustc --version
cargo --version
```

**Expected Output:**
```
rustc 1.82.0 (f6e511eec 2024-10-15)
cargo 1.82.0 (f6e511eec 2024-10-15)
```

### Step 2: Configure Rust

**Add Rust to PATH** (if not automatically added):
```bash
# Add to ~/.bashrc, ~/.zshrc, or system PATH
export PATH="$HOME/.cargo/bin:$PATH"
source ~/.bashrc
```

**Update Rust** (recommended):
```bash
rustup update stable
```

**Add Nightly Toolbox** (optional, for advanced features):
```bash
rustup toolchain install nightly
rustup component add rustfmt --toolchain nightly
rustup component add clippy --toolchain nightly
```

### Step 3: Install Project Dependencies

**Clone the Repository:**
```bash
git clone <repository-url>
cd top_drives
```

**Build the Project:**
```bash
cargo build
```

**Expected Output:**
```
   Compiling top_drives v0.2.0
    Finished dev [unoptimized + debuginfo] target(s) in 1m 30s
```

**Verify Build:**
```bash
cargo check  # Fast compilation check
cargo build --release  # Optimized build
```

## Dependency Analysis

### Core Dependencies

#### 1. `tokio` - Asynchronous Runtime
```toml
tokio = { version = "1.44.1", features = ["full"] }
```

**Purpose**: Provides async/await functionality for concurrent operations

**Features Used**:
- `fs`: File system operations
- `net`: Network I/O
- `rt-multi-thread`: Multi-threaded runtime
- `macros`: `#[tokio::main]` and `#[tokio::test]`
- `sync`: Async primitives (Semaphore, Mutex)
- `task`: Task spawning and management

**Why Needed**:
- Concurrent processing of 1000+ car files
- Rate-limited HTTP downloads
- Non-blocking file I/O operations

**Memory Impact**: ~5-10MB overhead for runtime

#### 2. `reqwest` - HTTP Client
```toml
reqwest = "0.12.15"
```

**Purpose**: HTTP client for downloading car data and track definitions

**Features Used**:
- JSON parsing from HTTP responses
- Custom headers and user agents
- Connection pooling and reuse
- HTTPS support with certificate validation

**Why Needed**:
- Download car database from GitHub
- Fetch individual car timing files from API
- Retrieve track definitions

**Network Behavior**:
- Connection pool: 10-20 concurrent connections
- Timeout: 30 seconds default
- Retry logic: Built-in exponential backoff

#### 3. `serde` - Serialization Framework
```toml
serde = { version = "1.0.210", features = ["derive"] }
serde_json = "1.0.140"
```

**Purpose**: JSON parsing and serialization for data structures

**Features Used**:
- `Deserialize` derive macro for structs
- JSON parsing with `from_str()` and `from_reader()`
- Error handling for malformed JSON
- Custom field renaming with `#[serde(rename = "...")]`

**Why Needed**:
- Parse `cars_repo.json` (car database)
- Parse `my_cars.json` (user collection)
- Parse individual car timing files
- Parse `tracks_repo.json` (track definitions)

**Performance**: Zero-copy parsing where possible

#### 4. `futures` - Future Utilities
```toml
futures = "0.3.31"
```

**Purpose**: Utilities for working with asynchronous operations

**Features Used**:
- `join_all()`: Wait for multiple futures concurrently
- `Future` trait implementations
- Stream processing utilities

**Why Needed**:
- Coordinate multiple async file downloads
- Process multiple car timing files concurrently
- Handle collections of futures

#### 5. `once_cell` - Lazy Initialization
```toml
once_cell = "1.20.2"
```

**Purpose**: Thread-safe lazy initialization of global statics

**Features Used**:
- `Lazy`: One-time initialization of expensive resources
- Thread-safe access patterns
- Interior mutability with `Mutex`

**Why Needed**:
- Global `BEST_CARS` cache initialization
- Thread-safe access during concurrent parsing
- Avoid expensive computation on every access

#### 6. `scraper` - HTML Parsing (Club Module Only)
```toml
scraper = "0.23.1"
```

**Purpose**: HTML parsing and CSS selector engine for web scraping

**Features Used**:
- CSS selector parsing with `Selector::parse()`
- HTML document parsing with `Html::parse_document()`
- Element selection and text extraction

**Why Needed**:
- Alternative data source for club.rs module
- Parse car data from topdrives.club HTML tables
- Extract performance information from web pages

### Development Dependencies

#### 7. Build Tools and Quality

**Recommended additions for development:**
```toml
[dev-dependencies]
tokio-test = "0.4"
criterion = "0.5"  # Benchmarking

[[bench]]
name = "parsing_benchmark"
harness = false
```

## Development Environment Setup

### VS Code Configuration

**Recommended Extensions:**
1. **rust-analyzer**: Rust language support
2. **Better TOML**: TOML file support
3. **crates**: Rust package search
4. **Rust Test Explorer**: Test discovery and execution

**Settings** (`settings.json`):
```json
{
    "rust-analyzer.check.command": "clippy",
    "rust-analyzer.cargo.loadOutDirsFromCheck": true,
    "rust-analyzer.procMacro.enable": true,
    "files.associations": {
        "*.rs": "rust"
    }
}
```

### Debugging Setup

**VS Code Launch Configuration** (`.vscode/launch.json`):
```json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug Records",
            "type": "lldb",
            "request": "launch",
            "program": "${workspaceFolder}/target/debug/records",
            "args": ["best"],
            "cwd": "${workspaceFolder}"
        }
    ]
}
```

## Build Configuration

### Cargo.toml Analysis

**Project Structure:**
```toml
[package]
name = "top_drives"
version = "0.2.0"
edition = "2021"

[[bin]]
name = "club"
path = "src/club.rs"

[[bin]]
name = "records"
path = "src/records.rs"
```

**Binary Configuration:**
- **records**: Main analysis tool (JSON-based)
- **club**: Alternative scraper (web-based)

**Dependency Versions:**
- All dependencies pinned to specific versions for reproducibility
- Compatible versions tested together
- Security updates monitored

## Performance Optimization

### Compilation Settings

**Debug Build:**
```bash
cargo build              # Fast compilation, slow execution
```

**Release Build:**
```bash
cargo build --release    # Slow compilation, fast execution
```

**Profile Configuration** (Optional `Cargo.toml`):
```toml
[profile.release]
opt-level = 3
lto = true
codegen-units = 1
panic = "abort"
```

### Runtime Performance

**Memory Settings:**
```bash
# Increase stack size for deep recursion (if needed)
RUST_MIN_STACK=8388608 cargo run

# Enable backtraces for debugging
RUST_BACKTRACE=1 cargo run
```

**Concurrency Tuning:**
- Default: 20 concurrent downloads (configurable via Semaphore)
- Thread count: Automatic based on CPU cores
- Memory usage: ~100-200MB for full dataset

## Testing Setup

### Running Tests
```bash
cargo test              # Run all tests
cargo test --release    # Run tests on release build
cargo test -- --nocapture  # Show test output
```

### Writing Tests

**Example Test Structure:**
```rust
#[cfg(test)]
mod tests {
    use super::*;

    #[tokio::test]
    async fn test_parse_car_times() {
        // Test implementation
    }
}
```

**Test Data:**
- Create sample JSON files in `tests/fixtures/`
- Mock HTTP responses for network tests
- Use `tempfile` crate for temporary files

## Troubleshooting

### Common Installation Issues

**"rustc not found"**
- **Cause**: Rust not in PATH
- **Solution**: Add `$HOME/.cargo/bin` to PATH

**"Permission denied" (Unix)**
- **Cause**: Insufficient permissions
- **Solution**: Check file permissions or use `sudo` for system-wide install

**"Linker not found" (Linux)**
- **Cause**: Missing build tools
- **Solution**: Install `build-essential` (Ubuntu) or `gcc` (CentOS)

### Build Issues

**"Package not found"**
- **Cause**: Missing dependencies
- **Solution**: Run `cargo fetch` or `cargo build` with internet

**"Conflicting versions"**
- **Cause**: Dependency version conflicts
- **Solution**: Update Cargo.lock with `cargo update`

**"Compilation failed"**
- **Cause**: Syntax or type errors
- **Solution**: Check error messages, fix code issues

### Runtime Issues

**"Thread panicked"**
- **Cause**: Unhandled error conditions
- **Solution**: Add proper error handling, check unwrap() calls

**"Out of memory"**
- **Cause**: Large dataset processing
- **Solution**: Process data in batches, increase system memory

## Development Workflow

### 1. Code Changes
```bash
# Make changes to source files
code src/records.rs

# Check compilation
cargo check

# Run tests
cargo test

# Format code
cargo fmt

# Lint code
cargo clippy
```

### 2. Testing Changes
```bash
# Test specific functionality
cargo test parse_car_times

# Benchmark performance
cargo bench

# Check for memory leaks
valgrind cargo run
```

### 3. Debugging
```bash
# Debug build with symbols
cargo build

# Run with debugger
gdb target/debug/records

# Profile performance
cargo flamegraph --bin records -- best
```

## Deployment

### Binary Distribution

**Release Build:**
```bash
cargo build --release
```

**Binary Location:**
```
target/release/records    # Main analyzer
target/release/club       # Club scraper
```

**Cross-Platform Building:**
```bash
# Build for Windows from Linux/macOS
cargo build --target x86_64-pc-windows-gnu --release

# Build for macOS from Linux/Windows
cargo build --target x86_64-apple-darwin --release
```

## Maintenance

### Regular Tasks

**Update Dependencies:**
```bash
cargo update           # Update to latest compatible versions
cargo outdated         # Check for outdated packages
```

**Security Updates:**
```bash
cargo audit           # Check for security vulnerabilities
cargo update          # Apply security patches
```

**Performance Monitoring:**
```bash
cargo bloat           # Check binary size
cargo profdata        # Profile-guided optimization
```

## Getting Help

### Documentation
- **Rust Book**: https://doc.rust-lang.org/book/
- **Cargo Guide**: https://doc.rust-lang.org/cargo/
- **Tokio Tutorial**: https://tokio.rs/tokio/tutorial

### Community Resources
- **Rust Users Forum**: https://users.rust-lang.org/
- **Stack Overflow**: Tag `rust`
- **Discord**: Rust community servers

### Project-Specific Help
- Check existing documentation in `docs/` directory
- Review issue tracker for known problems
- Create new issues for bugs or feature requests

---

**Development Environment Summary:**
- **Rust**: 1.82.0+ (latest stable recommended)
- **Build Tool**: Cargo (included with Rust)
- **Editor**: VS Code with rust-analyzer (recommended)
- **Testing**: Built-in Rust testing framework
- **Debugging**: GDB/LLDB integration available