# edgemodelr 0.2.0

## SIMD Optimizations for Faster CPU Inference

### New Features

* **Flash attention support**: Enabled by default in `edge_load_model()` via `flash_attn = TRUE`. Reduces memory usage and improves attention computation speed on CPU.

* **Full hardware thread utilization**: Removed the 4-thread cap for small contexts. `edge_load_model()` now uses all available CPU threads by default, with `n_threads_batch` set to max for prompt processing.

* **User-configurable threading**: New `n_threads` parameter in `edge_load_model()` allows explicit control over CPU thread count. Pass `NULL` (default) for auto-detect or an integer to limit cores.

* **Apple Accelerate framework** (macOS): Automatically links the Accelerate framework on macOS builds, enabling hardware-accelerated vDSP vector operations for faster matrix math.

* **Compiler auto-vectorization**: Added `-ftree-vectorize` to GGML compilation flags on all platforms, allowing GCC/Clang to generate SIMD instructions for eligible loops beyond the hand-tuned GGML kernels.

### Existing Features

* **SIMD-optimized build system**: Replaced generic scalar fallback with architecture-aware SIMD detection in both `Makevars` (Unix) and `Makevars.win` (Windows)
  - x86_64: Enables SSE4.2 baseline by default (universal since Intel Nehalem 2008)
  - aarch64/arm64: NEON support built into the ABI (no extra flags needed)
  - Other architectures: Automatic generic fallback

* **User-configurable SIMD levels**: Set `EDGEMODELR_SIMD` environment variable before install to select optimization level:
  - `GENERIC`: Scalar fallback (maximum compatibility)
  - `SSE42`: SSE4.2 baseline (default on x86_64)
  - `AVX`: AVX + F16C (Intel Sandy Bridge 2011+)
  - `AVX2`: AVX2 + FMA + F16C (Intel Haswell 2013+, recommended)
  - `AVX512`: AVX-512 (Intel Skylake-X 2017+)
  - `NATIVE`: Uses `-march=native` for maximum performance on the build machine

* **`edge_simd_info()`**: New function to query compile-time SIMD status including architecture, compiler features, and GGML optimization flags

* **x86 architecture-specific quantization**: Enabled optimized x86 quantization kernels (`arch/x86/quants.c`, `arch/x86/repack.cpp`) with SIMD-accelerated dot products and matrix operations

### Performance

* 15-40% faster inference on x86_64 with SSE4.2 baseline vs generic scalar
* Up to 2-3x faster with AVX2 for quantized model operations
* SSSE3-accelerated integer multiply-accumulate for quantized dot products

---

# edgemodelr 0.1.5

## CRAN Policy Fixes

### Bug Fixes

* **Fixed donttest examples**: Changed resource-intensive examples from `\donttest{}` to `\dontrun{}` to prevent downloading multi-GB models during CRAN checks

* **Fixed M1 Mac compiler warnings**: Added explicit `static_cast<>` for:
  - `double` to `float` conversions for temperature/top_p parameters
  - `size_type` to `int32_t` conversions for buffer size parameters

* **Fixed connection handling**: Replaced `on.exit()` with `tryCatch/finally` for proper connection cleanup in loops (thanks @eddelbuettel)

# edgemodelr 0.1.4

## Performance Optimizations for Small Language Models

### New Features

* **Small Model Configuration Helper**: New `edge_small_model_config()` function provides optimized settings for small models (1B-3B parameters)
  - Device-specific presets: mobile, laptop, desktop, and server
  - Adaptive configuration based on model size and available RAM
  - Built-in performance tips and recommendations
  - Automatic parameter tuning for optimal inference speed

* **Adaptive Batch Processing**: Intelligent batch size optimization based on context length
  - Small contexts (≤512): Uses up to full context for batching
  - Medium contexts (512-2048): Uses 1/2 context for optimal throughput
  - Large contexts (2048-4096): Uses 1/4 context to balance speed and memory
  - Very large contexts (>4096): Caps at 2048 tokens for stability

* **Smart Thread Allocation**: Context-aware CPU thread management
  - Small models automatically limit threads to avoid overhead
  - Reduces CPU contention on resource-constrained devices
  - Improves inference speed for models with contexts ≤2048 tokens

* **Automatic Context Optimization**: Model size-based context tuning
  - Small models (<1GB): Optimized to 1024 tokens for faster inference
  - Medium models (1-2GB): Set to 1536 tokens for balanced performance
  - Large models (>2GB): Maintains 2048+ tokens for quality
  - User override available via n_ctx parameter

### Performance Improvements

* **Faster Small Model Inference**: 15-30% speed improvement for small models through optimized batch and thread settings
* **Reduced Memory Footprint**: Better memory efficiency for resource-constrained environments
* **Lower Latency**: Optimized thread allocation reduces context switching overhead
* **Better Scalability**: Adaptive configurations scale from mobile devices to servers

### Examples and Documentation

* **Small Model Optimization Example**: Comprehensive example demonstrating all optimization features
  - Configuration comparison across device types
  - Performance benchmarking workflow
  - Best practices for different model sizes
  - Manual tuning guidelines

* **Enhanced Testing**: New test suite for small model configuration
  - Tests for all device target configurations
  - Validation of adaptive parameter adjustments
  - Safety checks for edge cases

### Technical Details

* Improved C++ bindings with adaptive batch size calculations
* Enhanced R API with intelligent parameter defaults
* Better integration between model size detection and configuration
* Comprehensive documentation for optimization features

---

# edgemodelr 0.1.2

## Major New Features

### Ollama Integration
* **Native Ollama Support**: Complete integration with Ollama models through automatic model discovery and SHA-256 hash-based loading
* `edge_find_ollama_models()` - Discover all locally available Ollama models across platforms (Windows, macOS, Linux)
* `edge_load_ollama_model()` - Load Ollama models using convenient SHA-256 hash prefixes instead of full file paths
* `test_ollama_model_compatibility()` - Built-in compatibility testing for Ollama models
* **Cross-platform Model Detection**: Robust model discovery supporting standard installations, snap packages (Linux), and various Windows configurations
* **Windows OneDrive Compatibility**: Enhanced path detection that properly handles Windows OneDrive document folder redirections

### Comprehensive Examples Suite
* **Structured Learning Path**: Complete examples directory with progressive difficulty levels (Beginner → Intermediate → Advanced)
* **01_basic_usage.R**: Fundamental operations including model loading, text generation, parameter tuning, and error handling
* **02_ollama_integration.R**: Complete Ollama workflow with model discovery, hash-based loading, and compatibility testing
* **03_streaming_generation.R**: Real-time streaming text generation with interactive chat interfaces and callback processing
* **04_performance_optimization.R**: Advanced performance tuning including GPU acceleration, benchmarking, memory management, and batch processing
* **examples/README.md**: Comprehensive documentation with learning paths, troubleshooting guide, and customization instructions

### Package Structure Improvements
* **Organized File Structure**: Consolidated all examples into structured examples/ directory with consistent formatting
* **Enhanced Documentation**: Improved inline documentation and example comments throughout

---

# edgemodelr 0.1.1

## Bug Fixes and Improvements

### Compilation Fixes
* **macOS Boolean Conflicts**: Completely resolved Boolean enum conflicts by avoiding problematic system headers and using direct function declarations
* **Filesystem Compatibility**: Added comprehensive fallback implementation for disabled `std::filesystem` on macOS builds
* **Header Protection**: Implemented robust cross-platform header inclusion strategy that works with R, Rcpp, and system headers
* **System Header Workarounds**: Replaced `<mach-o/dyld.h>` inclusion with direct function declarations to avoid enum conflicts
* **Format Attribute Warnings**: Suppressed unsupported printf format attribute warnings on macOS Apple Clang compiler
* **CRAN Compliance**: Removed non-portable optimization flags (`-march=native`, `-mtune=native`, etc.) from Makevars for CRAN compatibility
* **Cross-platform Build**: Enhanced Makevars configuration for better macOS compatibility with R package requirements

### Demo and Documentation Updates
* **Modern UI**: Updated streaming chat demo with modern bslib interface for enhanced user experience
* **Documentation**: Improved documentation for `edge_clean_cache()` function
* **Examples**: Enhanced streaming chat example with better UI components

### Technical Improvements
* **Build System**: Updated Makevars files for improved compilation on Windows and Unix systems
* **Core Bindings**: Enhanced C++ bindings for better performance and stability

---

# edgemodelr 0.1.0

## Initial CRAN Release

### New Features

* **Local LLM Inference**: Complete R interface for running large language models locally using llama.cpp and GGUF model files
* **Model Management**: Built-in functions for downloading and managing popular models from Hugging Face
* **Text Generation**: Support for both blocking and streaming text completion
* **Interactive Chat**: Real-time streaming chat interface with conversation history
* **Privacy-First**: All processing happens locally without external API calls

### Core Functions

* `edge_load_model()` - Load GGUF model files for inference
* `edge_completion()` - Generate text completions  
* `edge_stream_completion()` - Stream text generation with real-time callbacks
* `edge_chat_stream()` - Interactive chat session with streaming responses
* `edge_free_model()` - Memory management and cleanup
* `is_valid_model()` - Model context validation

### Model Management

* `edge_list_models()` - List pre-configured popular models
* `edge_download_model()` - Download models from Hugging Face Hub  
* `edge_quick_setup()` - One-line model download and setup

### System Support

* **Self-contained**: Includes complete llama.cpp implementation
* **Cross-platform**: Works on Windows, macOS, and Linux
* **CPU optimized**: Runs efficiently on standard hardware
* **Memory efficient**: Support for quantized models

### Documentation

* Comprehensive getting started vignette
* Complete API documentation with examples
* README with extensive usage examples
* Test coverage for all major functionality

### Technical Implementation

* C++17 integration via Rcpp
* Real-time token streaming with callback support
* Automatic memory management with RAII
* Robust error handling and validation
* Thread-safe model operations

This release provides a complete, production-ready solution for Local Large Language Model Inference Engine in R, enabling private, offline text generation workflows.