EthanCornell/mini_malloc

A comprehensive memory allocation library implementation featuring multiple levels of sophistication, from basic first-fit allocation to security-enhanced allocators with extensive debugging capabilities.

β˜… 1Forks 0CGitHub β†—Compare

Project website β†—

mallocmemorythread

README

Mini Malloc - Memory Allocator Implementation

A comprehensive memory allocation library implementation featuring multiple levels of sophistication, from basic first-fit allocation to security-enhanced allocators with extensive debugging capabilities.

🎯 Project Overview

This project implements a complete memory allocator ecosystem with three distinct implementation levels:

  • Level 1: Basic first-fit allocator with simple free list management
  • Level 2: Advanced allocator with metadata hardening and multi-threading support
  • Level 3: Security-enhanced allocator with extensive debugging features and corruption detection
  • Bonus: tcmalloc optimization framework for real-world performance tuning

πŸ“‹ Table of Contents

✨ Features

Core Features

  • βœ… Memory Alignment - 8-byte alignment for optimal performance
  • βœ… First-fit Algorithm - Efficient free block allocation
  • βœ… Block Splitting & Coalescing - Memory fragmentation reduction
  • βœ… Double-free Protection - Safety against common errors
  • βœ… Large Block Support - mmap for allocations β‰₯64KB
  • βœ… Thread Safety - Multi-threaded arena-based allocation

Security Features (Level 2+)

  • πŸ”’ Metadata Hardening - Canary values and boundary tags
  • πŸ›‘οΈ Corruption Detection - Runtime validation of block integrity
  • 🏟️ Arena-based Architecture - Striped locking for scalability
  • πŸ” Debugging Support - Comprehensive allocation tracking

Level 3 Security Enhancement

  • πŸ” Advanced Canary Protection - Multiple corruption detection mechanisms
  • 🧡 Re-entrancy Guards - Protection against recursive allocation calls
  • πŸ“Š Extensive Validation - Aggressive block and pointer validation
  • πŸ”„ Safe Coalescing - Protected memory block merging
  • πŸ“ˆ Debug Instrumentation - Detailed allocation tracking and reporting

πŸ—οΈ Implementation Levels

Level 1: Basic Implementation

Target: Simple first-fit allocator with basic metadata tracking

// Core functions
void* my_malloc(size_t size);
void my_free(void* ptr);
void* my_realloc(void* ptr, size_t size);
void* my_calloc(size_t num, size_t size);

Key Features:

  • sbrk/brk system call integration
  • Basic block metadata (size, next, free status)
  • Simple free list management
  • Memory alignment handling
  • Protection against segmentation faults

Files: mini_malloc.c, mini_malloc.h, test_mini_malloc.c

Level 2: Advanced Implementation

Target: Thread-safe allocator with metadata hardening

Additional Features:

  • Multi-arena architecture with striped locking
  • Canary values for corruption detection
  • Boundary tags for overflow protection
  • mmap support for large allocations
  • Advanced coalescing algorithms
  • Debug and validation functions

Challenges:

  • Thread safety without performance degradation
  • Memory corruption detection and recovery
  • Complex metadata management
  • Race condition prevention

Level 3: Security-Enhanced Implementation

Target: Security-focused allocator with extensive debugging features

Security Enhancements:

  • Advanced Canary System: Multiple canary values with XOR encoding
  • Re-entrancy Protection: Guards against recursive malloc calls
  • Aggressive Validation: Comprehensive block integrity checking
  • Safe Memory Operations: Protected coalescing and splitting
  • Corruption Recovery: Automatic detection and handling of corrupted blocks
  • Debug Instrumentation: Extensive logging and allocation tracking

Security Challenges:

  • Lock-free corruption detection
  • Safe recovery from memory corruption
  • Performance impact of extensive validation
  • Thread-safe debugging without deadlocks
  • Cross-platform security features

πŸš€ Quick Start

Basic Compilation

# Level 1 - Basic Implementation
gcc -o basic_malloc mini_malloc.c test_mini_malloc.c -lpthread
./basic_malloc

# Level 2 - With debugging
gcc -DMINI_MALLOC_TESTING -g -O2 -o advanced_malloc mini_malloc.c test_mini_malloc.c -lpthread
./advanced_malloc

# Level 3 - Security-enhanced build
gcc -DMINI_MALLOC_TESTING -g -O2 -o security_malloc mini_malloc.c test_mini_malloc.c -lpthread
./security_malloc

Using the Allocator

#include "mini_malloc.h"

int main() {
    // Initialize for security features (Level 3)
    mini_malloc_enable_debug(1);
    
    // Basic allocation
    void *ptr = my_malloc(1024);
    if (ptr) {
        // Use memory
        memset(ptr, 0, 1024);
        my_free(ptr);
    }
    
    // Check heap integrity
    if (mini_malloc_validate_heap()) {
        printf("Heap is healthy!\n");
    }
    
    // Print debug information
    mini_malloc_debug_print_arenas();
    
    return 0;
}

tcmalloc Optimization (Bonus)

#include "tcmalloc_optimizer.h"

using namespace tcmalloc_optimizer;

int main() {
    auto& optimizer = TcmallocOptimizer::GetInstance();
    
    // Initialize with custom configuration
    optimizer.Initialize(OptimizationProfile::ADAPTIVE);
    
    // Enable adaptive optimization
    optimizer.EnableAdaptiveOptimization();
    
    // Your application code here...
    
    // Get performance metrics
    auto metrics = optimizer.GetMetrics();
    optimizer.PrintReport();
    
    optimizer.Shutdown();
    return 0;
}

πŸ“š API Reference

Core Functions

Function Description Level
my_malloc(size) Allocate memory block 1+
my_free(ptr) Free memory block 1+
my_realloc(ptr, size) Resize memory block 1+
my_calloc(num, size) Allocate zero-initialized array 1+

Debug Functions (Level 2+)

Function Description
mini_malloc_enable_debug(flag) Enable/disable debug output
mini_malloc_validate_heap() Check heap integrity
mini_malloc_debug_print_arenas() Print arena information
mini_malloc_get_global_stats(...) Get allocation statistics

Security Functions (Level 3)

Function Description
validate_block_aggressive(...) Comprehensive block validation
debug_free_call(...) Debug version of free with tracking

tcmalloc Optimizer API

// Initialization
bool Initialize(OptimizationProfile profile);
bool Initialize(const AllocatorConfig& config);

// Configuration
void SetProfile(OptimizationProfile profile);
void SetConfig(const AllocatorConfig& config);
AllocatorConfig GetConfig() const;

// Optimization Control
void EnableAdaptiveOptimization();
void DisableAdaptiveOptimization();
void ForceOptimization();

// Monitoring
PerformanceMetrics GetMetrics();
void PrintReport() const;
BenchmarkResult RunBenchmark(size_t iterations);

πŸ›οΈ Architecture

Level 1 Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚   Application   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ my_malloc/free  β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚   Free List     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚      sbrk       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Level 2+ Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚           Application               β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚        malloc/free API              β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  Arena 0  β”‚  Arena 1  β”‚ ... β”‚Arena Nβ”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€
β”‚ sbrk/heap β”‚           β”‚     β”‚ mmap  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”˜

Level 3 Security Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚           Application               β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚      Re-entrancy Guards             β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚   Validation & Corruption Detection β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚     Canary-Protected Metadata       β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚    Multi-Arena System (8 arenas)    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Small/Med   β”‚    Large Allocations  β”‚
β”‚ (sbrk/brk)  β”‚     (mmap/munmap)     β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ§ͺ Testing

Test Suites

  1. Basic Functionality Tests (test_mini_malloc.c)

    • Allocation and deallocation
    • Memory alignment verification
    • Realloc and calloc functionality
    • Edge case handling
  2. Security Tests (Level 3)

    • Corruption detection validation
    • Double-free protection
    • Re-entrancy guard testing
    • Boundary tag verification
  3. Thread Safety Tests

    • Multi-threaded allocation patterns
    • Arena contention scenarios
    • Race condition detection

Running Tests

# Basic test suite
gcc -o test mini_malloc.c test_mini_malloc.c -lpthread
./test

# Security testing
gcc -DMINI_MALLOC_TESTING -g -o security_test mini_malloc.c test_mini_malloc.c -lpthread
./security_test

# Memory debugging with Valgrind
valgrind --leak-check=full ./test

Test Output Example

πŸ”Ή Test 1: Basic Allocation
  βœ“ Allocated 100 bytes
  βœ“ Allocated 200 bytes
  βœ“ Written to allocated memory: Hello World!
  βœ“ Basic allocation test passed

πŸ”Ή Test 2: Free and Reuse
  βœ“ Freed memory block
  βœ“ Memory was reused (ptr1 == ptr2)
  βœ“ Free and reuse test passed

πŸ”Ή Test 3: Memory Alignment
  βœ“ All allocations aligned to 8 bytes
  βœ“ Alignment test passed

βœ… All tests passed!

βš™οΈ Configuration

Compile-time Configuration

// Enable testing features
#define MINI_MALLOC_TESTING

// Memory alignment (default: 8 bytes)
#define ALIGNMENT 8

// Arena configuration
#define NUM_ARENAS 8

// Canary value for corruption detection
#define CANARY_CONST 0xBADC0FFEE0DDF00DULL

// mmap threshold
#define MMAP_THRESHOLD (64 * 1024)

Runtime Configuration (Level 3)

// Enable debug output
mini_malloc_enable_debug(1);

// Validate heap integrity
if (!mini_malloc_validate_heap()) {
    fprintf(stderr, "Heap corruption detected!\n");
}

Environment Variables

# Enable detailed debugging
export MINI_MALLOC_DEBUG=1

# Set allocation tracking
export MINI_MALLOC_TRACK_ALLOCS=1

πŸ› Troubleshooting

Common Issues

Segmentation Fault

# Enable debug mode
gcc -DMINI_MALLOC_TESTING -g -fsanitize=address -o debug_malloc mini_malloc.c test_mini_malloc.c
gdb ./debug_malloc

Memory Leaks

# Run with Valgrind
valgrind --leak-check=full --show-leak-kinds=all ./mini_malloc

Corruption Detection

# Check debug output for corruption messages
[DEBUG] Canary corruption in free: got deadbeef123456789
[DEBUG] Invalid block pointer 0x7fffe123dead in validate_block

Thread Safety Issues

# Compile with thread sanitizer
gcc -fsanitize=thread -g -o thread_test mini_malloc.c test_mini_malloc.c -lpthread

Debug Output Analysis

=== Mini Malloc Debug Info ===
Arena 0:
  Block 0: size=1024, free=0, mmap=0, addr=0x55d2a5c6b010
  Block 1: size=512, free=1, mmap=0, addr=0x55d2a5c6b420
Mmap Blocks:
  (no mmap blocks)
==============================

πŸ”§ Building and Installation

Prerequisites

# Ubuntu/Debian
sudo apt-get install build-essential

# CentOS/RHEL
sudo yum install gcc glibc-devel

# macOS
xcode-select --install

Build Options

# Debug build
gcc -DMINI_MALLOC_TESTING -g -O0 -o debug_malloc mini_malloc.c test_mini_malloc.c -lpthread

# Release build
gcc -O2 -o release_malloc mini_malloc.c test_mini_malloc.c -lpthread

# Security-enhanced build
gcc -DMINI_MALLOC_TESTING -g -O2 -fstack-protector-all -o security_malloc mini_malloc.c test_mini_malloc.c -lpthread

πŸ“Š Monitoring and Metrics

Built-in Metrics (Level 2+)

// Get global statistics
size_t total_blocks, free_blocks, total_size, free_size, mmap_blocks, mmap_size;
mini_malloc_get_global_stats(&total_blocks, &free_blocks, &total_size, 
                            &free_size, &mmap_blocks, &mmap_size);

printf("Total blocks: %zu, Free blocks: %zu\n", total_blocks, free_blocks);
printf("Total size: %zu bytes, Free size: %zu bytes\n", total_size, free_size);

Security Monitoring (Level 3)

// Validate heap integrity
if (!mini_malloc_validate_heap()) {
    printf("Heap corruption detected!\n");
    mini_malloc_debug_print_arenas();
}

// Check specific arena
uint8_t arena_idx = mini_malloc_get_current_arena_idx();
printf("Current thread using arena: %d\n", arena_idx);

🀝 Contributing

Development Setup

git clone https://github.com/your-repo/mini-malloc.git
cd mini-malloc
gcc -DMINI_MALLOC_TESTING -g -o test mini_malloc.c test_mini_malloc.c -lpthread

Code Style

  • Follow Linux kernel coding style
  • Use meaningful variable names
  • Add comprehensive comments
  • Include unit tests for new features

Submitting Changes

  1. Fork the repository
  2. Create feature branch (git checkout -b feature/security-enhancement)
  3. Commit changes (git commit -m 'Add security feature')
  4. Push to branch (git push origin feature/security-enhancement)
  5. Open a Pull Request

πŸ“„ License

This project is licensed under the MIT License - see the LICENSE file for details.

πŸ™ Acknowledgments

  • Inspired by production allocators like tcmalloc and jemalloc
  • Security concepts from hardened malloc implementations
  • Thread safety techniques from glibc malloc
  • Debugging approaches from development malloc variants

πŸ“– References


Note: This project is designed for educational purposes and includes production-ready security features. The Level 3 implementation focuses on security enhancement and debugging capabilities rather than raw performance optimization.

Contributors

EthanCornell

Issues