bigcache - High-performance concurrent in-memory cache reducing garbage collection impact

Introduction to BigCache

BigCache is a remarkable in-memory caching system designed for speed and efficiency. It caters to environments that need to handle a large number of entries without sacrificing performance. The project, managed by Allegro, is implemented in the Go programming language and requires version 1.12 or newer.

Features and Characteristics

BigCache is noted for several key features that make it stand out:

In-Memory Storage: Keeps all entries on the heap while cleverly avoiding garbage collection, which can often slow down systems. It operates directly on byte slices, necessitating serialization and deserialization of data when interfacing with the cache.
Concurrent Access: Supports fast and concurrent operations, allowing multiple threads to interact with the cache simultaneously without significant performance degradation.
Eviction Policy: Employs an eviction mechanism based on time, where data is removed after a specified duration unless explicitly renewed.

Initializing BigCache

Simple Initialization: For users seeking a straightforward setup, BigCache offers a default configuration that allows quick integration with minimal settings.

import (
	"fmt"
	"context"
	"github.com/allegro/bigcache/v3"
)

cache, _ := bigcache.New(context.Background(), bigcache.DefaultConfig(10 * time.Minute))

cache.Set("my-unique-key", []byte("value"))

entry, _ := cache.Get("my-unique-key")
fmt.Println(string(entry))

Custom Initialization: For scenarios where the cache load is predictable, users can customize initialization. This reduces unnecessary memory allocation, as shown in the sample code. Customize aspects like the number of shards, time windows for entry life and cleanup, and memory usage limits.

Configuration Details

Life and Clean Windows: Configure how long data remains in the cache (LifeWindow) and how frequently BigCache performs cleanup (CleanWindow).

Benchmark Performance

Benchmarking reveals BigCache's efficiency under various operations and workloads. It has demonstrated superior write and read speeds compared to other caching solutions like freecache and map structures, specifically in both single-threaded and parallel scenarios.

Benchmark Tests Results

Write and Read Speed: BigCache exhibited fast operations, with times for setting and getting cache entries being notably quick compared to other tested solutions.
Garbage Collection (GC) Pause Time: In scenarios laden with heavy entry counts, BigCache maintained impressive GC pause times, minimizing disruptions during under-the-hood memory management.

Memory Management

According to the Go language's memory management practices, memory usage may appear exponentially high but is efficiently managed by the runtime environment itself. Memory chunks are allocated and released in a controlled manner to optimize performance and efficiency.

How BigCache Works

BigCache capitalizes on enhancements in Go's GC handling. By using a hash map structure (map[uint64]uint32), it minimizes pointers to entries and aids in reducing GC impact. The entries themselves are stored as byte slices, allowing significant data volume management without burdening the GC.

Comparison with Freecache

While both BigCache and Freecache aim to minimize GC overhead, they do so with different methodologies. Unlike Freecache, which requires pre-determined cache size, BigCache can dynamically allocate extra space when needed, offering more flexibility.

Additional Features

BigCache includes an HTTP server package for deployment convenience, making it adaptable for various network-based caching applications.

Learn More

For a deeper understanding of BigCache's origins and internal workings, interested individuals can read the detailed blog post by Allegro.

Licensing

BigCache is open-source, distributed under the Apache 2.0 license. This allows for broad usage and adaptation in personal and commercial projects. The complete license details are accessible here.

Through its efficient design and rapid performance metrics, BigCache emerges as a compelling choice for developers seeking a powerful caching solution in Go.