- The CPU cache memory (L1, L2, L3 and even L4) drastically reduces latency compared to RAM and is key to real-world performance.
- The cache hierarchy balances capacity and speed: L1 and L2 per core, shared L3, and in some cases L4 as GPU support.
- Latency, bandwidth, and cache hit rate determine performance in games and intensive workloads, along with RAM speed.
- Technologies like AMD's 3D V-Cache expand the L3 cache through 3D stacking, significantly improving gaming performance.
When discussing processors, the focus is almost always on cores, frequency, and manufacturing process, but cache memory and its latency remain largely overlooked despite being key to the actual performance of the system. Understanding what happens between the CPU, cache, and RAM allows us to see why two processors with the same frequency can perform very differently.
In recent years, concepts like cache latency, L1/L2/L3 bandwidth , and technologies such as AMD's 3D V-Cache have become key players in benchmarks, games, and C++ performance tests, where measuring even a few nanoseconds of difference can completely change the outcome. Let's break down everything behind CPU cache latency, why it's so important, and how it's measured in practice.
From the CPU-RAM gap to the birth of cache memory
In the 80s, processor speeds grew much faster than memory speeds . CPUs began executing instructions at a breakneck pace, while RAM access times remained relatively long. The result was a constant bottleneck: the processor spent a significant amount of time waiting for data to arrive.
To address this performance gap, engineers introduced cache memory as an intermediate layer between the CPU and RAM. The idea is simple yet powerful: to store, very close to the processor and in extremely fast memory, the data and instructions that the CPU is most likely to need in the short term. This reduces effective latency and masks the relative slowness of RAM.
In a modern PC we can clearly distinguish three levels of storage: mass storage (HDD, SSD), huge but slow; RAM , much faster but still with significant latency; and the cache memory integrated into the CPU , tiny in capacity, but by far the fastest.
Cache isn't exclusive to the processor: hard drives, SSDs, GPUs, printers, and other devices also have their own internal caches to speed up data access. However, the CPU cache has the most direct impact on everyday user experience.
What is CPU cache memory and how does it work?
The CPU cache is an SRAM memory integrated into the processor chip , capable of operating at incredibly high speeds and with latencies of fractions of a nanosecond. Unlike RAM (DRAM), it doesn't require constant refreshing, is more expensive to manufacture, and has a much smaller capacity; therefore, it's reserved only for very short-term, critical data.
Its primary function is to act as an ultra-fast buffer between the CPU cores and RAM . When a program is launched, its code and data are first loaded from storage into RAM's address space. From there, the CPU's integrated memory controller brings the blocks most likely to be used into the cache, following access patterns and prediction algorithms.
The typical process is as follows: the CPU requests a memory address, first checking the L1 cache , then L2, and finally L3. If the data is found in any of these levels, a cache "hit" occurs, the request is served quickly, and many clock cycles are saved. If it is not found in any of these levels, a cache "miss" occurs, and the data must be retrieved from RAM, which has higher latency and less effective bandwidth for small operations.
The design of this memory hierarchy aims for a balance: small, extremely fast caches close to the core , and increasingly larger but slower levels as they move further away from the execution units. All the performance magic of a modern CPU hinges on maximizing the number of hits and minimizing the cost of misses.
Cache levels: L1, L2, L3 and even L4
Modern processors typically have three levels of cache integrated into the chip itself: L1, L2, and L3 . Some specific models also include an L4 level, usually implemented as a separate chip similar to eDRAM for very specific tasks, such as providing extra resources to the integrated GPU.
Each cache level is characterized by a triangle of factors: physical distance from the core, latency, and capacity. The closer it is to the execution units, the lower the latency, but also the smaller its size, because the cost in transistors increases significantly.
Cache L1: the first line of fire
The L1 cache is closest to the kernel and is the fastest of all the caches . It is usually divided into two distinct blocks: the L1 data cache (L1D) and the L1 instruction cache (L1I). The former stores the operands to be processed, while the latter stores the decoded instructions that the kernel will execute.
Each core has its own L1 cache, not shared with the others, meaning there is no direct L1 coherence between different cores. Typically, we're talking about 32 KB of L1D and 32 KB of L1I per core in many modern architectures, although some designs increase these values. Slightly larger amounts can be found in some high-end processors or servers.
In terms of performance, L1 offers sub-nanosecond latencies and peak read speeds of thousands of GB/s in benchmark tools like AIDA64. For example, a modern Ryzen processor can achieve L1 latency around 0,7 ns with bandwidths exceeding 2.700 GB/s, at the cost of a small size, such as 512 KB total L1 cache across all cores.
L2 Cache: the balance between size and speed
The L2 cache sits just behind L1 as a second layer of backup . Its latency is slightly higher, but still very low compared to RAM. Its function is to store data that doesn't fit in L1 but is still used quite frequently.
In most consumer architectures, each core has its own private Layer 2 cache , which is not divided into data and instructions and typically ranges in size from 256 KB to 1 MB per core, although in the latest generation of servers, sizes of 1 MB per core or more are common. In terms of performance, bandwidths of over 1.300 GB/s with latencies of around 2,7 ns are typical.
Being larger, the L2 layer helps reduce errors that would otherwise reach L3 or RAM if only L1 existed, maintaining a good balance between capacity and latency . In some topologies, multiple cores are grouped into clusters that share a common L2 layer, introducing additional coherence features and internal buses.
L3 or LLC (Last Level Cache)
The L3 cache is also often called LLC (Last Level Cache) because, in most desktop processors, it's the last level of cache before RAM. It's the largest and also the slowest cache within the CPU's main package.
Unlike L1 and L2 caches, L3 cache is typically shared among all cores or large groups of cores (for example, blocks of eight in some AMD designs). This allows any core to reuse data brought in by another, increasing the overall hit rate, although it significantly complicates the consistency logic.
In a mid-range desktop processor, it's not uncommon to see L3 cache ranging from 4 MB to 32 MB, while server CPUs, such as some AMD EPYC series, easily reach hundreds of MB . The price to pay is higher latency, which can be around 10 ns, and lower bandwidth than L1 and L2 cache, although still quite significant: around 900 GB/s in powerful systems.
L4 cache: special cases
The L4 cache is a special case that doesn't appear in most consumer CPUs . It's typically implemented as eDRAM external to the main CPU package, but physically very close on the motherboard, and is used in processors with integrated GPUs to provide an extra boost to graphics bandwidth.
A classic example was the Intel Core i5-5775C, which combined 6 MB of L3 cache with 128 MB of eDRAM used as an L4 cache for its integrated Iris Pro 6200 GPU. This memory acted as a buffer for graphics data, reducing the strain on system RAM and improving gaming performance compared to other iGPUs without this support.
Cache latency and its impact on performance
The key word when we talk about CPU cache is latency: the time it takes the CPU to access data stored in a specific level of memory. This latency is measured in nanoseconds (ns), and although these times may seem ridiculously small, added up to millions of accesses per second, they make a significant difference.
In a typical cache hierarchy, L1 latency is the lowest (less than 1 ns), L2 latency is multiplied by several factors (for example, 2,7 ns), and L3 latency increases again (it can reach 10 ns or more, especially if it's on a separate chip). When there's no hit in any of the caches, the processor must access the RAM, where latencies can skyrocket to several tens of nanoseconds, even with fast DDR5 memory.
A real-world example: with a Ryzen 7 7700X and DDR5 memory at 6.000 MT/s and CL30, we can see average RAM latencies of around 70 ns, compared to 0,7 ns in L1, 2,7 ns in L2, and about 10 ns in L3. This difference explains why it's so important for the CPU to find the data in the cache: each cache miss forces a significantly longer wait.
Latency depends not only on the memory technology used, but also on the physical distance and topology . In older designs where L2 and L3 caches were mounted on the motherboard, much further from the CPU, latency was much higher and performance suffered. Integrating all caches into the CPU package drastically reduced these delays.
In addition to latency, the effective cache bandwidth (measured in GB/s) indicates how much data can be transferred per unit of time. Here, L1 again reigns supreme with read speeds exceeding 2.000 GB/s in synthetic benchmarks, followed by L2 and L3 with more modest values, but still well above RAM.
Hits, cache misses, and cross-core consistency
When the CPU looks for data and finds it in one of the cache levels, we call it a cache hit . If it doesn't find it, a cache miss occurs , and the CPU has to resort to the next level or, in the worst-case scenario, to main memory. The more hits achieved in the higher levels, the better the overall system performance.
Cache misses not only add latency; they can also cause the CPU to have to repeat work cycles or reorder instructions because the data is not yet available, which "breaks" the processor's internal pipeline and reduces the utilization of its functional units.
In multi-core processors, cache coherence also comes into play ; that is, maintaining a consistent view of the data shared between cores that have their own L1 and L2 caches. Complex coherence protocols handle invalidating and updating cache lines when another core modifies data, which adds internal traffic and can affect effective latency.
Internal architecture also plays a role. In monolithic designs like many Intel Core processors, all cores access a single L3 cache with fairly uniform latencies. In chiplet or MCM architectures, such as the first generations of AMD Ryzen, the L3 cache is organized into blocks (CCX, CCD), and some cores can only directly access a portion of the L3 cache, introducing extra latencies when moving from one block to another.
Cache latency versus RAM: frequency, timings and bus
When information is not in the cache, RAM comes into play . Here, two concepts are often confused: latency and frequency. Frequency (typically expressed in MT/s or MHz) indicates how much data can be transferred per second, while latency (in ns or through timings like CL16, CL30, etc.) indicates how long it takes for the first piece of data to be delivered.
RAM with high frequencies but very high latencies can offer good bandwidth but worse response time, which penalizes latency-sensitive processes such as games or certain workloads with non-sequential access. In contrast, modules with lower timings improve first-access speed, even if their raw bandwidth isn't as impressive.
The bus width between the CPU and RAM also matters. On most desktop platforms, each memory channel has a 64-bit bus, and with a dual-channel configuration, this increases to 128 effective bits, doubling the amount of data that can travel simultaneously between the processor and RAM.
The more efficient the combination of frequency, latency, and bus width , the less painful a cache miss will be. Even so, no current RAM can approach the latency and bandwidth figures of L1, L2, or L3 caches, so the priority remains maximizing cache hits.
Scratchpad RAM vs Auto Cache
Inside the processor, we find not only hardware-managed automatic caches. Some designs also include Scratchpad RAM , a type of very fast internal memory that, unlike cache, is not self-managed.
The difference is clear: cache memory transparently replicates lines of data close to the addresses being used, following internal algorithms, and the programmer has no direct control over it. In contrast, a Scratchpad RAM functions as a small local RAM, where the software itself decides what data to store, how to organize it, and when to clear it.
This approach is more common in embedded processors, DSPs, and some GPUs, where highly predictable access patterns are needed and the uncertainty of automatic caching is to be avoided. On desktop PCs and servers, the end user rarely interacts explicitly with a Scratchpad RAM.
Cache latency, performance tests, and curious effects
In the world of benchmarks, it is common to measure cache latencies in nanoseconds and bandwidths in GB/s for each level (L1, L2, L3) using tools such as AIDA64, or by developing your own tests in C++ that perform specific scans of large arrays.
These tests can reveal curious behaviors. For example, when measuring L2 and L3 read bandwidth on a seemingly idle system, it's possible to observe that L2 values sometimes drop from around 80 GB/s to about 60 GB/s , and L3 values can fall from around 45 GB/s to just over 35 GB/s for no apparent reason.
A common explanation is throttling, or dynamic changes in the CPU's internal frequency and power states. However, there are scenarios where, upon opening a monitoring tool like HWINFO, the scores magically return to "good" values. When the tool is closed, the scores fluctuate downwards again.
What usually happens is that programs like HWINFO keep the CPU in higher performance states , forcing more stable frequencies and preventing parts of the inner ring or cache from entering deep sleep modes. If the benchmark is run with nothing else active, the processor tends to conserve power, which can intermittently reduce the measured effective bandwidth, even though average latencies don't change significantly.
In such cases, solutions have been proposed such as maintaining a "keep-alive" thread that does light work to prevent the CPU from going to sleep, but it is not always easy to reproduce the effect of a monitoring tool, because each microarchitecture has its own set of sleep states and internal policies.
Cache memory and gaming performance
In video games, the L3 cache plays a crucial role. Many gaming workloads benefit greatly from being able to store more data and instructions closer to the CPU , from scene structures to physics data and game logic. The fewer RAM accesses, the more stable the FPS and the less frame rate fluctuations.
A processor with a generous and well-utilized L3 cache can offer significant improvements in frame rate stability, especially in modern games that handle a large number of objects and perform many repetitive queries to data structures. Instead of constantly accessing RAM (which is much slower and has higher latency), the CPU can serve most requests directly from its L3 cache.
However, cache doesn't work miracles. A CPU with few cores, low IPC, and modest frequencies will still be limited, even with massive amounts of L3 cache. The winning combination for gaming is usually high IPC, good frequency, ample cache, and fast RAM with low latency , so that the whole package minimizes bottlenecks.
When a demanding game is launched, if measured in parallel with a benchmark like AIDA64, you can see how cache accesses spike . Many of the micro-stutters noticeable in some titles are due to cache misses that force the system to access RAM, or to thread changes that break the established data locality.
AMD's 3D V-Cache and the race for more L3
To break the physical limit of what can be integrated into an L3 cache on a single flat piece of silicon, AMD introduced its 3D V-Cache technology , which involves vertically stacking an additional L3 cache chip on top of a processor CCD (chiplet). This greatly expands L3 cache capacity without increasing the die's base size.
Ryzen processors with the X3D suffix, such as the Ryzen 7 7800X3D or the 7950X3D, take this idea to the extreme. The 7800X3D, for example, combines 32 MB of L3 cache in the base CCD with another 64 MB stacked, totaling 96 MB of L3 cache accessible by that set of cores. The 7950X3D goes even further with 128 MB of L3 cache, in addition to 1 MB of L1 cache and 16 MB of L2 cache distributed among all its cores.
The impact on gaming is enormous because cache hit rates skyrocket , reducing the need to access RAM for many structures that previously couldn't fit entirely in the L3 cache. This explains why these X3D models have become the benchmark for pure gaming performance, even compared to CPUs with more cores or slightly higher clock speeds.
For now, its main competitor, Intel, hasn't replicated an identical solution. The company has indicated that, for the time being, it prefers to focus on balanced overall performance rather than a gaming-oriented approach like large blocks of stacked L3 cache. This doesn't preclude it from gradually increasing the size of its own caches in future generations, but it doesn't appear likely to adopt an exactly the same V-Cache scheme in the short term.
How to see how much cache your processor has
If you want to know what L1, L2, and L3 cache your CPU has without going crazy searching for technical documentation, one of the quickest ways is to use tools like CPU-Z in Windows. This free program displays, in dedicated tabs, the sizes of each cache level, as well as the number of cores, threads, and other relevant data.
Another option is to visit the manufacturer's official website (Intel ARK, AMD product sheets, etc.), although often only the L3 cache is clearly detailed, and you have to dig through technical documents to find the L1 and L2 caches. Reviews and analyses from specialized media outlets also usually break down this information, along with latency and bandwidth benchmarks.
Choosing and making good use of RAM and cache memory
The cache can mask many of the shortcomings of RAM, but not all. When building a computer, it's important to choose RAM with the right frequency, latency, and capacity for its intended use. A computer for office work or light browsing can function well with 8-16 GB and mid-range modules, while for gaming and video editing, 16 GB or 32 GB is usually recommended.
It's always advisable to check the RAM's compatibility with the motherboard and CPU (type DDR3, DDR4, DDR5, maximum supported speeds, XMP/EXPO profiles, etc.). It's also a good idea to find a balance between high frequency and reasonable timings, rather than focusing solely on a flashy MHz number.
To get the most out of memory and cache, it helps a lot to keep your system free of unnecessary background programs , update your BIOS and drivers, and, on heavily used computers, delete temporary files and browser caches that no longer serve a purpose. Anything that reduces the strain on RAM and CPU helps free up those fast resources for what really matters.
At the BIOS level, enabling automatic memory profiles (XMP, DOCP, EXPO) is usually a simple way to ensure that the RAM works at the speed promised by the manufacturer, avoiding being stuck on conservative values that limit bandwidth.
Understanding how cache latency works, its L1/L2/L3 hierarchy, its relationship with RAM, and technologies like 3D V-Cache allows you to read CPU specifications and benchmark results in a new light: beyond GHz and the number of cores, much of the system's fluidity is decided in those few nanoseconds it takes the processor to find or not find data in its nearest memory.