NUMA Explained Why Memory Location Affects Server Performance

NUMA Explained: Why Memory Location Affects Server Performance

A specification sheet tells you how many cores a processor has and how much RAM sits behind it, but it stays silent on something that can matter just as much: where, physically, that memory sits in relation to each core. On a server with more than one processor, this is not a trivial detail. It is the entire subject of NUMA, and understanding it explains a category of performance problems that core counts and clock speeds cannot.

So what is NUMA? It stands for Non-Uniform Memory Access, and the name describes the situation precisely: memory access is not uniform, because some memory sits close to a given processor and some sits further away, and the distance changes how fast that memory can be reached.

๐Ÿ“– The rest of the hardware picture

NUMA is one layer of processor architecture. Read CPU Cores Explained: Cores, Threads, and vCPUs, on the other numbers that define what a processor can actually do.


What NUMA Actually Is

Every processor in a multi-socket server has its own bank of memory wired directly to it. This is the fastest possible path: the processor reads from memory that sits right next to it, on the same physical node. But a dedicated server with two processors has two of these memory banks, one per socket, and each processor can also reach the other processor’s memory when it needs to.

That second path is where the name earns itself. Reaching the other socket’s memory means leaving the local node entirely, crossing an interconnect built specifically for this purpose, Intel calls its version UPI, AMD calls its version Infinity Fabric, and arriving at memory that was never local to begin with. This detour takes measurably longer than staying local, and that gap between local and remote is the entire phenomenon NUMA describes.

Each processor and its directly attached memory form what the industry calls a NUMA node. A dual-socket server has two nodes. A core on node 0 reading data that lives on node 0 pays the local price. The same core reading data that lives on node 1 pays the remote price, every single time, for as long as that data stays where it is.


Why Remote Memory Access Costs More

The gap is not marginal, and it is well documented in the systems research literature.

According to a detailed analysis published in ACM Queue, the practitioner-focused magazine of the Association for Computing Machinery, a remote memory access on modern NUMA hardware takes approximately 30 percent longer than a local one. On older server generations, that gap was far wider, in some cases up to seven times longer. Modern interconnects have narrowed the penalty considerably, but they have not eliminated it, and 30 percent is still a real, measurable cost paid every time a core reaches across sockets instead of staying local.

The same analysis makes a second point worth taking seriously: the penalty gets worse under contention. When many cores on a system are heavily accessing the same memory node at once, the queueing and congestion on that node’s memory controller can push latency far beyond the simple local-versus-remote gap, in the source’s own measurements, well beyond what either figure alone would suggest. A NUMA-unaware workload does not just pay a fixed 30 percent tax. Under real, busy conditions, it can pay considerably more.

Local vs remote memory access, on modern hardware

Local memory accessBaseline
Remote memory access~30% slower

Approximate penalty on modern NUMA hardware, per ACM Queue analysis. Older hardware generations saw gaps up to seven times wider, and contention can widen the gap further still.

Bar chart comparing local and remote memory access on modern NUMA hardware. Local access is the baseline. Remote access takes approximately 30 percent longer.

๐Ÿ“– Where this shows up in practice

NUMA effects are one of several causes behind unexplained slowdowns. Read What Is Server Load and Why Websites Slow Down, on reading the metrics that reveal where a bottleneck actually sits.


NUMA-Aware Software vs Software That Ignores It

Whether the 30 percent penalty ever affects you depends less on the hardware and more on the software running on it.

What NUMA-Aware Software Does

Modern operating system schedulers, hypervisors, and most serious database engines are built with NUMA in mind. They try to keep a given process’s memory allocated on the same node as the cores running that process, and they try to keep that process scheduled on the same cores over time rather than letting it drift between sockets. According to Intel’s own performance analysis documentation, an application that is not assigned, or pinned, to specific sockets and cores gets scheduled onto whichever cores happen to be free at any moment, which frequently means the operating system moves it across sockets over its lifetime. Each time that happens, the memory it already allocated becomes remote memory, and every subsequent access to it pays the NUMA penalty until something moves again.

What Happens Without It

An application that has no NUMA awareness at all, or one that allocates a large, unpredictable memory footprint accessed from unpredictable cores, cannot benefit from local placement even when the hardware and operating system are both willing to provide it. The memory ends up scattered relative to the cores using it, remote access becomes routine rather than occasional, and the aggregate cost compounds across millions of memory operations per second.

This is why two servers with identical processors, identical core counts, and identical RAM can perform noticeably differently under the same workload. The difference is not the hardware. It is whether the software running on it respects the geography of that hardware.


Who Actually Needs to Think About This

NUMA is not a universal concern, and treating it as one would be dishonest. Its relevance concentrates in specific, identifiable workloads.

Large in-memory databases feel it most directly, because a large buffer pool spread across a dataset bigger than what fits neatly on one node all but guarantees some fraction of every query touches remote memory. Virtualisation hosts running many virtual machines feel it too, particularly when a single VM is sized larger than the memory attached to one socket, forcing it to straddle both nodes permanently. High-performance computing and latency-sensitive real-time applications, where every microsecond of memory access time compounds into a measurable result, also belong on this list.

Most standard web applications, by contrast, rarely notice NUMA at all. Their memory footprints are modest, their access patterns are unpredictable in ways that make the penalty statistically small, and the operating system’s default scheduling handles them well without any special configuration. For this majority of workloads, NUMA is a fact about the hardware worth understanding, not a problem requiring active management.

๐Ÿ“– When large in-memory workloads are the point

Large databases are exactly the workload where NUMA-aware configuration pays off. Read How Dedicated Servers Support Large Databases and Big Data, on what these workloads demand from the hardware underneath them.


Single-Socket vs Dual-Socket: Where NUMA Fits the Decision

This is worth stating plainly, because it resolves a question that follows naturally from everything above: a single-socket server has no NUMA question to answer at all. One processor, one memory bank, every access is local, every time. The entire distinction this article describes simply does not exist on that hardware.

A dual-socket server trades that simplicity for more total cores and more total memory bandwidth, both real and often valuable gains. What it adds alongside them is the NUMA consideration: the performance those extra resources deliver depends partly on whether the workload running on them is NUMA-aware enough to keep memory access mostly local. For a workload that is, dual-socket delivers close to its full theoretical potential. For a workload that is not, some of that potential quietly leaks away as remote-access penalties, and the specification sheet never mentions it.

Neither architecture is universally correct. The right one depends on whether the workload benefits from the extra capacity enough to justify managing, or accepting, the NUMA question that comes with it.

Hardware specified to match your workload

Swify dedicated servers run on Intel Xeon Gold processors in single and dual-socket configurations, from a Netherlands data centre with full root access, enterprise SSD and NVMe storage, and 1Gbps unmetered bandwidth. From โ‚ฌ120/month.

โ†’ Explore Swify Dedicated Servers


Frequently Asked Questions

What is NUMA in simple terms?

NUMA, Non-Uniform Memory Access, describes how memory works on a server with more than one processor. Each processor has its own bank of memory attached directly to it, which it can reach quickly. Reaching the other processor’s memory takes longer, because the request has to cross an interconnect between the two. This difference in speed, depending on where the memory physically sits, is what “non-uniform” refers to.

NUMA only exists on multi-socket hardware; a single-socket server has no such distinction to make. Read CPU Cores Explained: Cores, Threads, and vCPUs for the other hardware numbers that shape server performance.


Does NUMA affect single-socket servers?

No. NUMA only exists where there is more than one memory node to be non-uniform between, which requires more than one processor socket. On a single-socket server, every core shares the same memory bank at the same distance, so there is no local-versus-remote distinction and no NUMA penalty to manage.

This is one of the trade-offs worth weighing when choosing between single and dual-socket hardware. Read How to Choose the Right CPU for Your Dedicated Server for the fuller decision framework.


How much slower is remote memory access under NUMA?

On modern hardware, a remote memory access typically takes around 30 percent longer than a local one, according to systems research published in ACM Queue. Older server generations saw a far wider gap, in some cases up to seven times longer. Under heavy contention, when many cores access the same memory node simultaneously, the effective penalty can grow considerably beyond that baseline figure.

This is why identical hardware can perform differently depending on how memory access is distributed. Read What Is Server Load and Why Websites Slow Down for other causes of unexplained performance gaps.


Is NUMA a problem for databases?

It can be, particularly for large in-memory databases whose buffer pool spans more data than fits comfortably on one NUMA node. When that happens, a meaningful share of queries touch remote memory, and the aggregate cost adds up across the high volume of memory operations a database performs. Most modern database engines are built to be NUMA-aware and try to minimise this, but the underlying hardware topology still shapes how well that awareness pays off.

This is one of several hardware factors that matter more as a database grows. Read How Dedicated Servers Support Large Databases and Big Data for the fuller picture.


Can software avoid NUMA penalties?

To a large extent, yes. NUMA-aware software, including most modern operating system schedulers, hypervisors, and database engines, actively tries to keep a process’s memory allocated on the same node as the cores running it, and tries to keep that process running on the same cores over time rather than letting it drift across sockets. This keeps most memory access local rather than remote.

Software with no such awareness, or with unpredictable, scattered memory access patterns, cannot benefit from this even on hardware and an operating system that support it. Read Bare Metal vs Virtualized Servers for how virtualisation adds another layer to this picture.


Should I choose single-socket or dual-socket to avoid NUMA issues?

A single-socket server avoids the NUMA question entirely, since there is only one memory node and no remote-access distinction to worry about. A dual-socket server introduces the NUMA consideration in exchange for more total cores and memory bandwidth, which is a genuine gain for workloads that can use it well.

The right answer depends on whether your workload benefits enough from that extra capacity to justify the added complexity. Read How to Choose the Right CPU for Your Dedicated Server for how to weigh that trade-off for your own workload.