QUANTUM SHARMA · SILICON ATLAS
Understand compute groups, SIMT, GDDR and HBM; compare GB202 products and examine what VRAM upgrades or pooling really require.
A GPU organises parallel work into compute groups with schedulers, register files, execution lanes and local data resources. NVIDIA calls a major such group a streaming multiprocessor (SM). A lane performs an operation on part of the data; the surrounding group supplies instructions and state. A marketed “GPU core” count is therefore not a count of complete CPU cores.
Read the primary source: NVIDIA: GPU abstract machine and SIMT ↗
In NVIDIA CUDA, a warp groups 32 software threads. SIMT means single instruction, multiple threads: active threads perform an instruction using their own data. Those threads are scheduled onto shared execution resources; 32 threads does not promise 32 separate CPU-like engines or completion in one clock. Other GPU families use their own group terminology.
A scheduler can issue another ready group while one waits for data. Divergent branches can require different paths with different active threads, reducing useful work on shared lanes. Modern scheduling details vary; the useful question is whether the workload supplies enough ready, suitably organised parallel work.
Read the primary source: NVIDIA: CUDA execution and memory model ↗
A GPU needs a memory hierarchy as much as arithmetic. Registers keep per-thread working values. Local shared-memory resources and caches help reuse data; a wider cache/interconnect and memory controllers serve accesses to device DRAM. In CUDA terminology, “local memory” is a per-thread address space, not a promise of a physical on-chip register.
Read the primary source: NVIDIA: data transfer and memory access ↗
| Memory | Physical arrangement | Remember |
|---|---|---|
| GDDR | Graphics-oriented DRAM packages connected to the processor on a board. | External memory chips are not the GPU execution lanes. |
| HBM | Vertically stacked DRAM dies with dense connections close to the processor in a package. | Stacking and packaging are part of the memory system design. |
Read the primary source: Micron: graphics DRAM ↗ · Micron: HBM structure ↗
A discrete GPU often has dedicated device memory; an integrated GPU can share system DRAM. Address-space convenience does not remove the physical differences in locality, capacity and bandwidth.
Read the primary source: NVIDIA: data transfer and memory access ↗
Rendering a picture involves many related operations across vertices, samples and pixels. GPUs combine programmable execution with specialised graphics hardware. Similar data-parallel structure appears in image processing, simulations and matrix operations. Suitability depends on the algorithm, data movement, numerical formats and available software—not just the number of arithmetic lanes.
Read the primary source: NVIDIA: GPU abstract machine and SIMT ↗
CPUs also have multiple cores and vector instructions; GPUs also execute control flow. A useful design distinction is emphasis: CPU designs often pursue low-latency progress on varied instruction streams, while GPU designs supply high throughput across many work items. A small or dependency-heavy task can lose any GPU advantage to scheduling and transfer overhead.
Read the primary source: NVIDIA: data transfer and memory access ↗
The documented GB202 example specifically compares GeForce RTX 5090 with RTX PRO 6000 Blackwell Workstation Edition. It does not mean that all expensive, data-centre or Blackwell GPUs share this die design.
Read the primary source: NVIDIA: GeForce Blackwell whitepaper, table 1 ↗ · NVIDIA: RTX PRO Blackwell whitepaper, appendix A ↗
| Product | CUDA cores | Memory | Memory interface |
|---|---|---|---|
| GeForce RTX 5090 | 21,760 | 32 GB GDDR7 | 512-bit |
| RTX PRO 6000 Blackwell Workstation | 24,064 | 96 GB GDDR7; advertised ECC | 512-bit |
Read the primary source: NVIDIA: RTX 5090 published specifications ↗ · NVIDIA: RTX PRO 6000 Workstation datasheet ↗
The 96 GB product shows that 32 GB is not an inherent GB202-family capacity ceiling. It does not establish a 96 GB upgrade for a retail 5090 board or firmware. Compare the whole product: enabled units, memory, board design and supported features can differ even with a shared die design.
Read the primary source: NVIDIA: GeForce Blackwell whitepaper, table 1 ↗
A memory controller exposes particular channels and supported device organisations. The board must route fast signals with suitable timing and electrical margins, supply power, remove heat and support the intended memory arrangement. Firmware must train and initialise a supported configuration. Different DRAM density can raise capacity without widening the external data bus, but only within a working, qualified design.
Read the primary source: Micron: GDDR7 and controller compatibility ↗
Economic interpretation: product segmentation can package capacity, enabled hardware, validation and support for different customers. The documents show product differences; they do not establish how much a particular restriction comes from yield, engineering cost, support policy or profit incentives. Engineering constraints and commercial choices can both matter.
Each device has its local memory and controllers. Supported peer-to-peer access can let one GPU access or transfer data to another through a suitable PCIe or NVLink path. Availability depends on the devices, topology, configuration and software. A link by itself does not guarantee one transparent, physically unified pool of local VRAM.
Read the primary source: NVIDIA: multi-GPU systems and peer access ↗
For example, two hypothetical 16 GiB GPUs do not automatically give an unmodified program one local 32 GiB allocation. An application may partition data or a model and communicate between devices; duplication, intermediate buffers and the interconnect then matter. Unified virtual addresses or managed-memory APIs simplify programming, but they do not turn a remote link into the same physical channels as local GDDR.
Read the primary source: NVIDIA: multi-GPU systems and peer access ↗
| Layer | What must be checked |
|---|---|
| Physical | Supported links, routing, bandwidth and memory controllers. |
| System | Driver, operating-system and peer-access support. |
| Application | Where data resides, how it is split, and what must be exchanged. |
Suppose C[i] = A[i] + B[i] for 1,000,000 independent elements, with each value occupying four bytes. A simple one-pass model reads two inputs and writes one output: 12 bytes per element, or 12,000,000 bytes (12 MB) total device-memory traffic. This is a teaching lower-bound model; it ignores caches, transaction overhead and transfers to or from the host.
Start with one block you can explain and test. Keep the inputs, expected outputs and limits in a notebook before expanding the design.
Try the selector or half adder and check every input combination.
Open Logic Lab →See how a checked design connects to layout, process choices and real hardware.
Explore making →Try a transistor, binary addition or CPU instruction model →
