Incorporating a Data Center GPU into a Gaming PC for £200: Achieving 32GB VRAM with the V100 SXM2

Local LLM Enthusiasts Confront the VRAM

Barrier When attempting to run large language models (LLMs) locally, many developers face the challenge of limited VRAM (video memory). The article’s author, Oscar Molnar, was no exception. Despite already owning an NVIDIA RTX 4080 with 16GB of VRAM—sufficient for gaming purposes—he found this capacity inadequate for running larger models locally. The next logical step was either to purchase a more expensive consumer GPU with higher VRAM or to find an entirely different solution. Molnar chose the latter. His unconventional solution was to acquire a data center GPU at a low cost and integrate it into his custom-built PC.

The Legacy of Data Centers:

What is the Tesla V100 SXM2? Molnar set his sights on NVIDIA’s Tesla V100 SXM2 16GB, a GPU originally designed for use in NVIDIA DGX servers and hyperscaler racks. The SXM2 form factor, however, cannot be directly connected to standard PCIe slots. It also lacks display output ports and standard power connectors, as it was designed to communicate via NVLink on dedicated server rack boards. In other words, this GPU cannot be plugged into a standard motherboard as-is. However, it’s important to note that the V100 is built on NVIDIA’s Volta architecture, equipped with 16GB of HBM2 memory and 5,120 CUDA cores. Incredibly, Molnar was able to purchase this GPU for just £150 (approximately ¥28,000) on eBay. Its computing power and VRAM are genuine, and the memory bandwidth yields remarkable results.

The Superiority of HBM2:

Outperforming Modern GPUs in Bandwidth The true strength of the V100 lies in its memory bandwidth. HBM2 (High Bandwidth Memory 2) is a class of memory technology that is fundamentally different from GDDR. The V100 features a 4096-bit memory bus, offering a bandwidth of 900 GB/s. By comparison, the RTX 4080, released in 2022, offers 736 GB/s with its GDDR6X memory. This means the V100, released in 2017, still boasts 22% higher memory bandwidth than a consumer GPU launched five years later. The comparison becomes even starker when looking at Apple’s products. The M3 Max offers 400 GB/s, the M4 Max provides 546 GB/s, and even the latest M5 Max only reaches 614 GB/s. The V100, a data center GPU from 2017, outperforms even these laptop chips, which cost over £3,000 (approximately ¥560,000). In AMD’s lineup, the RX 7900 XTX, a competitor to the RTX 4080, offers a memory bandwidth of 960 GB/s, slightly edging out the V100. However, it is priced over £700 (approximately ¥130,000), and its ROCm platform for LLM inference is still developing compared to NVIDIA’s CUDA. The V100 provides 94% of the RX 7900 XTX’s bandwidth at less than a quarter of the cost and can run llama.cpp models immediately. The only consumer GPU that clearly surpasses the V100 in memory bandwidth is the RTX 5090, with a bandwidth of 1792 GB/s. However, it costs over £2,000 (approximately ¥370,000). For LLM inference, memory bandwidth is a key factor that determines token generation speed. In this regard, the V100 offers exceptional cost-performance efficiency.

Making the Impossible Possible:

The Adapter The primary challenge was connectivity. NVIDIA does not officially support connecting an SXM2 form factor GPU to a standard PC motherboard. However, third-party “SXM2-to-PCIe adapters” do exist. These unofficial adapters feature an SXM2 socket on one side and a PCIe edge connector on the other, essentially a bare printed circuit board (PCB). Molnar purchased one such adapter for around £50 (approximately ¥9,400). Half-jokingly, he remarked that “half the cost might just be for the copper.” With this setup, he obtained a 16GB VRAM GPU for a total investment of about £200 (approximately ¥38,000), which he could install alongside his RTX 4080 on his motherboard. Combined, the two GPUs provided 32GB of VRAM. In comparison, a single RTX 5090 with 32GB of VRAM costs over £2,000. Molnar emphasized, “This isn’t the same experience, but the amount of VRAM is equivalent.” The critical point is that in LLM inference, VRAM capacity directly limits the size of the model that can be run.

”The Fan from Hell”

and Cooling Challenges The phrase “The fan from hell,” mentioned midway through the article, hints at the practical challenges of running a data center GPU in a desktop environment. These GPUs are designed to operate in server racks with forced air cooling. Keeping them quiet and efficiently cooled in a desktop PC case requires additional ingenuity. Molnar’s article reportedly goes into detail about the noise and cooling solutions he explored. When adapting server GPUs for desktop use, addressing power supply needs, cooling systems, and physical installation are critical engineering hurdles to overcome.

Balancing Cost and Practicality The appeal of

this project lies in its extraordinary cost-performance ratio. With an investment of just £200, Molnar was able to build a system with 32GB of VRAM, capable of running a 27B parameter model at 32 tokens per second. This setup is not only affordable but also delivers practical performance for local LLM developers. However, this approach is not for everyone. The SXM2-to-PCIe adapter is not an official product, and there are risks in terms of compatibility and stability. Data center GPUs lack display outputs, making them less versatile for general gaming PC use. This setup is primarily suitable for specialized tasks, such as LLM inference and machine learning workloads. It is best viewed as an experimental project for users who are comfortable with technical challenges and willing to accept the risks.

Conclusion:

A Second Life for High-Performance Hardware This case study prompts reflection on the lifecycle of hardware in the tech industry. High-performance GPUs retired from data centers often find their way into secondary markets, where they can still provide substantial computing power and memory bandwidth. Molnar’s project demonstrates a creative way for individual developers and researchers to repurpose such “legacy” hardware at a fraction of the cost. While the method lacks official support and requires technical expertise, it showcases the potential for building high-performance computing environments at an exceptionally low cost. As local LLM adoption grows, such innovative hardware utilization cases may become increasingly common.