Buy or rent a GPU server

Start with the workload, rather than the price of a graphics card. Buying and renting serve different needs, and value depends on the useful work the server completes. This guide provides a comparison process for teams choosing infrastructure for inference, training or compute experiments.

1. Define useful utilisation

Record the model, precision, data volume, concurrent jobs and required response time. Separate steady demand from seasonal peaks and experiments. Average occupied hours do not show their distribution: short bursts and a continuous queue require different capacity reserves.

Run a pilot on comparable equipment. Measure completed jobs or requests at the required quality, rather than GPU utilisation alone. Include data preparation, weight transfers, restarts and maintenance in the schedule. Define the comparison period and a scenario in which demand is lower than expected.

Compare the cost of a useful result over the same planning period.

2. Match the complete configuration

A GPU name is not a complete specification. NVIDIA documents H200 SXM and H200 NVL separately, including their form factors and connection options. Record accelerator count, memory, topology, CPU, RAM, storage, networking and resource allocation in both proposals.

Establish whether the offer covers a whole physical server, individual GPUs or a virtual configuration. Check administrative access, operating-time limits and the ability to change the system image. AMD publishes a ROCm compatibility matrix for hardware and software environments; confirm support for your stack before comparing benchmark results.

3. Separate spending, control and flexibility

Criterion	Purchase	Rental
Planning period	Assess useful life and the eventual disposition of the equipment	Check minimum term, renewal and exit conditions
Utilisation	Idle time remains part of ownership cost	Check idle-time billing and whether capacity can be stopped
Control	Assign responsibility for the site and operations	Access and permitted changes depend on the contract
Changing needs	Allow for upgrades or replacement	Check terms for moving to another configuration
Failure	Plan spare capacity and the repair process	Clarify replacement, recovery and responsibility

Confirm rental flexibility and ownership control in the actual terms.

4. Build the complete cost

For a purchase, include equipment, delivery and installation, site preparation, hosting, electricity, cooling, networking, licences, staff time and maintenance provision. Treat the estimated residual value as a separate assumption, not a guaranteed cash recovery.

For a rental, check the billed period, storage, traffic, backups, licences, setup and administration assistance. Include data migration at entry and exit. Do not compare a single GPU-hour with a complete node: first align the equipment and service scope.

Calculate a base case and a lower-utilisation case. If a small change in demand reverses the decision, a pilot that clarifies the workload is more useful than declaring one option universally cheaper.

5. Turn the comparison into a quotation request

Send the same specification for both options. Ask suppliers to separate included services from extras, describe configuration changes, confirm resource availability for the required period and propose an acceptance process. Agree identical test data and success criteria so that the proposals can be compared by outcome.

Clarify the warranty channel and installer eligibility separately. For example, NVIDIA links DGX H100/H200 warranty conditions to installation by authorised personnel; this does not establish a rule for every server. Request the terms of the chosen manufacturer and supplier. Base the decision on the confirmed scope, rather than assumed service.

Choose the option that supports your demand scenario and has clear operating responsibilities.

Checklist

Before making the decision

The workload profile and a measurable useful outcome are defined.Both options use the same period, including idle time and peaks.GPU, topology, CPU, RAM, storage and networking match.Site, software, operations and data migration are included.Access, recovery, warranty and exit terms are recorded.A quotation and pilot support the terms; unverified assumptions are marked.

Sources and documentation

These guides help you prepare requirements. The exact configuration and terms are set out in the quotation.