GPUs Are Snowflakes: How Vast.ai Turned Messy Hardware Into a Market
Most of the AI infrastructure conversation is about uniformity: gigawatt campuses, identical H100 fleets, three-year contracts priced to the cent. Vast.ai is built on the opposite idea. Travis Cannell, Vast’s COO, joined Kony on Where Capital Meets Compute to explain why the messy end of the market (barn-hosted servers, idle neocloud nodes, lab capacity nobody is using) might be the most efficient part of it. Vast now serves around 38,000 paying customers from roughly 2,600 independent suppliers, at utilization near 95 percent. Here is how it works, and why it matters beyond its own numbers.
A company that could have died in 2022
Vast launched as a GPU marketplace on Reddit in 2018, started by Travis’s brother Jake out of AI research at Berkeley. Back then, if you needed a GPU, you bought a server. Vast let you rent someone else’s for a few hundred dollars instead, and the early users were researchers, quant traders and Ethereum miners.
Then the Merge arrived. Ethereum stopped running on GPUs, and the team braced for a supply glut and a price collapse. Generative AI took off the same summer and absorbed that capacity. The developers who discovered Vast while experimenting carried the habit into their jobs. Eight years in, Vast shows up in the training data of the models those developers now ask for compute recommendations. That is a distribution advantage no marketing budget can buy.
What Vast actually sells
Travis dislikes the word “marketplace” because it suggests eBay for hardware. The better description is an operating layer. A GPU owner installs Vast’s daemon on bare metal, and from then on Vast handles virtualization, health checks, benchmarking, billing, payouts and the customer who needs help on a Saturday afternoon.
The owner’s only job is to keep the machine running and set prices for GPU hours, bandwidth and storage, which can be repriced every few seconds through the API. Storage pricing sounds like a detail until you consider the failure mode: price it too low and someone parks terabytes on your machine, leaving the GPU stranded.
Travis compares it to real estate, and the comparison holds. Vast does for GPU servers what property managers did for rental housing: it turns an operating business into an asset a busy person can own. Many of its hosts have day jobs.
Why cheap compute isn’t a race to the bottom
The standard objection to compute marketplaces is margin compression: if everyone competes on price, everyone loses. Travis’s rebuttal is that GPUs are snowflakes. Location, connectivity, security and redundancy all vary, and even identical servers in the same data center differ by around 5 percent in performance because of the silicon lottery and heat dissipation.
His favorite example is a cattle farmer in Nebraska who filled his barn with GPUs because he already had the power. He pulled fiber but never bought a diesel backup generator. His H100 is cheaper than CoreWeave’s because it is a different product.
Neoclouds average out that variation and charge everyone the same price, which means they overprice for some buyers and underprice for others. Vast prices each machine on its own terms and matches it to a buyer whose workload fits. An inference job with redundancy built in can run on the farmer’s box. A training run that can’t afford to die halfway should pay for the data center. The margin comes from making that match, not from the hardware.
It just means that you can price these things and matchmake, and that is where you earn your money.
Availability is the product
Asked why customers choose Vast over AWS, Travis gave three reasons. The first is simplicity: an API key and a credit card, compared with an onboarding flow built for 30-person enterprise teams. The second is price, from a wider range of GPU types and locations. The third matters most right now: availability.
In a shortage, advertised prices stop meaning much. Everyone wants the $2 H100, and nobody has one to sell. Because owners set their own prices on Vast, new supply lists at whatever the market will bear and clears. Buyers anchored on old prices find that frustrating, but it means something is usually available. It also makes Vast one of the few places where you can watch the real spot price of compute form.
The economics
Hosts receive exactly what they set. Vast adds a markup, currently a little under 25 percent, which covers support, payment processing and engineering. Every provider gets the same terms. When you have thousands of suppliers, standard terms are how you earn their trust.
Who’s buying
Almost half of demand is still classified as research: engineers renting a couple of B200s, running a job and leaving without ever talking to sales. The fastest-growing segment is startups building specialized models for weather, robotics and physics. They test on a single node before committing to a cluster.
That pattern led Vast into brokerage. It has placed almost a dozen full clusters by using its supplier network to price-shop offtake contracts. This is where it starts to matter for capital markets. An offtake contract sourced through Vast is something an emerging neocloud can take to a lender, so a rental platform is quietly becoming part of the credit stack.
Inference is getting more varied. Voice workloads follow daylight hours and need very low latency. Image and video generation are large. Bigger customers reserve their base load and burst onto Vast for peaks they can’t forecast, using an autoscaler that remembers how each machine handled their workload.
Where stranded compute goes
The supply side shows the same pattern. Individuals and businesses buy servers partly for US accelerated depreciation, with no desire to build a sales team. Neoclouds list idle nodes for a few weeks, with no minimum term, and use Vast’s pseudo-anonymity to test prices they couldn’t change on their own websites. Labs resell capacity from contracts they aren’t fully using and take it back when their own teams need it.
Put together, Vast functions as the secondary market for capacity. Any compute that would otherwise sit idle has somewhere to go.
The moat
The moat is a two-sided network effect. A competitor would need to pull 38,000 buyers onto a platform with thin supply, and convince hosts to give up the daemon, the support and 95 percent utilization. Supply is roughly 80/20 today, but Travis expects it to spread out further as people run private models on their own hardware and sell the excess.
Agents as the ideal customer
The most interesting idea in the conversation was the agent as a buyer. An agent’s mind literally runs on compute, so it will trade speed against cost all the time: think faster at a premium, or slower, cheaper and for longer. Vast can’t always tell which API users are agents, but usage patterns are already changing.
If that plays out, the most price-sensitive and least loyal buyer in history will be shopping for compute every few seconds. A venue that is API-first, transparent and built for comparison shopping is where that buyer will go.
Google for computation
Travis summed up the vision in one line:
Google is for information. Vast is for computation.
Google doesn’t make websites, and Vast doesn’t own GPUs. Both organize what other people produce. Travis expects training to start looking more like inference: on demand, spread across mixed hardware and locations. If it does, the split between the two markets narrows, and on-demand venues address much more of the total.
Is it a bubble?
Travis was upfront that rising prices help Vast more than falling ones. But the model is partly self-hedging. If prices drop, efficiency becomes everyone’s priority, and paying only for the reliability you need becomes the selling point. He expects corrections for companies running on venture money without a business model. He doesn’t expect a collapse.
Humans love intelligence and we can manufacture it now, and we’re going to manufacture a lot of it.
As the big labs absorb more capacity, everyone below them looks further down the supply chain. That is where Vast operates, and in a market that keeps pretending compute is uniform, pricing the differences is a durable business.
Listen to the full episode of Where Capital Meets Compute for the complete conversation.