The ZML inference server LLMD has arrived as a free product, promising to run open-source large language models at full speed across a sprawling range of chips: Nvidia, AMD, Google’s TPU, Apple Metal, and Intel Arc.
The Paris-based startup, founded by Steeve Morin and endorsed by Turing Award winner Yann LeCun, is aiming squarely at one of enterprise AI’s most persistent frustrations: the software lock-in that forces companies to build for one chip vendor and stay there.
What the ZML Inference Server Actually Does
Inference, the process of handling live prompts once a model is trained, has quietly overtaken training as the pressing performance problem for most AI deployments. The bottleneck is rarely the model itself; it is the patchwork of software and architecture constraints that prevent companies from using whatever hardware is cheapest or most available.
LLMD is ZML’s answer to that. ‘The idea is to give people back the power to create their own system and achieve real efficiency gains that allow [AI] to be disseminated,’ Morin told TechCrunch.
Unlike ZML’s earlier ML framework, released in 2024 and updated in March, LLMD is not open source. It is, however, free at launch, with the company using the period to measure real-world usage before deciding where to charge. ‘I’d rather measure and [then generate revenue] where it is most effective without hindering my growth stupidly because I have been too greedy from the get-go,’ Morin said.
The startup counts a team of just 20 people, which Morin credits as a structural advantage: fewer dependencies, faster releases. ZML has raised $20 million from a roster that includes Harry Stebbings’ 20VC, Xavier Niel’s Kima Ventures, LocalGlobe, Kindred Capital, and several others. Its cap table also includes Docker and Dagger founder Solomon Hykes, Hugging Face co-founders Clément Delangue and Julien Chaumond, and LeCun, now with AMI Labs.
The Market ZML Is Entering: Expensive, Fast-Moving, and Already Contested
The inference infrastructure space has attracted capital at a pace that makes ZML’s $20 million look modest by comparison. Baseten, founded in 2019, is reportedly raising a $1.5 billion round in a split-priced structure: some investors are coming in at a $13 billion valuation, others at $11 billion, according to the Wall Street Journal as reported by TechCrunch. That follows a $300 million Series E just months earlier, at a $5 billion valuation, with Nvidia among the investors.
Baseten’s revenue trajectory helps explain that appetite. Startup Fortune reports the company tripled its annualised revenue run-rate from $200 million to $600 million in a single quarter, with customers including Cursor and Mercor citing cost savings of up to 30% over closed-source APIs.
Morin also names Inferact, from the team behind the open-source vLLM project, and RadixArk, the commercial entity behind SGLang, as competitors whose products partly overlap with LLMD. His response is to frame ZML’s scope as wider: ‘We have reached the point where we are co-designing silicon,’ he said, suggesting ambitions that extend well beyond software.
The chip vendor landscape ZML targets includes several European chipmakers: Axelera, Fractile, Kalray, OLIX, Q.ANT, SiPearl, SpiNNcloud, and VSORA. Morin’s interest in them is less geographical than technical. What draws ZML to these companies, he said, is the chance to work on ‘things that haven’t been done before anywhere in the world.’
He is clear that this is not a bet against Nvidia. ZML has a working relationship with the company, and Morin points to Nvidia’s existing installed base as a practical reality any inference provider must accommodate rather than circumvent.
Paris as a Feature, Not a Footnote
Morin’s credibility with investors traces in part to his time as VP of engineering at Zenly, the French social mapping app that Snapchat acquired in 2017. The deal, described in the original reporting only as a ‘nine-figure’ transaction, carried a confirmed cash consideration of $213 million, according to Snap’s SEC filings as reported by TechCrunch and corroborated by Business Insider.
That exit gave Morin both the network and the credibility to build ZML from Paris, a city he treats as non-negotiable. ‘I couldn’t do ZML anywhere but in Paris,’ he said.
For now, LLMD is free and the clock is running on how long that lasts. The more interesting question is whether a 20-person team, however well-funded and well-connected, can hold ground in a segment where competitors are closing rounds measured in billions. Morin’s answer, implicit in the product’s launch, is that speed and hardware agnosticism beat scale, at least until they do not. When the free tier ends, investors will find out which argument held.
