European institutions aren’t known for moving fast. Quite the opposite. So when Olivier Debeugny received a call from the European Commission in June 2024, inviting him to Brussels to receive a prize from Thierry Breton, he couldn’t remember what he had applied for.
“I asked them: what did we win?” he says.
A few days later, he found himself in Brussels alongside three other winners, receiving €250,000 in cash. The real prize, however, was access to 2 million GPU hours—an opportunity that would normally cost between €7 million and €10 million. “They said that if we needed more, we should tell them. Actually, we got twice that amount!”
Six months earlier, Debeugny’s company, Lingua Custodia, had pitched an idea to a contest at the EU AI office. The brief called for an original concept: creating a foundation model for a large language model (LLM). The French fintech, which expanded into Luxembourg in 2019, had originally focused on machine translation for the financial sector. The arrival of LLMs made it possible to do much more—and CEOs are now racing to integrate AI into their processes. But there’s a catch: LLMs require Graphics Processing Units (GPUs), specialized electronic circuits that handle complex computational tasks by processing large amounts of data simultaneously. And GPUs are notoriously power-hungry.
Debeugny’s team had an idea for a new kind of architecture to dramatically reduce that power consumption: small, frugal LLMs dedicated to specific tasks.
Ditching the Transformers
Most LLMs are built on transformer architectures, which analyze the sum of all available data to respond to a prompt. This approach is both time- and energy-intensive. Debeugny’s team sidestepped this model. In their design, when someone asks the LLM a question, it consults only specific blocks of memory. The result: faster responses and smaller machines, saving energy.
“The idea is to make generative AI work for you, even on the servers you’re already using,” says Debeugny. “This is the first time in years that a European company is rolling out a new LLM architecture,” he adds. The last example was French LLM Mistral AI in 2023.
In October, Lingua Custodia—renamed Dragon LLM to reflect its core focus—rolled out a demonstration foundation model to prove the architecture works. Three models with varying parameter counts will be released before the end of 2025. Parameters determine how closely a model tracks patterns in words. Dragon LLM’s demonstration model behaves like a 7-billion-parameter model but uses just 3.6 billion active parameters.
“Our goal is a model with the equivalent of 70 billion parameters, using only 7 billion active ones,” Debeugny says. A smaller model is also planned.
The platform will be open source, allowing scientists and developers to train their own models using the architecture. Meanwhile, the company continues to fine-tune models for clients integrating generative AI into their business processes.
“For instance, in KYC, your LLM doesn’t need to be an expert in Mexico,” Debeugny explains. “That’s where a frugal, task-specific architecture really makes sense for proper deployment.”
A Missed Opportunity
At the start of 2025, Debeugny sought to position Dragon LLM as Luxembourg’s sovereign AI solution. “It made sense for a company like ours to become the national leader in AI, given our historical ties to finance,” he says. But he was too late: Luxembourg had already committed to Mistral AI.
As a result, Dragon LLM is focusing on France, partnering with a major local client to leverage financial-sector data and fine-tune the LLM.
“First you need good architecture, then you need the data,” Debeugny says. The challenge is choosing data that is both relevant for Europe and energy-efficient.
Far from being bitter about missing the opportunity to support Luxembourg’s public sector, Debeugny accepts that consolidation in Europe’s AI ecosystem will be necessary to build an LLM capable of competing with models from the US, China, or the Emirates. He remains optimistic that Europe can create its own AI champion, an Airbus of artificial intelligence.
“That’s why I found this challenge from the EU Commission extremely energising and positive,” he says. “For a scientist in a small company, having access to massive computing power to publish in the scientific community is more valuable than getting a big paycheck from Microsoft in Seattle.”
This article was published in the Silicon Luxembourg magazine.
