Euro open source specialist proposes LLM gateway to ease token bills

European open source database service provider Aiven says it has saved 80 percent of its AI token costs internally by moving to open-weight models and is building a gateway to help devs swap models in and out as required.

The Finland-based company built its business on hosting and providing open-source databases as a service, and in the last couple of years has moved into machine learning.

Speaking to The Register, Ville Lehto, Aiven’s VP of strategy, said that when dealing with open source AI infrastructure, it's not trivial to make something that can run across the different clouds. However, in giving its developers access to LLMs, the company wanted to manage its own token costs and control its "own destiny when it comes to AI usage."

After a wave of debate around so-called token-maxxing, developers have generally become more interested in reducing their costs by using a mix of models, including cheaper Chinese alternatives to the Western frontier models, and …