← Back to stories
AI

Dev squeezes an AI language model onto a floppy for 486DX and Pentium PC stalwarts

An inspired developer has shoehorned a language model onto a floppy disk created especially for 486DX and Pentium PC users running DOS. Please note that WollyMonjac’s project isn’t a large language model (LLM). Though it shares the transformer model architecture of its much bigger siblings, Tiny 2.1 is humbly described as “a tiny question-answering transformer written in Turbo C 2.0.”

There isn’t much room for modern software on a floppy disk, with a typical PC formatted disk in the 486 era boasting just shy of 1,475,000 bytes of data capacity. Tiny 2.1 makes the best use of this limited headroom as possible, fitting in the following:

367,000 parameters: 4 layers, 96 wide, reads and writes letter by letter

8-bit weights (one scale per row), 380 KB in all

Trained on about 3,500 facts, each asked in many wordings

Your creaky old PC will need about 540KB of free conventional memory to run Tiny 2.1. …

You're reading a preview. The full article is published by Tom's Hardware on their website.

Read the full story on Tom's Hardware