Open source
Fine Tune an 8B Model on a 4 GB Laptop GPU
Soup is an open source command line tool that fine tunes large language models from one YAML config. It keeps the frozen base model in your computer's memory and feeds the graphics card one layer at a time, so an 8B model can train on a 4 GB laptop GPU.
Watch on YouTube · Watch on TikTok
The story
What the video said
You want to fine tune a large language model on your own laptop. You tried before, but the model would not fit in your small graphics card memory. Soup is an open source command line tool that fine tunes large language models. You write one YAML file and run one command to start training. Soup keeps the frozen base model in your computer memory. It sends one layer at a time to the graphics card, so the card only holds a small piece. The project reports fine tuning Llama 3.1 8B on a 4 GB laptop GPU. It used 3.32 GB of memory at the highest point. This means you can train a useful model on the laptop you already own. It is free and open source. Would you use this workflow for your own project?
This page was generated from the Ming Dao AI publication record.
Ming Dao AI is independent and unaffiliated with GitHub or the featured project. Read our editorial policy.