I built Andrej Karpathy’s “LLM Council” on my own hardware, and now no single model gets the last word
I have been in an on-and-off relationship with local LLMs. Initially, I tried 1st-gen or 2nd-gen 8B models, but they were not always up to my expectations. I gave up on them and moved back to cloud models. A couple of weeks ago, I gave local LLMs another shot. I even tested three local LLMs side by side on my RTX 4070 Ti and decided to keep one of them as the default. Not because that one single model beat the other two, but because running all three side by side for every single prompt wasn't practical. Every model had its own strengths, though, so I kept the other two around and reached them whenever the prompt really mattered. That got me thinking: what if I could use all three to get better output? That’s when I came across Karpathy's LLM Council, initially designed to work across cloud APIs. I liked the idea and decided to rebuild...
I have been in an on-and-off relationship with local LLMs. Initially, I tried 1st-gen or 2nd-gen 8B models, but they were not always up to my expectations. I gave up on them and moved back to cloud models. A couple of weeks ago, I gave local LLMs another shot. I even tested three local LLMs side by side on my RTX 4070 Ti and decided to keep one of them as the default. Not because that one single model beat the other two, but because running all three side by side for every single prompt wasn’t practical. Every model had its own strengths, though, so I kept the other two around and reached them whenever the prompt really mattered. That got me thinking: what if I could use all three to get better output? That’s when I came across Karpathy’s LLM Council, initially designed to work across cloud APIs. I liked the idea and decided to rebuild it around local models running on my own hardware.
John Doe
New York
New York
Published by: aplhsindia.in
