Getting local with the LLMs
I’m now open to new opportunities where I can help solve problems through design, strategy, and research—whether that’s improving user experiences, guiding product decisions, or uncovering insights that drive better outcomes. Happy to chat about how we might work together.
You might have heard, there’s this AI thing out there that everyone is talking about. It’s either going to save the world from the Office Suite production line or end the world in the next five minutes. Like just about anything in life, from smoking to having kids, a thing that is everywhere is something you think you should give a go. With AI you open ChatGPT, Claude, etc. and get your tax advice or ideas for world peace. That seemed way too easy. Thus, I pondered long and hard about how I could make this esoteric, arcane knowledge especially harder to use for myself. So I decided to set it up myself.
Enter Local LLMs, or AIs you run on your own computer, completely free, private as can be but generally not exactly user friendly. But easy somehow wasn’t the point. The easy route to do it yourself is with something like LM Studio or Jan.ai where you download the app, then the models through the app, then you type in your requests for poop jokes or the aforementioned tax advice. I’ve been using Ollama. and trying to shoehorn various other things with it which involves typing lines into a black screen to setup. There is something though about spending an hour trying to figure out how to configure, pounding your head into the wall trying to understand why this number being high is good. All whilst trying to keep your laptop from catching fire.
You might be asking yourself why one would do this to themselves. I wanted to learn how it worked. Learning the hard way so I actually learn how the word sausage is made. Running local models means that you understand the material of AI better, you stir that damn cauldron of magic yourself. This is still early days and we should all be tinkering in all sorts of way and trying to keep as much control of things as we can. People used to setup their own forums and mail servers. The internet was for a long time, whilst it was being figured out, not all up there in the clouds hovering above us. We should think about AI the same way, like tinkerers figuring out how we can evolve it well.
- Jim