One of the more interesting entries in the competitive large language model (LLM) space has been Llama from Meta (nee Facebook). In a direct challenge to the Microsoft-OpenAI partnership, Meta decided to fully open source their Llama platform. Interestingly, this true open-sourcing has led to a lot of enthusiasts trying to run various models on their own hardware at home, tweaking and learning as they go. While these do-it-yourselfers running small Llama models on re-purposed Ethereum mining rigs won’t challenge corporate AI data centres soon, it does allow for lots of new ideas to flourish.
Community
One of the better discussion forums for this community effort is found on the LocalLLaMA sub-reddit. Besides the silly memes, there is suprisingly good comparison of current models along with tips and tricks for running them yourself. This includes the hardware requirements (Nvidia GPU) and Python setup on Linux.
ChatGPT’s Take
Here’s how easy this is to setup:
Conclusions
For the enthusiat crowd, running open-sourced models will be the way forward for the forseeable future. I hope that we don’t end up in a spot where this technology can only be run by the largest tech companies, using their proprietary algorithms and models.
More in this series…
- Google Gemini - Google Gemini
- Anthropic Claude - Anthropic Claude
- ChatGPT 4o - ChatGPT 4o
- Anthropic Claude in Canada - Anthropic Claude in Canada
- LLMs on Android - LLMs on your Phone