Most AI models are not open-source. That is the fact. Yet on the internet, there is no shortage of content parroting "open-source" models seemingly releasing on. a daily basis.
While it's an innocent slip in vocabulary, and while the differences may not seem all that serious, they are. That's because something that's seemingly "open-source" may have people drop critical defenses in security, and how they choose to engage with the model.
Open-Weight Models, What They Actually Are
What most people are actually referring to when they speak of "open-source" models, are actually open-weight models.
These are models that you can run fully on your own computer hardware without any cloud middleman to provide you the service. These models are completely free.
Alongside the ability to run it yourself, the model "weights" which we'd loosely define as the mathematical representation of the AI's neural network, is also publicly available.
Those two realities alone lead people to believing that open-weight = open-source, when that is factually incorrect, and potentially dangerous thinking if casual explorers choose to download the model.
Open-Weight Models Are Like A Non-GMO Food Certification
The market is littered with open-weight models these days. There is no shortage, and in our opinion, it is close to meaningless outside of the fact that you can run them on your computer.
The reality is that open-weights are no more than numbers. They are mathematical representations that you cannot possibly interpret at the moment. You still do not know what biases the model has based on its training data, you don't know if someone poisoned the training data with malicious instructions, you do not know anything at all.
It's like a health conscious individual thinking that non-gmo means the food has no chemical addidives or pesticides.
The Consequence Of Poisoned Models
Performance can degrade on certain topics and triggers and you'll never know, because you could not see into the system even if you wanted to.
Real backdoor triggers can exist as well that create vulnerablities with your data, as well as your machine at large.
Open-Source Models
Open Source means you can genuinely (if you wanted to) inspect the training data. Not just the weights. The training data that actually created the model weights is publicly available.
There are genuinely few of these, because economically, a market exists for building proprietary models for the sake of competition. If you hide your training data, you can remain competitive, claim market share with your model, and if you have ill-intentions, you can shield them.
OLMo is an example of a genuinely open-source model that we suggest for people getting started. As time goes on, these models should only improve as people value transparency and fair-training that does not involve the theft of intellectual property.
Regardless Of Model Provenance: Security Principals Should Remain The Same
As mentioned in our manuals, securing models properly when you're running them locally should be a standard.
That means regardless of if it's open-weight or open-source, you secure yourself with network firewalls, as well as virtual machine containers (especially if you're giving your AI permissions outside of a basic chat window.
Our manuals provide guidance on all of it.
Final Reminder
"Open-source" is not always open-source.
Treat every model like poison on your computeer, and protect yourself properly regardless of model provenance.
Where To Go From Here?
We wrote two manuals for independent AI ownership. These books are about training literacy, and outright, off-cloud AI ownership without cloud middlemen.
We believe every concern with AI is resolved with literacy and ownership. Be it data privacy concerns, hallucinations, and everything else you have less control over when you do not own the actual machine independently.
Until next time.
Endgate Systems
Author written.
Free · Read this first
7 things most people teaching independently owned AI aren’t telling you.
What most instructors leave out, either because they don’t know or because it makes the sale harder. The truth about hallucinations. What security actually means at the machine level. What “local” really is, and what it isn’t.
Until next time.
Endgate Systems
Author written.