> There are indeed drawbacks in a lack of freedom, but assuming that a government should not be able to filter the content diffused to the population is wrong in principle.
It boils down to what one considers to be relevant for humans: I think that well being is more important than freedom. Historically, freedom was not a predominant part in human societies. On the contrary: slavery, kingdoms, empires, took part in human history more than freedom. Authoritarian government is not wrong per se, as long as people are well. In the same way, freedom of knowledge anything at any time is not necessary good. Actually, the ability to immediately access any content, beneficial or not, is something that humans acquired very recently in their history, and it's absolutely not clear that this is in fact something good in the long term. I think it is, but it's just speculation. Being conservative and NOT giving free communication is, I think, a more sensible default for a government. Also, there are cases where we already know that freedom doesn't help: CSAM, revenge porn, and other nasty stuff. ()
() edit: no, I was partly mistaken with these examples. I provided example of things that are known and widely accepted to be damaging of other liberties, while I meant to provide something more subtle, like fake news.
People are "awful at that" because when two people communicate, we're using a lot more than words. Each person participating in a conversation is doing a lot of active bridge-building. We're supplying and looking for extra nonverbal context; we're leaning on basic assumptions about the other speaker, their mood, their tone, their meanings; we're looking at not just syntax but the pragmatics of the convo (https://en.wikipedia.org/wiki/Pragmatics). The communication of meaning is a multi-dimensional thing that everyone in the conversation is continually contributing to and pushing on.
In a way, LLMs are heavily exploitative of human linguistic abilities and expectations. We're wired so hard to actively engage and seek meaning in conversational exchanges that we tend to "helpfully" supply that meaning even when it's absent. We are "vulnerable" to LLMs because they supply all the "I'm talking to a person" linguistic cues, but without any form of underlying mind.
Folks like your wife aren't necessarily "bad" at LLM prompting—they're simply responding to the signals they get. The LLM "seems smart." It seems like it "knows" things, so many folks engage with them naturally, as they would with another person, without painstakingly feeding in context and precisely defining all the edges. If anything, it speaks to just how good LLMs are at being LLMs.
> It's the same programming with LLMs. Through experience, you build up intuition and rules of thumb that allow you to get good results, even if you don't get exactly the same result every time.
Friend, you have literally described a nondeterministic system. LLM output is nondeterministic. Identical input conditions result in variable output conditions. Even if those variable output conditions cluster around similar ideas or methods, they are not identical.
The problem is that this is completely false. LLMs are actually deterministic. There are a lot more input parameters than just the prompt. If you're using a piece of shit corpo cloud model, you're locked out of managing your inputs because of UX or whatever.
Ah, we've hit the rock bottom of arguments: there's some unspecified ideal LLM model that is 100% deterministic that will definitely 100% do the same thing every time.
We've hit rock bottom of rebuttals, where not only is domain knowledge completely vacant, but you can't even be bothered to read and comprehend what you're replying to. There is no non-deterministic LLM. Period. You're already starting off from an incoherent position.
Now, if you'd like to stop acting like a smug ass and be inquisitive as per the commenting guidelines, I'd be happy to tell you more. But really, if you actually comprehended the post you're replying to, there would be no need since it contains the piece of the puzzle you aren't quite grasping.
Strange then that the vast majority of LLMs that people use produce non-deterministic output.
Funnily enough I had literally the same argument with someone a few months back in a friends group. I ran the "non-shitty non-corpo completely determenistic model" through ollama... And immediately got two different answers for the same input.
> Now, if you'd like to stop acting like a smug ass and be inquisitive as per the commenting guidelines,
Ah. Commenting guidelines. The ones that tell you not to post vague allusions to something, not to be dismissive of what others are saying, responding to the strongest plausible interpretation of someone says etc.? Those ones?
> Strange then that the vast majority of LLMs that people use produce non-deterministic output.
> I ran the "non-shitty non-corpo completely determenistic model" through ollama... And immediately got two different answers for the same input.
With deterministic hardware in the same configuration, using the same binaries, providing the same seed, the same input sequence to the same model weights will produce bit-identical outputs. Where you can get into trouble is if you aren't actually specifying your seed, or with non-deterministic hardware in varying configurations, or if your OS mixes entropy with the standard pRNG mechanisms.
Inference is otherwise fundamentally deterministic. In implementation, certain things like thread-scheduling and floating-point math can be contingent on the entire machine state as an input itself. Since replicating that input can be very hard on some systems, you can effectively get rid of it like so:
ollama run [whatever] --seed 123 --temperature 0 --num-thread 1
A note that "--temperature 0" may not strictly be necessary. Depending on your system, setting the seed and restricting to a single thread will be sufficient.
These flags don't magically change LLM formalisms. You can read more about how floating point operations produce non-determinism here:
In this context, forcing single-threading bypasses FP-hardware's non-associativity issues that crop up with multi-threaded reduction. If you still don't have bit-replicated outputs for the same input sequence, either something is seriously wrong with your computer or you should get in touch with a reputable metatheoretician because you've just discovered something very significant.
> Those ones?
Yes those ones. Perhaps in the future you can learn from this experience and start with a post like the first part of this, rather than a condescending non-sequitur, and you'll find it's a more constructive way to engage with others. That's why the guidelines exist, after all.
> These flags don't magically change LLM formalisms. You can read more about how floating point operations produce non-determinism here:
Basically what you're saying is "for 99.9% of use cases and how people use them they are non-deterministic, and you have to very carefully work around that non-determinism to the point of having workarounds for your GPU and making them even more unusable"
> In this context, forcing single-threading bypasses FP-hardware's non-associativity issues that crop up with multi-threaded reduction.
Translation: yup, they are non-deterministic under normal conditions. Which the paper explicitly states:
--- start quote ---
existing LLM serving frameworks exhibit non-deterministic behavior: identical inputs can yield different outputs when system configurations (e.g., tensor parallel (TP) size, batch size) vary, even under greedy decoding. This arises from the non-associativity of floating-point arithmetic and inconsistent reduction orders across GPUs.
--- end quote ---
> If you still don't have bit-replicated outputs for the same input sequence, either something is seriously wrong with your computer or you should get in touch with a reputable metatheoretician because you've just discovered something very significant.
Basically what you're saying is: If you do all of the following, then the output will be deterministic:
- workaround for GPUs with num_thread 1
- temperature set to 0
- top_k to 0
- top_p to 0
- context window to 0 (or always do a single run from a new session)
> The problem is that this is completely false. LLMs are actually deterministic. There are a lot more input parameters than just the prompt. If you're using a piece of shit corpo cloud model, you're locked out of managing your inputs because of UX or whatever.
When you decide to make up your own definition of determinism, you can win any argument. Good job.
Yes, that's my point. Neither driving nor coding with an LLM is perfectly deterministic. You have to learn to deal with different things happening if you want do do either successfully.
> Neither driving nor coding with an LLM is perfectly deterministic.
Funny.
When driving, I can safely assume that when I turn the steering wheel in the direction in turns. That the road that was there yesterday is there today (barring certain emergencies, that's why they are emergencies). That the red light in a traffic light means stop, and the green means go.
And not the equivalent "oh, you're completely right, I forgot to include the wheels, wired the steering wheel incorrectly, and completely messed up the colors"
>I don't mind using an LLM to shortcut areas that are just pure pain with no reward...
Enlightenment here comes when you realize others are doing the exact same thing with the exact same justification, and everyone's pain/reward threshold is different. The argument you are making justifies their usage as well as yours.
> PS: The rush was so great I was excitedly talking to my wife how I could port our emails away from google, considering all of the automatic opt in for AI processing and what not. The foolhardy me thought of even sabbatical breaks to work on long pending to-do's in my head.
I've been email self-hosting for a decade, and unfortunately, self-hosting your email will not help with this point nearly as much as it seems on first glance.
The reason is that as soon as you exchange emails with anyone using one of the major email services like gmail or o365, you're once again participating in the data collection/AI training machine. They'll get you coming or they'll get you going, but you will be got.
I've started experimenting with Claude Code, and I've decided that it never touches anything that isn't under version control.
The way I've put this into practice is that instead of letting claude loose on production files and services, i keep a local repo containing copies of all my service config files with a CLAUDE.md file explaining what each is for, the actual host each file/service lives on, and other important details. If I want to experiment with something ("Let's finally get around to planning out and setting up kea-dhcp6!"), Claude makes its suggestions and changes in my local repo, and then I manually copy the config files to the right places, restart services, and watch to see if anything explodes.
Not sure I'd ever be at the point of trusting agentic AI to directly modify in-place config files on prod systems (even for homelab values of "prod").
After years of cargo-culting this advice—"run ssh on a nonstandard port"—I gave up and reverted to 22 because ssh being on nonstandard ports didn't change the volume of access attempts in the slightest. It was thousands per day on port 22, and thousands per day on port anything-else-i-changed-it-to.
It's worth an assessment of what you _think_ running ssh on a nonstandard port protects you against, and what it's actually doing. It won't stop anything other than the lightest and most casual script-based shotgun attacks, and it won't help you if someone is attempting to exploit an actual-for-real vuln in the ssh authentication or login process. And although I'm aware the plural of "anecdote" isn't "data," it sure as hell didn't reduce the volume of login attempts.
Public key-only auth + strict allowlists will do a lot more for your security posture. If you feel like ssh is using enough CPU rejecting bad login attempts to actually make you notice, stick it behind wireguard or set up port-knocking.
And sure, put it on a nonstandard port, if it makes you feel better. But it doesn't really do much, and anyone hitting your host up with censys.io or any other assessment tool will see your nonstandard ssh port instantly.
Why?
reply