Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> The same input will always produce the same output

Not guaranteed even with the same seed. If you don't perform all operations in exactly the same order, even a simple float32 sum, if batched differently, will result in different final value. This depends on the load factor and how resources are allocated.



Yeah, the fact that floating point multiplication isn't associative is a real pain for producing deterministic outputs - especially when you're running massively parallel computations on GPUs (or multiple GPUs) making the order of operations even less predictable.


This doesn’t mean LLMs are inherently non-deterministic, just that current common implementations are non-deterministic.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: