The size of the OS threads with their stacks is what I am worried about as well. It is interesting the author hints that:
> It drops further when asked to allocate 8MiB stacks like C is doing
Memory, just like CPU is constrained and costs money especially if provisioned in the cloud. If I have 2G of memory available I can spawn 250 threads and keep them around without swapping.
EDIT: actually scratch the above, _wmd pointed out that stack memory is just like virtual memory, it is allocated but brought into physical memory as needed.
This changes how code is written. It is not just an internal detail! Now you cannot think "one-connection = one-task" now it is about locks, queues and mapping control blocks to connections. That is the biggest loss.
> GC is great for multi-threading, but not so great for hundreds of gigabytes of variable lifetime objects. Even if the GC can handle it, GC still has unacceptable 2x space overhead
Good point on GC. I see the main issues in a large concurrent system (and presumably Rust wants to be a good tool for concurrent systems) is non-blocking GC. A slow GC might be acceptable, but if one errant task allocated memory in certain pattern and GC has to lock all the tasks out to run, that could be a serious issue.
Responsiveness (liveliness) of a concurrent system is something that is often forgotten and everyone wants to talk about how fast one can transpose a matrix.
Now a concurrent GC is possible, Azul built one, for Java, it is very interesting how it work. I enjoy reading their whitepapers:
Just highlighting a small, but common miscomprehension (that on one project, caused a PM to mail everyone at 2am about a HUGE nonexistent memory leak)..
Thread stacks are virtual memory like everywhere else, so in reality an "8mb" stack means "8mb maximum size". Sure, they won't shrink once pages are faulted in to back them, but in the average application, especially on 64bit, this should never be a problem
If for the lifetime of your thread, its stack only ever grew to 32kb, then the OS will only have allocated 32kb of memory to back it.
You won't hit the worst case, but the common case is that threads will burst stack usage during task execution. If you have 100k threads your effective stack usage will be the high watermark.
It varies by workload, but for mine that is between 128 and 512 kilobytes per thread.
If you churn all your threads no problem. If you are hosting 100k persistent connections I suspect it would detract from available memory.
That's interesting. So if a thread briefly uses 1M stack, and relinquishes most of it, then the kernel still keeps that unused 1M memory in the page table?
I expected the linux kernel to be more intelligent, but maybe there's a reason it behaves like that?
Under pressure, it will eventually page the memory back out, but generally you never want to get your machine into a state where paging happens. Aside from that, there is no mechanism that allows the kernel to know the memory is no longer used unless you explicitly tell it, e.g. via madvise(2).
Though I can't think of a sane way to call madvise(2) involving a thread stack
That makes sense, thanks for clarifying. So if say there are 1000 client connection threads and they monitor some status and say ping their clients every 5 seconds, on wake-up, it could be only a small fraction of the stack would have to be paged-in. And it would depend how deep the stack was when the thread of put to sleep?
That's not to say threads aren't expensive, they still require several heavyweight struct allocations on the kernel side, e.g. struct task_struct, which I counted to 1k before getting bored (and wasn't even quarter way through the fields)
Edit: Linux git HEAD with Debian unstable .config:
> It drops further when asked to allocate 8MiB stacks like C is doing
Memory, just like CPU is constrained and costs money especially if provisioned in the cloud. If I have 2G of memory available I can spawn 250 threads and keep them around without swapping.
EDIT: actually scratch the above, _wmd pointed out that stack memory is just like virtual memory, it is allocated but brought into physical memory as needed.
This changes how code is written. It is not just an internal detail! Now you cannot think "one-connection = one-task" now it is about locks, queues and mapping control blocks to connections. That is the biggest loss.
> GC is great for multi-threading, but not so great for hundreds of gigabytes of variable lifetime objects. Even if the GC can handle it, GC still has unacceptable 2x space overhead
Good point on GC. I see the main issues in a large concurrent system (and presumably Rust wants to be a good tool for concurrent systems) is non-blocking GC. A slow GC might be acceptable, but if one errant task allocated memory in certain pattern and GC has to lock all the tasks out to run, that could be a serious issue.
Responsiveness (liveliness) of a concurrent system is something that is often forgotten and everyone wants to talk about how fast one can transpose a matrix.
Now a concurrent GC is possible, Azul built one, for Java, it is very interesting how it work. I enjoy reading their whitepapers:
http://www.azulsystems.com/resources/whitepapers