What makes you think that RAII- and arena-based strategies are in tension with one another? RAII and smart pointers are more related to the ownership and resource management model. Allocating items in bulk or from arenas is more about where the underlying resources and/or memory come from. These concepts can certainly be used in tandem. What is the substance of the argument that RAII, etc. are "hot garbage?"
The best argument I've ever come across against using RAII is that you end up with these nests of objects pointing to one another, and if something fails, the cleanup code can really only do one thing, which is unwind and deallocate (or whatever the cleanup path is). This structure, generally, precludes the possibility of context dependent resource re-usage on initialization failure, or on deallocation, because you kind of have to have only one deallocation path. Obviously, you could imagine supporting in an RAII context, but, the point is that you probably have to put a fair bit of conscious effort into doing that, whereas if you have a less .. rigid.. ownership model, it becomes completely trivial.
I agree that the allocation model and ownership model are independent concepts. I mentioned arena allocation because the people I know that reject the traditional C++ ownership model generally tend to favor arenas, scratch space, freelists, etc. I'm specifically interested in an ownership model that works with arenas, and tracks ownership of the group of allocations, as opposed to the typical case we think about with RAII where we track ownership of individual allocations.
That "nest of objects point to each other" makes no sense ... RAII is just a technique where you choose to tie resource management to the lifetime of an object (i.e. acquire in constructor, release in destructor).
If an exception gets thrown, causing your RAII object scope to be exited, then no problem - the object destructor gets called and the resource gets released (this is the entire point of RAII - to make resource allocation and deallocation automatic and bullet-proof).
If you are creating spaghetti-like cross-referencing data structures, then that is either poor design or something you are doing deliberately because you need it. In either case, it has nothing to do with RAII, and RAII will work as normal and release resources when the managing object is destroyed (e.g. by going out of scope).
RAII could obviously be used to allocate/free a resource (e.g. temp buffer) from a free list, but is not really relevant to arena allocation unless you are talking about managing the allocation/release of the entire arena. The whole point of an arena allocator is the exact opposite of RAII - you are deliberately disconnecting individual item allocation from release so that you can instead do a more efficient bulk (entire arena) release.
Another commentor succinctly pointed out one argument against RAII+friends is that it encourages thinking about single objects, as opposed to bulk processing.
In many contexts, the common case is in fact bulk processing, and programming things with the assumption that everything is a single, discrete element creates several problems, mostly wrt. performance, but also maintainability. [1][2]
> The whole point of an arena allocator is the exact opposite of RAII
Yes, agreed. And the internet is rife with people yelling about just how great RAII is, but comparatively few people have written specifically about it's failings, and alternatives, which is what I'm asking about today.
I don't think the "complex web of objects to be deallocated" scenario is usually a problem, but I generally agree with your points. As always, careful design and control of the software is important. Abstractions are limited; spend them carefully.
> one argument against RAII+friends is that it encourages thinking about single objects, as opposed to bulk processing.
RAII is just a way to guarantee correctness by tying a resource's lifetime to that of an object, with resource release guaranteed to happen. It is literally just saying that you will manage your resource (a completely abstract concept - doesn't have to be memory) by initializing it in an objects constructor and release it in the destructor.
Use of RAII as a technique is completely orthogonal to what your program is doing. Maybe you have a use for it in a few places, maybe you don't. It's got nothing to do with whether you are doing "bulk procesing" or not, and everything to do with whether you have resources whose usage you want to align to the lifetime of an object (e.g this function will use this file/mutex/buffer, then must release it before it exits).
The alternative to RAII is simply do-it-yourself ! For example, if you are writing multi-threaded code and need a mutex to protect some data structure, then you'd need an explicit mutex_lock() before the access, and an explicit mutex_unlock() afterwards... if your code might throw exceptions or branch due to errors, then make sure that you have mutex_unlock() calls everywhere necessary!
Automating this paired lock and unlock, to avoid any programmer error in missing an unlock in one of the error paths, just makes more sense, and this is all that RAII is doing - it's not some mysterious religion or design philosophy that needs to pervade your program (other than to extent that it would make sense to remove all such potential programmer errors, not just some!).
In this mutex example, RAII would just mean having a class "MutexLocker" (C++ provides std::lock_guard) that does the mutex_lock() in it's constructor and mutex_unlock() in it's destructor.
Without RAII, your code might have looked something like this:
try {
mutex_lock(mut);
// access data structure
mutex_unlock(mut);
} catch (...) { // make sure we unlock the mutex in the error case too !
mutex_unlock(mut);
}
With the RAII approach the mutex unlocks are automatic since they'll happen as soon as the mutex locker goes out of scope and its destructor is called, so the equivalent code would now look like this:
try {
MutexLocker locker(mut);
// access data structure
} catch (...) {
}
That's it - no big deal, but note that now there was no need to add multiple mutex unlock calls in every (normal + error) exit path, and no chance of forgetting to do it.
You can do the same thing anywhere where you want to guarantee that some paired "resource" cleanup activity take place, whether that is unlocking a mutex, or closing a file, or releasing a memory buffer, or whatever.
You may not think of it as RAII, but this approach of automatic cleanup is being used anywhere you have classes that own something then release it when they are done with it, for example things like std::string (and all the C++ container classes) is doing this, as are C++ <stream> file objects, C++ smart pointers, C++ lock_guard for mutexes, etc, etc.
The name "RAII" (resource acquisition is initialization) is IMO a poor name for the technique, since the value is more in the guaranteed, paired, resource release than the acquisition, which was never a problem. Scope-based resource management, or Lifetime-based resource management, would better describe it!
Of course RAII isn't always directly applicable because maybe you need to acquire a resource in one place and release it someplace else entirely, not in same scope, although you could choose to use a smart pointer together with a RAII resource manager to achieve that. For example create a resource with std::make_shared<ResourceManager>() in one place, and resource release will still happen automatically whenever reference counting indicates that all uses of the resource are done.
There are plenty of people who use RAII with arenas for nested group of objects.
Bloomberg for example had a strong focus on that, and they enhanced the allocator model quite significantly to be able to standardize this. This was the reason for stateful allocators, scoped allocators, uses-allocator construction and polymorphic memory resources.
In my library [1], wrapping the CUDA APIs in modern C++, I do allocations which are not exactly from an arena, but something in that neighborhood - memory spaces on context on GPU devices.
Unlike the GP suggests, and like you suggest, I have indeed embraced RAII in the library - generally, not just w.r.t. memory allocation. I have not, however, replicated that idioms of the standard library. So, for example:
* My allocations are never typed.
* The allocation 'primitives' return a memory_region type - essentially a pointer and a size; I discourage the user from manipulating raw pointers.
* Instead of unique_ptr's, I encourage the use of unique_span's: owning, typed, lightweight-ish containers - like a fusion of std::span<T> and std::unique_ptr<T[]> .
The parking spaces in question aren’t free; the city sold the long-term rights to operate the parking facilities to the private sector in a bid to balance one year’s budget.
Overprovisioning is much less aggressive than this in practice. A read-oriented SSD with 15.36 TB of storage typically has 16.384 TiB of flash. The same hardware can be used to implement a 12.8 TB mixed-use SSD (3 DWPD or more).
But, if model development stalls, and everyone else is stalled as well, then what happens to turn the current wildly-unprofitable industry into something that "it makes sense to keep spending billions" on?
I suspect if model development stalls we may start to see more incremental releases to models, perhaps with specific fixes or improvements, updates to a certain cutoff date, etc. So less fanfare, but still some progress. Worth spending billions on? Probably not, but the next best avenue would be to continue developing deeper and deeper LLM integrations to stay relevant and in the news.
The new OpenAI browser integration would be an example. Mostly the same model, but with a whole new channel of potential customers and lock in.
Because they’re not that wildly unprofitable. Yes, obviously the companies spend a ton of money on training, but several have said that each model is independently “profitable” - the income from selling access to the model has overcome the costs of training it. It’s just that revenues haven’t overcome the cost of training the next one, which gets bigger every time.
> the income from selling access to the model has overcome the costs of training it.
Citation needed. This is completely untrue AFAIK. They've claimed that inference is profitable, but not that they are making a profit when training costs are included.
I understand what GP meant, but extraction of values from a sparse matrix is an essential operation of multiplying two sparse matrices. Sparse matmult in turn is an absolutely fundamental operation in everything from weather forecasting to logistics planning to electric grid control to training LLMs. Radix sort on the other hand is very nice but (as far as I know) not nearly used as widely. Matrix multiplication is just super fundamental to the modern world.
I would love to be enlightened about some real-world applications of radix sort I may have missed though, since it's a cool algorithm. Hence my question above.
Not always, or rather not exclusively. For example, some types of distillation benefit from sparse-ifying the dense-ish matrices the original was made of [1]. There's also a lot of benefit to be had from sparsity in finetuning [2]. LLMs were merely one of the examples though, don't focus too much on them. The point was that sparse matmul makes up the bulk of scientific computations and a huge amount of industrial computations too. It's probably second only to the FFT in importance, so it would be wild if radix sort managed to eclipse it somehow.
Where did you get the pricing for vast.ai here? Looking at their pricing page, I don't see any 8xH200 options for less than $21.65 an hour (and most are more than that).
The last few generations of GPU architectures have been increasingly optimized for massive throughput of low-precision integer arithmetic operations, though, which are not useful for any of those other applications.
This is the common argument from proponents of compiler autovectorization. An example like what you have is very simple, so modern compilers would turn it into SIMD code without a problem.
In practice, though, the cases that compilers can successfully autovectorize are very limited relative to the total problem space that SIMD is solving. Plus, if I rely on that, it leaves me vulnerable to regressions in the compiler vectorizer.
Ultimately for me, I would rather write the implementation myself and know what is being generated versus trying to write high-level code in just the right way to make the compiler generate what I want.
It doesn’t sound like they are referring to newborns needing to be physically present to get a SSN. Instead, it seems to refer to persons who are registering to start receiving their Social Security benefits (or existing recipients who want to change their direct deposit information). Also, there is an existing supported method for identifying yourself electronically that is mentioned in the article. In that sense, the headline seems a bit misleading.
> It doesn’t sound like they are referring to newborns needing to be physically present to get a SSN.
Correct. They already tried to take care of that two weeks ago.
Since 1988, through a program called Enumeration at Birth (EAB), as part of their state's birth registration process (typically done at the hospital when the baby is born) allowed the parents to request Social Security registration. The state would then automatically send the information to the Social Security Administration (SSA) for processing.
Last week the acting SSA director ordered EAB cancelled in Maine, so parents would have to go to an SSA office to register their newborn [1][2][3]. The DOGE website showed that EAB was cancelled in Arizona, Maryland, Michigan, New Mexico, and Rhode Island (but did not list Maine).
There was enough objection to this that they then reversed the cancellation in Maine. The acting director said
> I recently directed Social Security employees to end two contracts which affected the good people of the state of Maine. The two contracts are Enumeration at Birth (EAB), which helps new parents quickly request a Social Security number and card for their newborn before leaving the hospital, and Electronic Death Registry (EDR) which shares recorded deaths with Social Security. In retrospect, I realize that ending these contracts created an undue burden on the people of Maine, which was not the intent. For that, I apologize and have directed that both contracts be immediately reinstated
He also said that EAB and EDR remain in place for every state.
As far as I know there was no explanation given on why specifically Maine, or on why those 5 other states were on the DOGE site.
reply