I'd be curious how ETS does on that problem. The Erlang efficiency guide says an ETS table takes:
> Initially 768 words + the size of each element (6 words + the size of Erlang data). The table grows when necessary.
This might be too large if there's a ton of them, but perhaps a single table per node could be workable? In particular, ordered_set tables with write_concurrency have been improved a lot on Erlang/OTP-22:
OTP-15128 Application(s): erts, stdlib
ETS option write_concurrency now also affects and
improves the scalability of ordered_set tables. The
implementation is based on a data structure called
contention adapting search tree, where the lock
granularity adapts to the actual amount of concurrency
exploited by the applications in runtime.
ETS has no facility to insert/remove an object and get the index at which the object was inserted to/removed from. This is needed in order to synchronize windows within the sorted set to our clients. We would have loved to use ETS, and use it in many places actually, and have open-sourced a few:
I don't know if it would be good enough,and there's certainly not a big reason for you to try, given your solution works great; but you could probably modify ets to give you the slot number where it inserted/removed and use that as a non-exact indicator of where in the sequence it was. (Depending on how accurate you need it to be, this might work?)
> Initially 768 words + the size of each element (6 words + the size of Erlang data). The table grows when necessary.
This might be too large if there's a ton of them, but perhaps a single table per node could be workable? In particular, ordered_set tables with write_concurrency have been improved a lot on Erlang/OTP-22: