Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I wonder how they horizontally scaled shards. If they had 2k logical shards they probably had much fewer real/physical shards. So a single database was holding many sharding keys. So when a new physical shard gets added, the data needs to somehow be replicated. That is only true if w reading is the problem. If only a relatively short time period of data is hot you can probably just move over the logical shard IDs to the new physical shard without moving existing data. This requires keeping track of when which physical shard became active.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: