This is great. What are some good keywords to search on to learn more about topics like this? "distributed systems" seems too broad, and I'm sure there are whole fields of research into these types of concurrency problems. Any suggestions?

Maybe not keywords, but I'll leave here some links that you'll probably like:

https://news.ycombinator.com/item?id=7421858

http://research.microsoft.com/en-us/um/people/lamport/pubs/p...

https://news.ycombinator.com/item?id=2290180

https://news.ycombinator.com/item?id=8159969

https://news.ycombinator.com/item?id=10017549

I love this guys blog. The jepsen/call me maybe series is critical reading.