Menu
I have a limited set of resources that I want to share between multiple processes than run on multiple servers. I guess what I need could be called a distributed semaphore.
I have found an old Perl implementation (based on memcached) that could be adapted. I have not fully investigated it yet.
Is there a library/component/software that already does that? An algorithm, perhaps? How about failover?
275
In order to implement a distributed semaphore, you need to define an authority amongst a cluster of node, then once it is selected, you need to centralize requests to it. Handling lock requests is a piece of cake, selecting the leader is more complicated. Paxos will solve that for you.
Implementation and properties of semaphores, while well understood in shared memory systems, have not received sufficient attention in distributed systems. The lack of shared memory makes implementation of semaphores difficult in a distributed system.
A semaphore is a synchronization mechanism that allows two or more concurrent, parallel or distributed software components, executing on a shared memory parallel platform, to block (wait) for an event to occur.
The algorithm for this is called Paxos. There are other algorithms, but they all reduce down to Paxos (or are incorrect). The most popular implementation of it is Apache Zookeeper. The Zookeeper servers run Paxos amongst themselves. Clients refer to named objects and can lock them etc.
70
I want to point out that from the use case you describe there is probably little need for the semaphore itself to be distributed. So you do not necessarily want a distributed semaphore, but a semaphore a for distributed system.
Throttle might fit this bill: https://github.com/pacman82/throttle
29
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
