Lecture 006 - Mutual Exclusion

Mutex Requirements:

• correctness and safety: one process hold critical section

• fairness: Any process that makes a request must be granted lock

  • deadlock-free
  • assume no process will hold onto a lock indefinitely
  • Eventual fairness: Waiting process will not be excluded forever
  • Bounded fairness: Waiting process will get lock within some bounded number of cycles

number of cycles (typically n)

We need consistency in distributed system accross different machines. (no inconsistency)

Distributed Mutex Requirements: no shared memory

1. low message overhead
2. no bottlenecks
3. tolerate out-of-order message
4. allow processes to join protocol or to drop it
5. tolerate failed processes
6. tolerate dropped messages

Assume: - total number of processes is fixed at $n$. - No process fails or misbehaves. - message never fails, but can be reordered.

Unicast: point-to-point Multicast: broadcast to multiple machines (might or might not be all). all messages are delivered in the same order to each receiver. Cycle: one transection (a complete round of the protocol with one process i entering its critical section and then exiting.)

Mutual Exclusion Algorithms

We assume we have $n$ copies of data in $n$ servers that are distributed in the world. Each server got request from client and want to access its database. But we need to make sure all the copies of database are identical upon modification by multiple servers. We only care about the locks, but we don't care about how exactly the data is modified. // QUESTION: is this the assumption?

Existing Algorithms:

• Centralized Mutual Exclusion: centralized server queueing up requests

  • Correctness: safe, fairness depends on queuing policy
  • Performance: "cycle" is nly 3 messages per cycle: 1 request, 1 grant, 1 release. Lock server create bottleneck
  • Issues: single point failure

• Bully Leader Election

  • Goal: avoid single point failure, choose the server with best internet connectivity to be a leader
  • Fairness: no fairness, assume no malicious process, assume centralized senerial
  • Performance: still bottleneck

• A Ring Algorithm: priority based

• Decentralized Mutual Exclusion: resource is replicated

• Totally-Ordered Multicast

• Lamport Mutual Exclusion

• Ricart & Agrawala Mutual Exclusion

• Token Ring Mutual Exclusion

Decentralized Mutual Exclusion: a way to access critical region shared by multiple server

1. When a server want to access critical region, ask all other node.
2. If majority node send agree respond, then access.
3. Otherwise, wait some seconds before asking again

Decentralized Mutual Exclusion: - correctness: majority vote ensure safety (very low failure rate with only $8$ copies of data in $8$ servers) - fairness: random chance - performance: $2m + m$ total message to get majority (but unbounded number of messages per cycle) - node failure and recovery is still a issue (when coordinator fail and forgot it has already granted someone resource, and made another grant) - backoff and retry might lead to starvation (when everybody is trying to access resource)

Bully Leader Election: Nobody want to be the leader, but everybody can appoint a leader. Everybody must accept appointment unless you can hand the task of the leader to other people. Youtube

1. Assume servers are enumerated from $S_0, S_1, ..., S_n$
2. Everything is fine until one server $S_i$ notice a leader does not respond
3. The server $S_i$ appoint new leader $\{S_j | j > i\}$
4. Whoever received the appointment try to hand the task to other. Say $S_j \in \{S_j | j > i\}$ received the appointment, it need to appoint new leader $\{S_k | k > j\}$.
5. Repeat the process until one cannot appoint his own leader task to someone else.

Ring Algorithm Election: processes arrange as a ring and collect information about all the alive servers Youtube

1. Assume servers are enumerated from $S_0, S_1, ..., S_n$ where each number preresent a priority.
2. Everything is fine until one server $S_i$ notice a leader does not respond
3. The server $S_i$ puts its name on a piece of paper [i] and send it to server $S_{i+1 \mod n}$, if it does not respond, then send it to $S_{i+2 \mod n}$ until someone reply
4. Say server $S_{i+1 \mod n}$ got the piece of paper, it add its own name on the paper too [i, i+1] and send it to the next person $S_{i + 2\mod n}$
5. The piece of paper will eventually get back to server $S_i$. And the server can now appoint the leader from [...] the piece of paper who has the highest priority.

Total Ordered Multicast using Total Order Lamport (TO-Lamport) Clock: we want all servers to process a specific list of command in order

1. Each command, when generated, is timestamped with the server's current total order lamport timestamp when the server first get the message (assume only 1 server in the system got the message). The command is then broadcast to other servers.
2. Each server obtain a sorted queue to store all received command. They are sorted based on total order lamport clock.
3. Each server, upon receive the broadcast command, send ACK to ALL servers in the system.
4. Each server, upon receive ACK, mark the corresponding command as ACKED.
5. Each server, repeatably check if the oldest element in the queue is ACKED. If so, process that command.

Property of Multicast using TO-Lamport:

• tolerate: crazy message delay

• need to assume: all messages sent by one sender are received in the order they were sent and that no messages are lost.

• Why does it work

  • By getting ACK, the the receiver must have received all prior command from the sender
  • Queue order ensure correctness
  • All procfesses will eventually have the same order of the local queue before deliver command

Lamport Mutual Exclusion: Total Ordered Multicast with slight change, assuming no replicate data among servers

• instead of sending command, we send p_i want to access critical region

• when processing command, we allow p_i to access critical region

• only ACK to one server (sender) instead of broadcasting ACK (only the requester need to know when it can access critical region, assuming no malicious server)

  • All server will have the same order of p_i want to access critical region, but might not have consistent ACKED status on every p_i want to access critical region.
  • The sender server will eventually get a ACK and have its p_i want to access critical region being the oldest in the queue

• broadcast RELEASE lock to all servers once sender has done with critical region

• upon receive RELEASE, remove element in the queue no matter whether p_i want to access critical region is ACKED

Performance of Lamport Mutual Exclusion: - process $i$ sends $n-1$ p_i want to access critical region messages - process $i$ receives $n-1$ ACK messages - process $i$ sends $n-1$ RELEASE messages

The algorithm does not tolerate server failure.

Ricart & Agrawala Mutual Exclusion: assume no message lost, mul

1. When a server want to access critical region, it broadcast REQUEST to all servers
2. The server is automatically granted permission when all server respond YES
3. Receiver, upon receive RESUEST:
   1. Reply YES if it is not interested in data.
   2. Does not reply if it is currently accessing data.
   3. Reply YES if it is interested in data, but has not gotten permission, and the lamport clock is the current REQUEST is lower than itself's REQUEST.
   4. Does not reply if it is interested in data, but has not gotten permission, and the lamport clock is the current REQUEST is greater than itself's REQUEST.

Properties of Ricart & Agrawala Mutual Exclusion: - Correctness: yes (by contradiction with total order lamport) - Deadlock Free: no cyclic wait with total order lamport - Starvation Free: we do not drop request, assume no dropped message, lamport clock strictly increase - Performance: $2(n-1)$ messages in a cycle ($n-1$ requests by $i$ and $n-1$ replies to $i$) - Issues: no failure tolerate, will introduce starvation if one server fails

Token Ring Algorithm:

1. There is only one token in a system of servers that is arranged in a ring
2. The server who currently has the token can access the critical region
3. The server will pass the token to the next server if it does no longer need to access critical region

Token Ring Algorithm: - Correctness: only one token - Fairness: deterministic order - Performance: If nobody has message, still pass around token. Latency is between $[0, n-1]$. - Issue: token lost due to failure. Hard to detect token lost. Difficult to add and remove server.