\documentclass[11pt]{article} \usepackage[margin=1in]{geometry} \usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} \usepackage{lmodern} \usepackage{microtype} \usepackage{amsmath,amssymb,amsthm,mathtools} \usepackage{thmtools} \usepackage{enumitem} \usepackage{csquotes} \usepackage[hidelinks]{hyperref} \usepackage[nameinlink,noabbrev]{cleveref} \usepackage{algorithm} \usepackage{algpseudocode} % Line-number prefix configuration (A/B/C) \renewcommand{\thealgorithm}{\Alph{algorithm}} % Float labels: Algorithm A, B, C \newcommand{\algletter}{} \algrenewcommand\alglinenumber[1]{\scriptsize\textbf{\algletter}#1} \usepackage{tikz} \usepackage{xspace} \usepackage[fr-FR]{datetime2} \usepackage{fancyhdr} \pagestyle{fancy} \fancyhf{} \fancyfoot[L]{Compilé le \DTMnow} \fancyfoot[C]{\thepage} \renewcommand{\headrulewidth}{0pt} \renewcommand{\footrulewidth}{0pt} \theoremstyle{plain} \newtheorem{theorem}{Theorem} \newtheorem{lemma}[theorem]{Lemma} \newtheorem{corollary}[theorem]{Corollary} \theoremstyle{definition} \newtheorem{definition}{Definition} \theoremstyle{remark} \newtheorem{remark}{Remark} \newcommand{\RB}{\textsf{RB}\xspace} \newcommand{\ARB}{\textsf{ARB}\xspace} \newcommand{\DL}{\textsf{DL}\xspace} \newcommand{\APPEND}{\textsf{APPEND}} \newcommand{\PROVE}{\textsf{PROVE}} \newcommand{\PROVEtrace}{\textsf{prove}} \newcommand{\READ}{\textsf{READ}} \newcommand{\ABbroadcast}{\textsf{AB-broadcast}} \newcommand{\ABdeliver}{\textsf{AB-deliver}} \newcommand{\RBcast}{\textsf{RB-cast}} \newcommand{\RBreceived}{\textsf{RB-received}} \newcommand{\ordered}{\textsf{ordered}} \newcommand{\Winners}{\mathsf{Winners}} \newcommand{\Messages}{\mathsf{Messages}} \newcommand{\ABlisten}{\textsf{AB-listen}} \newcommand{\delivered}{\mathsf{delivered}} \newcommand{\received}{\mathsf{received}} \newcommand{\prop}{\mathsf{prop}} \crefname{theorem}{Theorem}{Theorems} \crefname{lemma}{Lemma}{Lemmas} \crefname{definition}{Definition}{Definitions} \crefname{algorithm}{Algorithm}{Algorithms} \title{Upgrading Reliable Broadcast to Atomic Reliable Broadcast with a DenyList Primitive} \date{\vspace{-1ex}} \begin{document} % \maketitle \begin{abstract} We show how to upgrade a Reliable Broadcast (\RB) primitive to Atomic Reliable Broadcast (\ARB) by leveraging a synchronous DenyList (\DL) object. In a purely asynchronous message-passing model with crashes, \ARB is impossible without additional power. The \DL supplies this power by enabling round closing and agreement on a set of "+winners" for each round. We present the algorithm, its safety arguments, and discuss liveness and complexity under the assumed synchrony of \DL. \end{abstract} \paragraph{Keywords} Atomic broadcast, total order broadcast, reliable broadcast, consensus, synchrony, shared object, linearizability. \section{Introduction} Atomic Reliable Broadcast (\ARB)---a.k.a. total order broadcast---ensures that all processes deliver the same sequence of messages. In asynchronous message-passing systems with crashes, implementing \ARB is impossible without additional assumptions, as it enables consensus. We assume a synchronous DenyList (\DL) object and demonstrate how to combine \DL with an asynchronous \RB to realize \ARB. \section{Model} We consider a static set of $n$ processes with known identities, communicating by reliable point-to-point channels, in a complete graph. Messages are uniquely identifiable. \paragraph{Synchrony.} The network is asynchronous. Processes may crash; at most $f$ crashes occur. \paragraph{Communication.} Processes can exchange through a Reliable Broadcast (\RB) primitive (defined below) which's invoked with the functions \RBcast$(m)$ and \RBreceived$(m)$. There exists a shared object called DenyList (\DL) (defined below) that is interfaced with the functions \APPEND$(x)$, \PROVE$(x)$ and \READ$()$. \paragraph{Notation.} Let $\Pi$ be the finite set of process identifiers and let $n \triangleq |\Pi|$. Two authorization subsets are $\Pi_M \subseteq \Pi$ (processes allowed to issue \APPEND) and $\Pi_V \subseteq \Pi$ (processes allowed to issue \PROVE). Indices $i,j \in \Pi$ refer to processes, and $p_i$ denotes the process with identifier $i$. Let $\mathcal{M}$ denote the universe of uniquely identifiable messages, with $m \in \mathcal{M}$. Let $\mathcal{R} \subseteq \mathbb{N}$ be the set of round identifiers; we write $r \in \mathcal{R}$ for a round. We use the precedence relation $\prec$ for the \DL{} linearization: $x \prec y$ means that operation $x$ appears strictly before $y$ in the linearized history of \DL. For any finite set $A \subseteq \mathcal{M}$, \ordered$(A)$ returns a deterministic total order over $A$ (e.g., lexicographic order on $(\textit{senderId},\textit{messageId})$ or on message hashes). For any round $r \in \mathcal{R}$, define $\Winners_r \triangleq \{\, j \in \Pi \mid (j,\PROVEtrace(r)) \prec \APPEND(r) \,\}$, i.e., the set of processes whose $\PROVE(r)$ appears before the first $\APPEND(r)$ in the \DL{} linearization. We denoted by $\PROVE^{(j)}(r)$ or $\APPEND^{(j)}(r)$ the operation $\PROVE(r)$ or $\APPEND(r)$ invoked by process $j$. % For any round r ∈ R, define Winnersr ≜ { j ∈ Π | (j, prove(r)) ≺ APPEND(r) }. Pas bien on compare des tuples et des operations % ------------------------------------------------------------------------------ \section{Primitives} \subsection{Reliable Broadcast (RB)} \RB provides the following properties in the model. \begin{itemize}[leftmargin=*] \item \textbf{Integrity}: Every message received was previously sent. $\forall p_i:\ \RBreceived_i(m) \Rightarrow \exists p_j:\ \RBcast_j(m)$. \item \textbf{No-duplicates}: No message is received more than once at any process. \item \textbf{Validity}: If a correct process broadcasts $m$, every correct process eventually receives $m$. \end{itemize} \subsection{DenyList (DL)} The \DL is a \emph{shared, append-only} object that records attestations about opaque application-level tokens. It exposes the following operations: \begin{itemize}[leftmargin=*] \item \APPEND$(x)$ \item \PROVE$(x)$: issue an attestation for token $x$; this operation is \emph{valid} (return true) only if no \APPEND$(x)$ occurs earlier in the \DL linearization. Otherwise, it is invalid (return false). \item \READ$()$: return a (permutation of the) valid operations observed so far; subsequent reads are monotone (contain supersets of previously observed valid operations). \end{itemize} \paragraph{Validity.} \APPEND$(x)$ is valid iff the issuer is authorized (in $\Pi_M$) and $x$ belongs to the application-defined domain $S$. \PROVE$(x)$ is valid iff the issuer is authorized (in $\Pi_V$) and there is no earlier \APPEND$(x)$ in the \DL linearization. \paragraph{Progress.} If a correct process invokes \APPEND$(x)$, then eventually all correct processes will be unable to issue a valid \PROVE$(x)$, and \READ{} at all correct processes will (eventually) reflect that \APPEND$(x)$ has been recorded. \paragraph{Termination.} Every operation invoked by a correct process eventually returns. \paragraph{Interface and Semantics.} The \DL provides a single global linearization of operations consistent with each process's program order. \READ{} is prefix-monotone; concurrent updates become visible to all correct processes within bounded time (by synchrony). Duplicate requests may be idempotently coalesced by the implementation. % ------------------------------------------------------------------------------ \section{Target Abstraction: Atomic Reliable Broadcast (ARB)} Processes export \ABbroadcast$(m)$ and \ABdeliver$(m)$. \ARB requires total order: \begin{equation*} \forall m_1,m_2,\ \forall p_i,p_j:\ \ \ABdeliver_i(m_1) < \ABdeliver_i(m_2) \Rightarrow \ABdeliver_j(m_1) < \ABdeliver_j(m_2), \end{equation*} plus Integrity/No-duplicates/Validity (inherited from \RB and the construction). % ------------------------------------------------------------------------------ \section{Algorithm} % granularité diff commentaire de code et paragraphe pre algo \begin{definition}[Closed round]\label{def:closed-round} Given a \DL{} linearization $H$, a round $r\in\mathcal{R}$ is \emph{closed} in $H$ iff $H$ contains an operation $\APPEND(r)$. Equivalently, there exists a time after which every $\PROVE(r)$ is invalid in $H$. \end{definition} \subsection{Variables} Each process $p_i$ maintains: %on met toutes les variables locales ici \begin{algorithmic} \State $\received \gets \emptyset$ \Comment{Messages received via \RB but not yet delivered} \State $\delivered \gets \emptyset$ \Comment{Messages already delivered} \State $\prop[r][j] \gets \bot,\ \forall r,j$ \Comment{Proposal from process $j$ for round $r$} \end{algorithmic} \paragraph{DenyList.} The \DL is initialized empty. We assume $\Pi_M = \Pi_V = \Pi$ (all processes can invoke \APPEND and \PROVE). \subsection{Handlers and Procedures} \renewcommand{\algletter}{A} \begin{algorithm}[H] \caption{\RB handler (at process $p_i$)}\label{alg:rb-handler} \begin{algorithmic}[1] \Function{RBreceived}{$S, r, j$} % \State \textbf{on} $\RBreceived(S, r, j)$ \textbf{do} \State $\received \leftarrow \received \cup \{S\}$ \State $\prop[r][j] \leftarrow S$ \Comment{Record sender $j$'s proposal $S$ for round $r$} \EndFunction \end{algorithmic} \end{algorithm} % \paragraph{} An \ABbroadcast$(m)$ chooses the next open round from the \DL view, proposes all pending messages together with the new $m$, disseminates this proposal via \RB, then executes $\PROVE(r)$ followed by $\APPEND(r)$ to freeze the winners of the round. The loop polls \DL until (i) some winner’s proposal includes $m$ in a \emph{closed} round and (ii) all winners' proposals for closed rounds are known locally, ensuring eventual inclusion and delivery. % Partie avec le max-1 pas ouf; essayer de faire incr la boucle autrement \renewcommand{\algletter}{B} \begin{algorithm}[H] \caption{\ABbroadcast$(m)$ (at process $p_i$)}\label{alg:ab-bcast} \begin{algorithmic}[1] \Function{ABbroadcast}{$m$} \State $P \leftarrow \READ()$ \Comment{Fetch latest \DL state to learn recent $\PROVE$ operations} \State $r_{max} \leftarrow \max(\{ r' : \exists j,\ (j,\PROVE(r')) \in P \})$ \Comment{Pick current open round} \State $S \leftarrow (\received \setminus \delivered) \cup \{m\}$ \Comment{Propose all pending messages plus the new $m$} \vspace{1em} \For{\textbf{each}\ $r \in \{r_{max}, r_{max}+1, \cdots \}$} \State $\RBcast(S, r, i)$; $\PROVE(r)$; $\APPEND(r)$; \State $P \leftarrow \READ()$ \Comment{Refresh local view of \DL} \If{($\big((i, \PROVEtrace(r)) \in P\ \vee\ (\exists j, r': (j, \PROVEtrace(r')) \in P \wedge \ m \in \prop[r'][j]))$)} \State \textbf{break} \Comment{Exit loop once $m$ is included in some closed round} \EndIf \EndFor \EndFunction \end{algorithmic} \end{algorithm} % \paragraph{} TODO \renewcommand{\algletter}{C} \begin{algorithm}[H] \caption{\ABdeliver() at process $p_i$}\label{alg:delivery} \begin{algorithmic}[1] %local variables \State $next \gets 0$ \Comment{Next round to deliver} \State $to\_deliver \gets \emptyset$ \Comment{Queue of messages ready to be delivered} \vspace{1em} \Function{ABdeliver}{} % \State \textbf{on} \ABdeliver() \textbf{do} \Comment{Called when the process wants to receive the next message} \If{$to\_deliver = \emptyset$} \Comment{If no message is ready to deliver, try to fetch the next round} \State $P \leftarrow \READ()$ \Comment{Fetch latest \DL state to learn recent $\PROVE$ operations} \If{$\forall j : (j, \PROVEtrace(next)) \not\in P$} \Comment{Check if some process proved round $next$} \State \Return $\bot$ \Comment{Round $next$ is still open} \EndIf \State $\APPEND(next)$; $P \leftarrow \READ()$ \Comment{Close round $next$ if not already closed} \State $W_{next} \leftarrow \{ j : (j, \PROVEtrace(next)) \in P \}$ \Comment{Compute winners of round $next$} \If{$\exists j \in W_{next},\ \prop[next][j] = \bot$} \Comment{Check if we have all winners' proposals} \State \Return $\bot$ \Comment{Some winner's proposal for round $next$ is still missing} \EndIf \State $M_{next} \leftarrow \bigcup_{j \in W_{next}} \prop[next][j]$ \Comment{Compute the agreed proposal for round $next$} \For{\textbf{each}\ $m \in \ordered(M_{next})$} \Comment{Enqueue messages in deterministic order} \If{$m \notin \delivered$} \State $to\_deliver.push(m)$ \Comment{Append $m$ to the delivery queue} \EndIf \EndFor \State $next \leftarrow next + 1$ \Comment{Advance to the next round} \EndIf \State $m \leftarrow \text{to\_deliver.pop()}$ % \State $to\_deliver \leftarrow to\_deliver \setminus \{m\}$ \State $\delivered \leftarrow \delivered \cup \{m\}$ \State \textbf{return} $m$ \EndFunction \end{algorithmic} \end{algorithm} % ------------------------------------------------------------------------------ \section{Correctness} %attention au usage de "unique" %definition de APPEND* ssi r closed \begin{lemma}[Stable round closure]\label{lem:closure-stable} If a round $r$ is closed, then there exists a linearization point $t_0$ of $\APPEND(r)$ in the \DL, and from that point on, no $\PROVE(r)$ can be valid. Once closed, a round never becomes open again. \end{lemma} \begin{proof} By \Cref{def:closed-round}, some $\APPEND(r)$ occurs in the linearization $H$. \\ $H$ is a total order of operations, the set of $\APPEND(r)$ operations is totally ordered, and hence there exists a smallest $\APPEND(r)$ in $H$. We denote this operation $\APPEND^{(\star)}(r)$ and $t_0$ its linearization point. \\ By the validity property of \DL, a $\PROVE(r)$ is valid iff $\PROVE(r) \prec \APPEND^{(\star)}(r)$. Thus, after $t_0$, no $\PROVE(r)$ can be valid. \\ $H$ is a immutable grow-only history, and hence once closed, a round never becomes open again. \\ Hence there exists a linearization point $t_0$ of $\APPEND(r)$ in the \DL, and from that point on, no $\PROVE(r)$ can be valid and the closure is stable. \end{proof} \begin{definition}[First APPEND]\label{def:first-append} Given a \DL{} linearization $H$, for any closed round $r\in\mathcal{R}$, we denote by $\APPEND^{(\star)}(r)$ the earliest $\APPEND(r)$ in $H$. \end{definition} \begin{lemma}[Across rounds]\label{lem:across} If there exists a $r$ such that $r$ is closed, $\forall r'$ such that $r' < r$, r' is also closed. \end{lemma} \begin{proof} \emph{Base.} For a closed round $k=0$, the set $\{k' \in \mathcal{R},\ k' < k\}$ is empty, hence the lemma is true. \emph{Induction.} Assume the lemma is true for round $k\geq 0$, we prove it for round $k+1$. \smallskip Assume $k+1$ is closed and let $\APPEND^{(\star)}(k+1)$ be the earliest $\APPEND(k+1)$ in the DL linearization $H$. By Lemma 1, after an $\APPEND(k)$ is in $H$, any later $\PROVE(k)$ is rejected also, a process’s program order is preserved in $H$. There are two possibilities for where $\APPEND^{(\star)}(k+1)$ is invoked. \begin{itemize} \item \textbf{Case (B6) :} Some process $p^\star$ executes the loop (lines B5–B11) and invokes $\APPEND^{(\star)}(k+1)$ at line B6. Immediately before line B6, line B5 sets $r\leftarrow r+1$, so the previous loop iteration (if any) targeted $k$. We consider two sub-cases. \begin{itemize} \item \emph{(i) $p^\star$ is not in its first loop iteration.} In the previous iteration, $p^\star$ executed $\PROVE^{(\star)}(k)$ at B6, followed (in program order) by $\APPEND^{(\star)}(k)$. If round $k$ wasn't closed when $p^\star$ execute $\PROVE^{(\star)}(k)$ at B9, then the condition at B8 would be true hence the tuple $(p^\star, \PROVEtrace(k))$ should be visible in P which implies that $p^\star$ should leave the loop at round $k$, contradicting the assumption that $p^\star$ is now executing another iteration. Since the tuple is not visible, the $\PROVE^{(\star)}(k)$ was rejected by the DL which implies by definition an $\APPEND(k)$ already exists in $H$. Thus in this case $k$ is closed. \item \emph{(ii) $p^\star$ is in its first loop iteration.} To compute the value $r_{max}$, $p^\star$ must have observed one or many $(\_ , \PROVEtrace(k+1))$ in $P$ at B2/B3, issued by some processes (possibly different from $p^\star$). Let's call $p_1$ the issuer of the first $\PROVE^{(1)}(k+1)$ in the linearization $H$. \\ When $p_1$ executed $P \gets \READ()$ at B2 and compute $r_{max}$ at B3, he observed no tuple $(\_,\PROVEtrace(k+1))$ in $P$ because he's the issuer of the first one. So when $p_1$ executed the loop (B5–B11), he run it for the round $k$, didn't seen any $(1,\PROVEtrace(k))$ in $P$ at B8, and then executed the first $\PROVE^{(1)}(k+1)$ at B6 in a second iteration. \\ If round $k$ wasn't closed when $p_1$ execute $\PROVE^{(1)}(k)$ at B6, then the condition at B8 should be true which implies that $p_1$ sould leave the loop at round $k$, contradicting the assumption that $p_1$ is now executing $\PROVE^{(1)}(r+1)$. In this case $k$ is closed. \end{itemize} \item \textbf{Case (C9) :} Some process invokes $\APPEND(k+1)$ at C9. Line C9 is guarded by the presence of $\PROVE(\textit{next})$ in $P$ with $\textit{next}=k+1$ (C6). Moreover, the local pointer $\textit{next}$ grow by increment of 1 and only advances after finishing the current round (C20), so if a process can reach $\textit{next}=k+1$ it implies that he has completed round $k$, which includes closing $k$ at C9 when $\PROVE(k)$ is observed. Hence $\APPEND^\star(k+1)$ implies a prior $\APPEND(k)$ in $H$, so $k$ is closed. \end{itemize} \smallskip In all cases, $k+1$ closed implie $k$ closed. By induction on $k$, if the lemme is true for a closed $k$ then it is true for a closed $k+1$. Therefore, the lemma is true for all closed rounds $r$. \end{proof} \begin{definition}[Winner Invariant]\label{def:winner-invariant} For any closed round $r$, define \[ \Winners_r \triangleq \{ j : \PROVE^{(j)}(r) \prec \APPEND^\star(r) \} \] as the unique set of winners of round $r$. \end{definition} \begin{lemma}[Invariant view of closure]\label{lem:closure-view} For any closed round $r$, all correct processes eventually observe the same set of valid tuples $(\_,\PROVEtrace(r))$ in their \DL view. \end{lemma} \begin{proof} Let's take a closed round $r$. By \Cref{def:first-append}, there exists a unique earliest $\APPEND(r)$ in the DL linearization, denoted $\APPEND^\star(r)$. Consider any correct process $p$ that invokes $\READ()$ after $\APPEND^\star(r)$ in the DL linearization. Since $\APPEND^\star(r)$ invalidates all subsequent $\PROVE(r)$, the set of valid tuples $(\_,\PROVEtrace(r))$ observed by any correct process after $\APPEND^\star(r)$ is fixed and identical across all correct processes. Therefore, for any closed round $r$, all correct processes eventually observe the same set of valid tuples $(\_,\PROVEtrace(r))$ in their \DL view. \end{proof} \begin{lemma}[Well-defined winners]\label{lem:winners} For any correct process and round $r$, if the process computes $W_r$ at line C10, then : \begin{itemize} \item $\Winners_r$ is defined; \item the computed $W_r$ is exactly $\Winners_r$. \end{itemize} \end{lemma} \begin{proof} Let take a correct process $p_i$ that reach line C10 to compute $W_r$. \\ By program order, $p_i$ must have executed $\APPEND^{(i)}(r)$ at C9 before, which implies by \Cref{def:closed-round} that round $r$ is closed. So by \Cref{def:winner-invariant}, $\Winners_r$ is defined. \\ By \Cref{lem:closure-view}, all correct processes eventually observe the same set of valid tuples $(\_,\PROVEtrace(r))$ in their \DL view. Hence, when $p_i$ executes the $\READ()$ at C9 after the $\APPEND^{(i)}(r)$, it observes a set $P$ that includes all valid tuples $(\_,\PROVEtrace(r))$ such that \[ W_r = \{ j : (j,\PROVEtrace(r)) \in P \} = \{j : \PROVE^{(j)}(r) \prec \APPEND^\star(r) \} = \Winners_r \] \end{proof} \begin{lemma}[No APPEND without PROVE]\label{lem:append-prove} If some process invokes $\APPEND(r)$, then at least a process must have previously invoked $\PROVE(r)$. \end{lemma} \begin{proof}[Proof] Consider the round $r$ such that some process invokes $\APPEND(r)$. There are two possible cases \begin{itemize} \item \textbf{Case (B6) :} There exists a process $p^\star$ who's the issuer of the earliest $\APPEND^{(\star)}(r)$ in the DL linearization $H$. By program order, $p^\star$ must have previously invoked $\PROVE^{(\star)}(r)$ before invoking $\APPEND^{(\star)}(r)$ at B6. In this case, there is at least one $\PROVE(r)$ valid in $H$ issued by a correct process before $\APPEND^{(\star)}(r)$. \item \textbf{Case (C9) :} There exist a process $p^\star$ invokes $\APPEND^{(\star)}(r)$ at C9. Line C9 is guarded by the condition at C6, which ensures that $p$ observed some $(\_,\PROVEtrace(r))$ in $P$. In this case, there is at least one $\PROVE(r)$ valid in $H$ issued by some process before $\APPEND^{(\star)}(r)$. \end{itemize} In both cases, if some process invokes $\APPEND(r)$, then some process must have previously invoked $\PROVE(r)$. \end{proof} \begin{lemma}[No empty winners]\label{lem:nonempty} Let $r$ be a round, if $\Winners_r$ is defined, then $\Winners_r \neq \emptyset$. \end{lemma} \begin{proof}[Proof] If $\Winners_r$ is defined, then by \Cref{def:winner-invariant}, round $r$ is closed. By \Cref{def:closed-round}, some $\APPEND(r)$ occurs in the DL linearization. \\ By \Cref{lem:append-prove}, at least a process must have invoked a valid $\PROVE(r)$ before $\APPEND^{(\star)}(r)$. Hence, there exists at least one $j$ such that $\{j: \PROVE^{(j)}(r) \prec \APPEND^\star(r)\}$, so $\Winners_r \neq \emptyset$. \end{proof} \begin{lemma}[Winners must propose]\label{lem:winners-propose} For any closed round $r$, $\forall j \in \Winners_r$, process $j$ must have invoked a $\RBcast(S^{(j)}, r, j)$ \end{lemma} \begin{proof}[Proof] Fix a closed round $r$. By \Cref{def:winner-invariant}, for any $j \in \Winners_r$, there exist a valid $\PROVE^{(j)}(r)$ such that $\PROVE^{(j)}(r) \prec \APPEND^\star(r)$ in the DL linearization. By program order, if $j$ invoked a valid $\PROVE^{(j)}(r)$ at line B6 he must have invoked $\RBcast(S^{(j)}, r, j)$ directly before. \end{proof} \begin{definition}[Messages invariant]\label{def:messages-invariant} For any closed round $r$ and any correct process $p_i$ such that $\nexists j \in \Winners_r : prop^{[i)}[r][j] = \bot$, define \[ \Messages_r \triangleq \bigcup_{j\in\Winners_r} \prop^{(i)}[r][j] \] as the unique set of messages proposed by the winners of round $r$. \end{definition} \begin{lemma}[Non-empty winners proposal]\label{lem:winner-propose-nonbot} For any closed round $r$, $\forall j \in \Winners_r$, for any correct process $p_i$, eventually $\prop^{(i)}[r][j] \neq \bot$. \end{lemma} \begin{proof}[Proof] Fix a closed round $r$. By \Cref{def:winner-invariant}, for any $j \in \Winners_r$, there exist a valid $\PROVE^{(j)}(r)$ such that $\PROVE^{(j)}(r) \prec \APPEND^\star(r)$ in the DL linearization. By \Cref{lem:winners-propose}, $j$ must have invoked $\RBcast(S^{(j)}, r, j)$. Let take a process $p_i$, by \RB \emph{Validity}, every correct process eventually receives $j$'s \RB message for round $r$, which sets $\prop[r][j]$ to a non-$\bot$ value at line A3. \end{proof} \begin{lemma}[Eventual proposal closure]\label{lem:eventual-closure} If a correct process $p_i$ define $M_r$ at line C14, then for every $j \in \Winners_r$, $\prop^{(i)}[r][j] \neq \bot$. \end{lemma} \begin{proof}[Proof] Let take a correct process $p_i$ that computes $M_r$ at line C14. By \Cref{lem:winners}, $p_i$ computes the unique winner set $\Winners_r$. By \Cref{lem:nonempty}, $\Winners_r \neq \emptyset$. The instruction at line C14 where $p_i$ computes $M_r$ is guarded by the condition at C11, which ensures that $p_i$ has received all \RB messages from every winner $j \in \Winners_r$. Hence, when $p_i$ computes $M_r = \bigcup_{j\in\Winners_r} \prop^{(i)}[r][j]$, we have $\prop^{(i)}[r][j] \neq \bot$ for all $j \in \Winners_r$. \end{proof} \begin{lemma}[Unique proposal per sender per round]\label{lem:unique-proposal} For any round $r$ and any process $p_i$, $p_i$ invokes at most one $\RBcast(S, r, i)$. \end{lemma} \begin{proof}[Proof] By program order, any process $p_i$ invokes $\RBcast(S, r, i)$ at line B6 must be in the loop B5–B11. No matter the number of iterations of the loop, line B5 always uses the current value of $r$ which is incremented by 1 at each turn. Hence, in any execution, any process $p_i$ invokes $\RBcast(S, r, j)$ at most once for any round $r$. \end{proof} \begin{lemma}[Proposal convergence]\label{lem:convergence} For any round $r$, for any correct processes $p_i$ that define $M_r$ at line C14, we have \[ M_r^{(i)} = \Messages_r \] \end{lemma} \begin{proof}[Proof] Let take a correct process $p_i$ that define $M_r$ at line C14. That implies that $p_i$ has defined $W_r$ at line C10. It implies that, by \Cref{lem:winners}, $r$ is closed and $W_r = \Winners_r$. \\ By \Cref{lem:eventual-closure}, for every $j \in \Winners_r$, $\prop^{(i)}[r][j] \neq \bot$. By \Cref{lem:unique-proposal}, each winner $j$ invokes at most one $\RBcast(S^{(j)}, r, j)$, so $\prop^{(i)}[r][j] = S^{(j)}$ is uniquely defined. Hence, when $p_i$ computes \[ M_r^{(i)} = \bigcup_{j\in\Winners_r} \prop^{(i)}[r][j] = \bigcup_{j\in\Winners_r} S^{(j)} = \Messages_r. \] \end{proof} \begin{lemma}[Inclusion]\label{lem:inclusion} If some correct process invokes $\ABbroadcast(m)$, then there exist a round $r$ and a process $j\in\Winners_r$ such that $p_j$ invokes \[ \RBcast(S, r, j)\quad\text{with}\quad m\in S. \] \end{lemma} \begin{proof} Fix a correct process $p_i$ that invokes $\ABbroadcast(m)$ and eventually exits the loop (B5–B11) at some round $r$. There are two possible cases. \begin{itemize} \item \textbf{Case 1:} $p_i$ exits the loop because $(i, \PROVEtrace(r)) \in P$. In this case, by \Cref{def:winner-invariant}, $p_i$ is a winner in round $r$. By program order, $p_i$ must have invoked $\RBcast(S, r, i)$ at B6 before invoking $\PROVE^{(i)}(r)$ at B7. By line B4, $m \in S$. Hence there exist a closed round $r$ and a correct process $j=i\in\Winners_r$ such that $j$ invokes $\RBcast(S, r, j)$ with $m\in S$. \item \textbf{Case 2:} $p_i$ exits the loop because $\exists j, r': (j, \PROVEtrace(r')) \in P \wedge m \in \prop[r'][j]$. In this case, by \Cref{lem:winners-propose} and \Cref{lem:unique-proposal} $j$ must have invoked a unique $\RBcast(S, r', j)$. Which set $\prop^{(i)}[r'][j] = S$ with $m \in S$. \end{itemize} In both cases, if some correct process invokes $\ABbroadcast(m)$, then there exist a round $r$ and a correct process $j\in\Winners_r$ such that $j$ invokes \[ \RBcast(S, r, j)\quad\text{with}\quad m\in S. \] \end{proof} \begin{lemma}[Broadcast Termination]\label{lem:bcast-termination} If a correct process $p$ invokes $\ABbroadcast(m)$, then $p$ eventually quit the function and returns. \end{lemma} \begin{proof}[Proof] Let a correct process $p_i$ that invokes $\ABbroadcast(m)$. The lemma is true if $(i, \PROVEtrace(r)) \in P$; or there exists $j: (j, \PROVEtrace(r')) \in P \wedge m \in \prop[r'][j]$ (like guarded at B8). \begin{itemize} \item \textbf{Case 1:} there exists a round $r'$ such that $p_i$ invokes a valid $\PROVE(r')$. Hence by \DL \emph{Progress} and \emph{Semantics} the following $\READ()$ at line B7 will defined a P such as $(i, \PROVEtrace(r')) \in P$. Hence $p_i$ exits the loop at B8. \item \textbf{Case 2:} there exists no round $r'$ such that $p_i$ invokes a valid $\PROVE(r')$. In this case $p_i$ invokes infinitely many $\RBcast(S, r', i)$ at B6 with $m \in S$ (line B4).\\ The assumption that no $\PROVE(r')$ invoked by $p$ is valid implies by \DL \emph{Validity} that for every round $r'$, there exists at least one $\APPEND(r')$ in the DL linearization, hence by \Cref{lem:nonempty} for every possible round $r'$ there at least a winner. Because there is an infinite number of rounds, and a finite number of processes, there exists at least a correct process $p_k$ that invokes infinitely many valid $\PROVE(r')$ and by extension infinitely many $\ABbroadcast(\_)$. By \RB \emph{Validity}, $p_k$ eventually receives $p_i$ 's \RB messages. Let call $t_0$ the time when $p_k$ receives $p_i$ 's \RB message. \\ At $t_0$, $p_k$ execute \Cref{alg:rb-handler} and do $\received \leftarrow \received \cup \{S\}$ with $m \in S$ (line A2). For the first invocation of $\ABbroadcast(\_)$ by $p_k$ after the execution of \Cref{alg:rb-handler}, $p_k$ must include $m$ in his proposal $S$ at line B4 (because $m$ is pending in $j$'s $\received \setminus \delivered$ set). There exists a minimum round $r_1$ such that $p_k \in \Winners_{r_1}$ after $t_0$. By \Cref{lem:winner-propose-nonbot}, eventually $\prop^{(i)}[r_1][k] \neq \bot$. By \Cref{lem:unique-proposal}, $\prop^{(i)}[r_1][k]$ is uniquely defined as the set $S$ proposed by $p_k$ at B6, which by \Cref{lem:inclusion} includes $m$. Hence eventually $m \in \prop^{(i)}[r_1][k]$ and $k \in \Winners_{r_1}$. \end{itemize} The first case explicit the case where $p_i$ is a winner and also covers the condition $(i, \PROVEtrace(r')) \in P$. And in the second case, we show that if the first condition is never satisfied, the second one will eventually be satisfied. Hence either the first or the second condition will eventually be satisfied, and $p_i$ eventually exits the loop and returns from $\ABbroadcast(m)$. \end{proof} \begin{lemma}[Validity]\label{lem:validity} If a correct process $p$ invokes $\ABbroadcast(m)$, then every correct process that invokes a infinitely often times $\ABdeliver()$ eventually delivers $m$. \end{lemma} \begin{proof}[Proof] Let $p_i$ a correct process that invokes $\ABbroadcast(m)$ and $p_q$ a correct process that infinitely invokes $\ABdeliver()$. By \Cref{lem:inclusion}, there exist a closed round $r$ and a correct process $j\in\Winners_r$ such that $p_j$ invokes \[ \RBcast(S, r, j)\quad\text{with}\quad m\in S. \] By \Cref{lem:eventual-closure}, when $p_q$ computes $M_r$ at line C14, $\prop[r][j]$ is non-$\bot$ because $j \in \Winners_r$. By \Cref{lem:unique-proposal}, $p_j$ invokes at most one $\RBcast(S, r, j)$, so $\prop[r][j]$ is uniquely defined. Hence, when $p_q$ computes \[ M_r = \bigcup_{k\in\Winners_r} \prop[r][k], \] we have $m \in \prop[r][j] = S$, so $m \in M_r$. By line C16–C18, $m$ is enqueued into $to\_deliver$ unless it has already been delivered. Therefore, $p_q$ will eventually invokes $\ABdeliver()$, which will returns $m$. \end{proof} \begin{lemma}[Across \ABdeliver]\label{lem:across-abdeliver} A $\ABdeliver()$ invocation return $m$, a non-$\bot$ value, implie that there exist an invocation of $\ABdeliver()$ which is the earliest of the round $r$ where $m$ is proposed by a winner and which define $M_r$. \end{lemma} \begin{proof} Let take any $\ABdeliver()$ invocation which return a non-$\bot$ value, $m$ which was proposed in round $r$. To return a non-$\bot$ value we can distinguish two cases: \begin{itemize} \item This actual invocation of $\ABdeliver$ have the queue $to\_deliver \neq \emptyset$ which implies that there exist an earliest invocation of $\ABdeliver$ which reach the line C17 to fill this queue for round $next = r$. We can call this invocation $\ABdeliver^{(\star r)}$. $\ABdeliver^{(\star r)}$ have to fill $to\_deliver$ at line C17, hence by program order define $M_r$ at C14. \item This actual invocation of $\ABdeliver$ have the queue $to\_deliver = \emptyset$. To return a non-$\bot$ this execution have to pass the two conditions at lines C6 and C11 to compute $m_r$. Hence $\ABdeliver()$ is the $\ABdeliver^{(\star r)}()$ as described in the first case. \end{itemize} \end{proof} \begin{definition}[First \ABdeliver]\label{def:first-abdeliver} For any process which invoke an infinite times $\ABdeliver()$. There exist for any round $r$ an unique earliest invocation which defined $M_r$ and return a non-$\bot$ value. We denote by $\ABdeliver^{(\star r)}()$ this invocation. \end{definition} \begin{lemma}[No duplication]\label{lem:no-duplication} No correct process delivers the same message more than once. \end{lemma} \begin{proof} Let consider two invokations of $\ABdeliver()$ made by the same correct process which returns respectively $m_1$ and $m_2$. Let call these two invokations respectively $\ABdeliver^{(A)}()$ and $\ABdeliver^{(B)}()$. By \Cref{def:first-abdeliver} we denote $\ABdeliver^{(\star A)}()$ and $\ABdeliver^{(\star B)}()$ the two earliest invocations of each $\ABdeliver()$ respectively in theirs rounds with $A$ the round where $m_1$ is delivered and $B$ the round where $m_2$ is delivered. Let consider the following cases : \begin{itemize} \item \textbf{$A < B$ :} Let consider $m_1 = m_2 = m$. In the execution of $\ABdeliver^{(\star A)}$ the process iterate on line C17 and push $m$ in $to\_deliver$ since $m$ is not in $\delivered$. When the process invoke later $\ABdeliver^{(\star B)}$ he empties the queue. The only way to empties the queue is at line C22 which implie by program order to add $m$ to the $\delivered$ set. Hence when $\ABdeliver^{(\star B)}$ iterate on line C17 he will never be able to push $m$ in the queue because the condition $m \not\in \delivered$ at line C16 is never satisfied. Hence in this case $\ABdeliver^{(B)}()$ can't happen if $m_1 = m_2$. \item \text{$A = B$ :} Let consider $m_1 = m_2 = m$. $\ABdeliver^{(\star A)}$ and $\ABdeliver^{(\star B)}$ reference the same invocation of $\ABdeliver()$. In this unique execution, when the process define $M_r$ he's making a union operation on all the sets which can contains a multiple times the same message $m$. This operation must result in a unique set which contain a unique time $m$. Hence when the iteration on C17 is done, $m$ is pushed only once, and can be delivered only once too. So $\ABdeliver^{(B)}()$ can't happen if $m_1 = m_2$. \end{itemize} \end{proof} \begin{lemma}[Total order]\label{lem:total-order} For any two messages $m_1$ and $m_2$ delivered by correct processes, if a correct process $p_i$ delivers $m_1$ before $m_2$, then any correct process $p_j$ that delivers both $m_1$ and $m_2$ delivers $m_1$ before $m_2$. \end{lemma} \begin{proof} Consider any correct process that delivers both $m_1$ and $m_2$. By \Cref{lem:validity}, there exist closed rounds $r'_1$ and $r'_2$ and correct processes $k_1 \in \Winners_{r'_1}$ and $k_2 \in \Winners_{r'_2}$ such that \[ \RBcast(S_1, r'_1, k_1)\quad\text{with}\quad m_1\in S_1, \] \[ \RBcast(S_2, r'_2, k_2)\quad\text{with}\quad m_2\in S_2. \] Let consider three cases : \begin{itemize} \item \textbf{Case 1:} $r_1 < r_2$. By program order, any correct process must have waited to empty the queue $to\_deliver$ (which contains $m_1$) to exit at round $r_1$ before invoking $\ABdeliver()$ that returns $m_2$. Hence, for any correct process that delivers both $m_1$ and $m_2$, it delivers $m_1$ before $m_2$. \item \textbf{Case 2:} $r_1 = r_2$. By \Cref{lem:convergence}, any correct process that computes $M_{r_1}$ at line C14 computes the same set of messages $\Messages_{r_1}$. By line C16–C18, messages are enqueued into $to\_deliver$ in a deterministic order defined by $\ordered(\_)$. Hence, for any correct process that delivers both $m_1$ and $m_2$, it delivers $m_1$ and $m_2$ in the deterministic order defined by $\ordered(\_)$. \end{itemize} In all possible cases, any correct process that delivers both $m_1$ and $m_2$ delivers $m_1$ and $m_2$ in the same order. \end{proof} \begin{lemma}[Fifo Order]\label{lem:fifo-order} For any two messages $m_1$ and $m_2$ broadcast by the same correct process $p_i$, if $p_i$ invokes $\ABbroadcast(m_1)$ before $\ABbroadcast(m_2)$, then any correct process $p_j$ that delivers both $m_1$ and $m_2$ delivers $m_1$ before $m_2$. \end{lemma} \begin{proof} Let take two messages $m_1$ and $m_2$ broadcast by the same correct process $p_i$, with $p_i$ invoking $\ABbroadcast(m_1)$ before $\ABbroadcast(m_2)$. By \Cref{lem:validity}, there exist closed rounds $r_1$ and $r_2$ and correct processes $k_1 \in \Winners_{r_1}$ and $k_2 \in \Winners_{r_2}$ such that \[ \RBcast(S_1, r_1, k_1)\quad\text{with}\quad m_1\in S_1, \] \[ \RBcast(S_2, r_2, k_2)\quad\text{with}\quad m_2\in S_2. \] By program order, $p_i$ must have invoked $\RBcast(S_1, r_1, i)$ before $\RBcast(S_2, r_2, i)$. By \Cref{lem:unique-proposal}, any process invokes at most one $\RBcast(S, r, i)$ per round, hence $r_1 < r_2$. By \Cref{lem:total-order}, any correct process that delivers both $m_1$ and $m_2$ delivers them in a deterministic order. In all possible cases, any correct process that delivers both $m_1$ and $m_2$ delivers $m_1$ before $m_2$. \end{proof} \begin{theorem}[FIFO-\ARB] Under the assumed \DL synchrony and \RB reliability, the algorithm implements FIFO Atomic Reliable Broadcast. \end{theorem} \bibliographystyle{plain} \begin{thebibliography}{9} % (left intentionally blank) \end{thebibliography} \end{document}