I am going to ignore rotations in order to simplify the problem for your understanding. You have to enforce a series of inelastic relationship of the form
$$\vec{n}_{k}^\top (\vec{v}_i^+-\vec{v}_j^+) = 0 $$
where $\vec{n}_k$ is the normal direction of the $k$-th contact, and $i$, $j$ are the bodies this contact affects. The superscript $\phantom{c}^+$ denotes condition after the impact. You enforce this relationship with a series of $k$ impulses $J_k$ such that
$$ \vec{v}_i^+ = \vec{v}_i + \frac{\vec{n} J_k}{m_i} $$
$$ \vec{v}_j^+ = \vec{v}_j - \frac{\vec{n} J_k}{m_j} $$
Since it all has to happen at the same time it is best to form the problem with matrices.
Consider a Contact matrix $A$ where each column $k$ has +1 in the $i$-th row and -1 in the $j$-th row. For example $A = \begin{pmatrix}0&-1 \\ -1 & 0 \\ 0 & 0 \\ 1 & 1 \end{pmatrix}$ means there are two contacts, one between body 2 and body 4 and another between body 1 and 4. (actually each 0 and 1 are 2×2 for 2D or 3×3 for 3D zero and identity matrices).
The inelastic relationships are
$$N^\top A^\top v^+ =0$$ with the contact normal block diagonal matrix
$$ N = \begin{pmatrix} \vec{n}_1 & 0 & \cdots & 0 \\ 0 & \vec{n}_2 & & 0 \\ \vdots & & & \vdots \\ 0 & 0 & \cdots & \vec{n}_K \end{pmatrix} $$ and
$$ v = \begin{pmatrix} v_1 \\ v_2 \\ \vdots \\ v_N \end{pmatrix} $$
The momentum exchange is described by the relationship
$$ M v^+ = M v - A N J $$ where $M$ is the block diagonal mass matrix $M=\begin{pmatrix}m_1& & & \\ &m_2 & & \\& & \ddots & \\ & & & m_N\end{pmatrix}$ and $J$ the vector of impulses $J^\top=(J_1\,J_2\,\cdots J_K)$
To solve the problem we combine the momentum with the inelastic collisions to get
$$ v^+ = v - M^{-1} A N J $$
$$ N^\top A^\top \left(v - M^{-1} A N J \right) = 0$$
$$ \left(N^\top A^\top M^{-1} A N\right) J = N^\top A v $$
$$ \boxed{ J = \left(N^\top A^\top M^{-1} A N\right)^{-1} N^\top A^\top v }$$
Example
With $A$ as above (4 2D bodies, 2 contacts) and $\vec{v}_i = (\dot{x}_i,\dot{y}_i)^\top$, $\vec{n}_1=(1,0)^\top$, $\vec{n}_2 = (0,1)^\top$ then
$$ A = \left(\begin{array}{cc|cc} 0 & 0 & -1 & 0\\ 0 & 0 & 0 & -1\\ \hline -1 & 0 & 0 & 0\\ 0 & -1 & 0 & 0\\ \hline 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0\\ \hline 1 & 0 & 1 & 0\\ 0 & 1 & 0 & 1 \end{array}\right) $$
$$ N = \left(\begin{array}{c|c} 1 & 0\\ 0 & 0\\ \hline 0 & 0\\ 0 & 1 \end{array}\right) $$
$$ M = \left(\begin{array}{cc|cc|cc|cc} m_{1} & 0 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & m_{1} & 0 & 0 & 0 & 0 & 0 & 0\\ \hline 0 & 0 & m_{2} & 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & m_{2} & 0 & 0 & 0 & 0\\ \hline 0 & 0 & 0 & 0 & m_{3} & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0 & m_{3} & 0 & 0\\ \hline 0 & 0 & 0 & 0 & 0 & 0 & m_{4} & 0\\ 0 & 0 & 0 & 0 & 0 & 0 & 0 & m_{4} \end{array}\right) $$
$$ v = \begin{pmatrix} \dot{x}_1 \\ \dot{y}_1 \\ \hline \dot{x}_2 \\ \dot{y}_2 \\ \hline \dot{x}_3 \\ \dot{y}_3 \\ \hline \dot{x}_4 \\ \dot{y}_4 \end{pmatrix} $$
$$ N^\top A^\top M^{-1} A N = \left(\begin{array}{cc} \frac{1}{m_{1}} + \frac{1}{m_{4}} & 0\\ 0 & \frac{1}{m_{2}} + \frac{1}{m_{4}} \end{array}\right) $$
$$ N^\top A^\top v = \begin{pmatrix} \dot{x}_4-\dot{x}_2 \\ \dot{y}_4 - \dot{y}_2 \end{pmatrix} $$
$$ J = \begin{pmatrix} \frac{\dot{x}_4-\dot{x}_2}{\frac{1}{m_2}+\frac{1}{m_4}}
\\ \frac{\dot{y}_4-\dot{y}_1}{\frac{1}{m_1}+\frac{1}{m_4}} \end{pmatrix} $$
Then
$$v^+ = v - M^{-1} A N J = \begin{pmatrix}
\dot{x}_1 \\
\frac{m_1 \dot{y}_1 + m_4 \dot{y}_4}{m_1+m_4} \\
\frac{m_2 \dot{x}_2 + m_4 \dot{x}_4}{m_2+m_4} \\
\dot{y}_2 \\
\dot{x}_3 \\
\dot{y}_3 \\
\frac{m_2 \dot{x}_2 + m_4 \dot{x}_4}{m_2+m_4} \\
\frac{m_1 \dot{y}_1 + m_4 \dot{y}_4}{m_1+m_4} \end{pmatrix} $$
Appendix
To include rotations follow the guidelines here:
