Wikipedia defines Wigner $D$-matrix as an irreducible representation of groups $SU(2)$ and $SO(3)$. What is a good way to visualize this representation? Is there any physical system which can be kept in mind as a simple example of the same?
A general explanation of the idea of irreps, beyond just the Wigner-$D$ matrix, would be appreciated.
Is there any physical system which can be kept in mind as a simple example of the same?
Yes. Consider a single spin $1/2$ particle, like an electron. In this case, the matrix will be $2$-by-$2$ since its a representation of $\mathrm{SU}(2)$ acting on the two-dimensional spin-$1/2$ Hilbert space. The idea here is that when you rotate the physical system by a rotation $R$ say, then the spin state "rotates" as well (the states in the Hilbert space "rotate" into each other if you will) as follows: \begin{align} |\tfrac{1}{2},m'\rangle\longrightarrow D^{1/2}_{m,m'}|\tfrac{1}{2},m'\rangle \end{align} In fact, for a rotation by an angle $\theta$ about the unit vector $\mathbf n = (n_x, n_y, n_z)$, we have \begin{align} (D^{1/2}_{m,m'})=\begin{pmatrix} \cos\frac{\theta}{2}-in_z\sin\frac{\theta}{2} & (-n_y-in_x)\sin\frac{\theta}{2} \\ (n_y-in_x)\sin\frac{\theta}{2} & \cos\frac{\theta}{2}+in_z\sin\frac{\theta}{2} \\ \end{pmatrix} \end{align} so that when $\theta = 0$, this is the identity matrix; nothing happens to the state, while when $\theta = 2\pi$ (a full rotation), this matrix is $-1$ times the identity, and the state therefore rotates into itself multiplied by $-1$; pretty strange isn't it?
A general explanation of the idea of irreps, beyond just the Wigner-D matrix, would be appreciated.
This is an extremely broad question. The general idea behind a representation of a group $G$ is that it is a mapping that assigns an invertible matrix $D(g)$ to each group element $g$ such that the group structure is preserved (the technical term for this is that it is a group homomorphism). The representation is said to be irreducible if it has no nontrivial invariant subspaces. Concretely, this means that there is no similarity transformation that puts all of the representation matrices into block diagonal form.
