In Quantum Mechanics, we call any triplet of operators $J_i$ (of any dimension) that satisfy the commutation conditions $[J_i, J_j]=\epsilon_{ijk}J_k$ an angular momentum operator.
Because for the following two reasons:
- $\hat{J} = \hat{L} \otimes 1 + 1 \otimes \hat{S}$.
$L_i$ clearly denotes the (orbital) angular momentum as $L_i = \epsilon_{ijk}r_jp_k = -i\hbar\epsilon_{ijk}x_j\partial_k$ (in position basis) and if you calculate, it will then satisfy the commutation relation $[L_i,L_j] = \epsilon_{ijk}L_k$ ... (I)
Now come to the point of (spin) angular momentum $S_i$. Long after the Stern-Gerlach Experiment, theory of the spin angular momentum was being developed (Outcome of the Stern-Gerlach Experiment says that the deflection of the silver beams must have an intrinsic angular momentum, thus $S_i$ corresponds to an angular momentum in principle; this was historically the first argument that the spin corresponds to an angular momentum). But as there is no classical analogue of 'quantum spin', so no such formulation (like $\hat{L} = \hat{r} \times \hat{p}$) was available from the classical mechanics knowledge. Now as you know, most of the formulation of QM was being done comparing that's analogous classical formulation, so in this case of spin angular momentum, what is the way out. The simplest way is to derive an 'algebra' similar to the orbital angular momentum (as at the end of the day, spin results an angular momentum also). So after this, the building block of the 'spin' angular momentum algebra became, $[S_i,S_j] = \epsilon_{ijk}S_k$ ... (II)
[We accept this algebra of $S_i$ because it is well fitted with experimental observations and no counter-explanation of this theory has yet been discovered.]
Add (I) and (II),
$[J_i,J_j] = \epsilon_{ijk}J_k$.
- Here is the answer of:
Does this have to do with how they interact with the rotation operators?
Infinitesimal rotation is the generator of $L_i$ (and hence $J_i$).
Outline of the proof of this statement:
Consider rotation around z-axis associated with the rotation matrix becomes, $R_{z}(\theta)$.
$\therefore \vec{r'} = \hat R_{z}(\theta) \vec{r}$
where, $\hat R_{z}(\theta) =
\begin{pmatrix}
cos\theta & -sin\theta & 0\\
sin\theta & cos\theta & 0\\
0 & 0 & 1
\end{pmatrix}$; $\hat R_{z}(\theta_1 + \theta_2) = \hat R_{z}(\theta_1) + \hat R_{z}(\theta_2)$.
Now consider $\psi(\vec r) \longrightarrow \psi(\vec{r'}) = \hat R_{z}(\theta) \psi(\vec{r})$ and an infinitesimal rotation $\delta\theta$,
$\therefore \hat R_{z}(\delta\theta) =
\begin{pmatrix}
1 & -\delta\theta & 0\\
\delta\theta & 1 & 0\\
0 & 0 & 1
\end{pmatrix}$
So after few lines, $\delta x = -y\delta\theta$, $\delta y = x\delta\theta$, $\delta z = 0$.
Now, $\psi(x + \delta x, y + \delta y, z + \delta z) = \psi(x, y, z) + \dfrac{\partial\psi}{\partial x}\delta x + \dfrac{\partial\psi}{\partial y}\delta y = \hat R_{z}(\delta\theta)\psi(x,y,z)$.
Finally, $\hat R_{z}(\delta\theta) \equiv \hat{I} + \dfrac{i}{\hbar}\delta\theta.\hat L_z$ [QED]
Note: For more clear concepts and primary intuitions on this topic, you may go through the relevant chapters of the Quantum Mechanics book by David J. Griffiths.