Given that Maxwell's equations can be thought of as the propagation equations for a one photon state as I discuss here, the classical and quantum answers are the same for the case where you have low light levels so that the probability of photon-photon interaction is negligibly small. I don't feel qualified to answer about the effect of such interaction.
This leaves the two mechanisms for loss in the one-photon / classical situation:
- Tunnelling, or frustrated TIR, when the low refractive index medium beyond the reflecting interface is of a finite thickness, and there is a medium beyond that one that would not totally internally reflect if it were in direct contact with the first medium;
- A beam's finite width means that it is not a plane wave, but a superposition of such. A Fourier transform of the transverse field will reveal that some plane waves in this superposition that do not undergo total internal reflexion.
For the first effect, the totally internally reflecting layer needs to be thin. I couldn't find the solution to frustrated total internal reflexion anywhere on the internet, so I quickly derived the formula for the transmitted power in terms of the incidence angle $\theta_1$, the refractive indices $n_1,\,n_2,\,n_3$ of the incidence layer, the reflecting layer and the layer beyond that, respectively and the center layer thickness $a$ using the scalar methods of my answer here. The photon transmission probability I reckon to be:
$$\frac{4 n_1^2 \cos^2\theta_1 \left(n_1^2 \sin^2\theta_1-n_2^2\right)}{\sinh^2\left(\frac{4\,\pi\,a}{\lambda}\, \sqrt{n_1^2 \sin^2\theta_1-n_2^2}\right) \left(-n_1^2 \sin^2\theta_1+n_1 \cos
\theta_1 \sqrt{n_3^2-n_1^2 \sin^2\theta_1}+n_2^2\right)^2+\left(n_1^2 \sin^2\theta_1-n_2^2\right) \left(\sqrt{n_3^2-n_1^2 \sin^2\theta_1}+n_1 \cos\theta_1\right)^2 \cosh^2\left(\frac{4\,\pi\,a}{\lambda}\, \sqrt{n_1^2
\sin^2\theta_1-n_2^2}\right)}$$
Since I've just derived this myself in five minutes in Mathematica, there are no guarantees, but what I am sure of is the $\sinh$ and $\cosh$ terms in the denominator, i.e. the transmission probability dwindles exponentially with reflecting layer thickness, i.e. like $\exp\left(-\frac{4\,\pi\,a}{\lambda}\,\sqrt{n_1^2\,\sin\theta_1^2-n_2^2}\right)$, so it's a pretty swift diminution.
To calculate the effec of 2., if one supposes the beam to be hard limited at its edges, one gets a $\operatorname{sinc}$-type Fourier transform for the plane wave superposition. So one can work out what the superposition weights of the waves skewed at large enough angles relative to the nominal angle of incidence that they do not undergo TIR.
