The fundamental equation that contains $I$ is:
$\vec{L} = I \vec{\omega}$
where $\vec{L}$ is the angular momentum vector, and $\vec{\omega}$ is the angular velocity vector. The fact that $I$ is a tensor (and not just a scalar) means that $\vec{L}$ and $\vec{\omega}$ do not necessarily point in the same direction.
When you want a particular matrix representation of the moment of inertia tensor, you may get off-diagonal elements. However, since by its very definition, any matrix representation of $I$ is a symmetric matrix. Any real, symmetric matrix can always be diagonalized by orthogonal matrices. What this physically means is that, for at least one choice of $x$, $y$, $z$ axes, the representation of $I$ will be perfectly diagonal, with all the off-diagonal elements zero.
However...
This does not at all mean that the off-diagonal elements are not important. And I will not escape the question of their meaning.
There is also the equation describing the dynamics of the whole business:
$\frac{d \vec{L}}{ dt} = \vec{\tau}$
This is analogous to $\frac{d\vec{p}}{dt} = \vec{F}$ which describes linear motion; here of course $\tau$ is the torque, while $\vec{L}$ is the angular momentum.
Substituting $\vec{L} = I \vec{\omega}$ and assuming a time-invariant moment of inertia tensor, this becomes:
$I \frac{d\vec{\omega}}{dt} = \vec{\tau}$
Now, the torque you are applying does not really have to coincide with a vector that diagonalizes your $I$ (such vectors are called principal axes). For instance, take the wheel of your car. It is driven by a rod, going through its axis. Let us take that to be the $z$-axis. So, in this case, $\tau_z$ is non-zero, while $\tau_x$ and $\tau_y$ are zero.
The above equation is actually three equations, let me show the third one explicitly:
$I_{zx} \frac{d\omega_x}{dt} + I_{zy} \frac{d\omega_y}{dt} + I_{zz} \frac{d\omega_z}{dt} = \tau_z$
Now, if the off-diagonal elements $I_{zx}$ and $I_{zy}$ are zero (assuming zero initial conditions) your wheel will just acquire $\omega_z$, that is, it will start rotating nicely about the axis you are applying the torque on.
However, if they are not zero... $\omega_x$ and $\omega_y$ will not remain zero. That is, the wheel will tend to start to rotate around the other axes as well! So, you will not get a rotation around the $z$-axis, and the wheel will tend to wobble! This is the condition where you take your car to the shop, and have the wheels balanced!
So, the off-diagonal elements really mean that any attempt to rotate the body by applying a torque about a given axis will not result in a rotation about just that axis and there will be rotation around the other axes as well.
Note that if, for instance, we have $I_{xy} \neq 0$ and $I_{xz} = I_{yz} = 0$, a torque around the $z$ axis will result in a balanced rotation, while a torque around the $x$ or $y$ axes will not...
Looks like it's directly analogous to principal components, about which there is some significant literature: http://en.wikipedia.org/wiki/Principal_components
– Paul Apr 12 '13 at 04:31