First, a caveat: There are several interpretations of quantum mechanics -- that is, different ways of thinking about what's happening that nevertheless give the same predictions for what we can measure in experiments. How best to explain measurement is somewhat dependent on your choice of interpretation. For this answer I'll stick with something like the traditional Copenhagen interpretation.
Also, "behave differently when we look at them" is sort of a shorthand. It would be more accurate to say that the state of the atom changes when you measure it. We're not necessarily "looking at" the atom at all -- for instance, sometimes you have multiple atoms in an entangled state where performing a measurement on one may change the state of both atoms.
But, putting aside these details, the basic idea is this. In quantum mechanics, atoms are described by a wave equation (the Schrodinger equation), and waves can be in a superposition. Think of a sound wave, which may contain multiple frequencies (that is, multiple pitches). But whereas with a classical wave (like sound) we'd perform a frequency measurement and find a little bit of each frequency component, with quantum mechanics our measurements give a definite answer. So even if the wave is a superposition of frequency X and frequency Y, your frequency measurement gives you just X, or it gives you just Y.
Each of these possible outcomes has some probability of occurring, and this probability depends on how much of each frequency we started with. And after the measurement, if you only measured frequency X, then you are really left with a state that only contains frequency X, even if that's not what you started with. The measurement changes the state of the system being measured, and does it in a way we can't predict ahead of time; we can only say what the probability of getting that result would be.
I used "frequency" to maintain my analogy with classical waves, but this can be any observable (that is, any measurable property of the atom). The position of the atom, the momentum of the atom, etc.
Moreover, there is a different rule, the uncertainty principle, that says the particle can't be in a state with a definite momentum and a state with a definite position at the same time. So measuring the position will change the state of the particle to one with a single value of position (at least if I pretend we could measure it with perfect precision), and measuring the momentum will change it to a different state, one with a single value of momentum.
People often confuse the effect of measurement with the uncertainty principle, but note that they are two separate things. With measurement, we say "Measuring property P puts the particle in a state where it has one value of P, not a state that's a superposition of states with two different P values." The uncertainty principle says "There's no state which both has only one value of position and only one value of momentum". (In fact it says more than that, it says something specific about how the range of momentum values grows the more precisely we narrow down the position value and vice versa, but I'm trying to keep things simple here.)