OK, I'll make a complete revision of my original reply since it was quite sloppy.
First, I originally confused two issues that are nevertheless related, I confused stability of matter and the impenetrability of matter.
But, it must be clear that the two questions are related. If I have two chunks of matter of the same type on top of one another, one can't imagine that the explanation for the fact that these chunks don't "fall through" each other would be unrelated to the explanation of why we do not fall through the ground. So, in ultimate analysis, the question is tied to the question of stability of matter.
Now, there are several steps in the problem. To explain the stability of matter, one has to explain why atoms are stable (and before that why nuclei are stable), then one has to explain why aggregates of atoms like solids or liquids can be stable, i.e. why bulk matter is stable. The stability of bulk matter will then serve as the basis to explain why "we can stand on the ground".
Starting with the last step and assuming we already know about the stability of bulk matter, we can imagine that when we exert a pressure on stable bulk matter, we can expect by what it means to be in a stable equilibrium, that the piece of matter would exert an opposing pressure, trying to restore itself to its most stable configuration, provided stresses are not too great. So solving the problem of stability of bulk matter will help us understand what the nature of the restoring force will be.
Now, as is well known, electromagnetic forces can not be the sole explanation. There exist no stable equilibria when there are only electric charges interacting electromagnetically. I won't go through the proof here, but it is accessible to undergraduates, it can be found in the Feynman Lectures, book 2, chapter 5. It's an application of Gauss' law in the static case. The dynamic case only complicates matters in the wrong direction. As we know, accelerated charges radiate energy away, thus an electron orbiting a nucleus would soon fall inward if nothing prevented it, to take the classic example.
Enters Elliott Lieb and his paper 'The stability of matter' that can be easily found online. So, I'll quote a lot from there. It reviews a lot of the results in the field of mathematical physics of the problem of stability of matter.
So what does Lieb essentially say about stability of atoms: that it is a consequence of a principle introduced by Sobolev. Sobolev's inequality states in a mathematically precise form that if one tries to compress the wavefunction anywhere, the kinetic energy will increase. It's a kind of stronger version of the HUP. (Note that at this point, Lieb does not use Pauli's exclusion principle. This is to be expected, take a hydrogen atom, it is stable, since there's only one electron, Pauli's exclusion principle can't be invoked here to explain its stability.)
Then, Lieb goes on to explain the stability of bulk matter by using Sobolev's inequality again. But this time around, he extends the inequality, and takes into account the fact that matter is made up of fermions. So, the Pauli exclusion principle is indeed used. So, again a lower bound for the kinetic energy is found, the interesting thing is that this lower bound is proportional to $N^{5/3}$ where $N$ is the number of fermions. If the particles were not fermions, the proportionality would have been $N$, which we can see by using the previous bound for 1 atom and multiplying by the number of atoms. So it is really the Pauli exclusion principle that contributes the factor $N^{2/3}$.
Lieb goes then to show that this factor is crucial. He uses Thomas-Fermi theory as a relevant approximation of the behaviour of bulk matter to demonstrate this. This is were the analysis becomes very intricate. I have no time to summarize it in more detail. So I'll just say that some theorems about the nature of TF theory are derived, these are then combined in the end to show that the minimum energy or ground state energy of the system is bounded from below. A numerical value for this bound is derived which is −23 Ry/particle, (1 Ry ≈ 13.6 eV).
The important take-away message is though that Fermi statistics or the Pauli exclusion principle is indeed essential to explain the stability of bulk matter.
In Lieb's paper, there is an extra chapter which tackles the question of why matter doesn't explode, instead of implode. The interesting thing is that pure EM is sufficient to answer this question.