This is a really interesting question, and I worry that you are getting overly bogged down by being unable to focus individually on the different perspectives which are happening at multiple scales. First, let's tackle the more microscopic quantum scale. If we want to understand how light is (or is not) being absorbed by a material, we must first understand what would cause that absorption in the first place. You hit the nail on the head by giving an example of a process that would allow light to be absorbed; absorption of UV-visible light often leads to the excitation of electrons about the various energy levels within the material. A related phenomena might be the absorption of IR light by molecules because of the excitation of the vibrational degrees of freedom into excited states. All together, the various ways that a material may absorb light are collectively determined by the quantum mechanical structure of the material and what levels and states are available. Of course, describing these levels gets increasingly complicated the more complex the material becomes (which is why I have a job!). But there is one key thing you are missing.
While there may be levels present to allow an absorption of light, we still have to ask how likely it is that the light is absorbed! The classic example of such a calculation is Fermi's Golden Rule in perturbation theory for a two level system which relates the probability of absorption to the transition dipole moment between the two states. See MacQuarrie's Physical Chemistry A Molecular Approach, or his Statistical Mechanics for derivations and details. So we not only have to worry about whether there are levels for the light to cause transitions between, but also the probability of this happening at all. This probability analysis becomes more difficult when we then have to consider how frequently the light will actually get an opportunity to be absorbed on its journey.
There are further complications to all of this as well. Will the light undergo scattering processes that may allow the light to move through a material, while changing directions and potentially even turning back on itself without being absorbed? Once light has been absorbed, does it simply dissipate the excess energy through the material as heat, fluoresce it back as visible light (and in what direction?), or even do more "forbidden" transitions such as phosphorescence? The answer to all of these questions is "yes, but in varying degrees specific to the material." It even depends on the physical size and amount of material that you have present! So your worries about the rate at which light is able to pass through different materials really just becomes a question of how likely it is for the light to become "sidetracked" as it passes through the material. The net result is an apparent change in how fast the light passes through.
So finally, we can connect all of these nuanced ideas which could each fill a book of discussion on their own (and have) to the macroscopic perspective. All of these properties, for a material like a bulky glass window pane, can be summarized with three simple parameters such as the coefficients of absorption, transmittance, and reflectance of the material. Obviously, for light moving through normal glass, there is an overwhelming victory of the transmittance effects over the absorption and reflectance effects. Of course, you have also clearly experienced that standing at an alternate angle changes the propensity of reflectance of different kinds of light, mostly because you have changed the relative orientation of the light with respect to the overarching structure of the glass itself. This is explained much more deeply within the field of chemical crystallography.
But at the end of the day we can see how the different scales of processes all combine to give the properties we experience every day. Different materials will have wildly different properties owning to their molecular and even bulk structure. A truly lovely example is the way a "two-way mirror" works. It is really just nothing more than an ordinary mirror constructed with glasses that mostly reflect light and transmit a little. The trick is just that the two sides are vastly unevenly lit, so one side gets an extremely biased amount of reflection, and the other gets a hugely biased amount of transmittance of light, making one side see through to the other, and the other see their reflection in a mirror. But to finally bring it home, it just so happens that the glass which makes up your window is remarkably transmissive of visible light. Thanks for a very interesting question.