This is almost an exact duplicate of What is the mechanism behind the slowdown of light/photons in a transparent medium?.
However, this question is about the particle model and its relation to refraction.
The short answer is that sometimes it seems better to model light as a particle, sometimes as a wave; in this case it makes sense for most to model light as a wave. However, we must remember that the wave and particle models of light are just that; models. We find them as very handy models to describe our observations of light, and to make predictions based on those models, however ultimately the nature of light is quite an elusive concept. We seem to know a lot about, and at the same time don't know a lot about, light.
Nevertheless, a photon itself can be thought of as having its own wave signature describing its motion. In that case you come full circle to thinking about the interaction of a wave with a material again.