In addition to the argument using Huygens' principle, there is an argument based on the uncertainty principle. Short version: a small slit produces a large distribution of transverse wave vector components and hence a large distribution of propagation direction.
We have a beam of light incident on a slit. A good model for this is a plane wave incident on a slit at normal incident angle. A plane wave is only a model because it has nothing constraining it perpendicular to the direction of travel. It's infinite in extent, which is why it's only a model of the radiation field. The wave vector is directed toward the slit. There is no component perpendicular (transverse) to the normal direction.
The slit changes things. The field is now constrained in the transverse direction. It's not a plane wave anymore. The uncertainty principle tells us that the extent of a property is inversely proportional to the extent of the conjugate property. A common example is a signal varying in time, such as a radio wave pulse. A short pulse means a larger extent of frequencies, and a long pulse means a narrow range of frequencies.
In our case, the slit constrains the transverse spatial extent of the field. This is the analogue of the pulse width of the radio wave. The conjugate is the transverse propagation vector, the analogue of frequency. (The units of the wavevector are inverse meters as the units of frequency are inverse seconds.) In analogy to the time/frequency situation, a small slit (transverse extent) means a large distribution of transverse wave vectors, and a large slit means a small distribution of transverse wave vector. A large distribution of transverse wave vector means a large distribution of wave vector angle, and Bob's your uncle.
The next question will be: what is the physical mechanism at the slit for the change in transverse wave vector? The short version: scattering at the edges of the slit. Not an easy problem to solve. But midway between scattering theory and the relative simplicity of the wave vector description is Huygens' principle, which takes us back to where we started.