In answer to a comment above
So are you saying that the photon is effectively travelling in all directions? If I have 2 slits 100 miles apart the photon will still go through both? If I put the slits behind the laser then the photons will still go through both?
Photons are elementary particles and in the standard model of particle physics they are point particles. They are not all over the place, classical light emerges from a superposition of the wavefunctions of a large number of photons.
Here is a double slit single photon at a time experiment

Single-photon camera recording of photons from a double slit illuminated by very weak laser light. Left to right: single frame, superposition of 200, 1’000, and 500’000 frames.
On the left each dot is the footprint of an individual photon, and it is not spread out all over the place. At the right is the probability distribution which coincides with the classical interference pattern, for the quantum mechanical problem "single photon impinging on double slits".
The photon is not "traveling in all directions", it is being scattered by the fields of the sides of the slit and there is a probability for an individual photon to scatter at a large angle. The probability depends on the photon energy, $h.ν$, and so the distances are related to the wavelength that will be built up by many photons, if one wants to see a clear interference pattern.
See also this answer of mine.