If your system is damped, after some periods, the resonance will occur regardless of at which point you apply the harmonic force on the swing and only the resonance frequency. There is a steady state solution:
$ x(t)= X \sin{(2 \pi f t +\phi)} $. So if you apply a force with resonant frequency, it will vibrate at resonance as well and your initial conditions (e.g. starting phase) can be ignored.
If your system is undamped, there will be two terms determined by initial conditions (see solution below), but their frequency is identical to the resonant frequency. So the resonance still occurs.
a undamped mass-spring system with harmonic force input:
which can be solved by Wolfram:
In conclusion, if you apply a harmonic force on a linear mass-spring system, the resonance occurs regardless of damping and the initial conditions (e.g. different phase of a swing vibration).