In a classical electromagnetic wave of a particular frequency (for example, red light), the magnitude of the electric field and the magnetic field oscillate in time at each point in a sinusoidal way, and the magnitude at each time changes in space (in the direction of the wave) in a sinusoidal way. So it is these two fields that are doing the “waving”, both as a function of time and as a function of position.
But these fields are not material things moving back and forth in space. It is just that, at each point, with time they get strong in one transverse direction, then weaker, then strong in the opposite direction, then weaker, etc. And, looking along the direction of the wave at a particular instant, they are strong in one transverse direction, then weaker, then strong in the opposite direction, then weaker. The direction of the two fields is perpendicular to the direction of the wave, and also perpendicular to each other.
When this classical theory is quantized, photons are the quantum excitations of this electromagnetic field, but before understanding what that means you should understand the classical picture of electromagnetic waves as oscillating transverse vector fields. As you realized, the photons do not move on a sinusoidal trajectory. Actually, they don’t have a meaningful trajectory at all because, as quantum particles, their position and momentum cannot be simultaneously measured with complete accuracy. However, they behave as if they move in straight lines at the speed of light.