At first glance, one might think that the two body problem with inverse square force is a symmetric problem. The force (in the reduced mass frame of reference) depends only on the distance $r$, without an angle dependence. However, most orbits are elliptical, hence something had to break the perfect symmetry. It cannot be angular momentum, since circular orbits, which are perfectly symmetric, are allowed. And in any case, angular momentum does not break 2D rotational symmetry.
So what breaks the 2D rotational symmetry in the two body problem with inverse square force? Is it the initial velocity?