The shape often doesn't matter. Most orbits are very far away. In the solar system, the sun and planets are modeled as points. Even ones like Earth, which has a large moon. This gave extremely good results.
For example, the orbit of Mercury is close to an ellipse. It is perturbed by the attraction of other planets, primarily Jupiter. Because of this, the perihelion does not occur in the same place every year. It slowly rotates around the Sun, at about 5000 seconds of arc per century. In the 1800's, Astronomers modeled all the effects they could think of and accounted for almost all of the drift. The remainder was a long standing mystery until Einstein predicted that General Relativity would account for the remaining 43 seconds of arc per century.
This is cheating a bit because the planets are close to spherical. We can model the Earth we are standing on as a point at its center. We can calculate our weight to pretty good accuracy.
Sometimes it does matter. There has been a lot of work on orbits of the Moon and other satellites around the Earth. They are close enough and orbits are measured accurately enough that the shape of the Earth does matter.
For a long cylinder, matter would not tend to orbit perpendicular to the axis. The force of gravity would be radially inward. It would have a $1/r$ dependency instead of $1/r^2$. A satellite at just the right velocity would have a circular orbit perpenducular to the axis.
The orbit would not be stable. Any deviation from the perfect speed, and it would fly off into space or crash. See https://en.wikipedia.org/wiki/Bertrand's_theorem and An intuitive proof of Bertrand's theorem
If the orbit was tilted with respect to the axis, it would have a screw shape. There would be no restoring force along the axis, so it would keep any along-axis velocity.
For more information about orbital mechanics, Goldstein is a good advanced book.