Everyone who has been interested in modern science has heard explanations (certainly simplifications) of general relativity, mostly that space is curved.
I'm afraid that those explanations that say space is curved are misleading. See Baez: "Similarly, in general relativity gravity is not really a 'force', but just a manifestation of the curvature of spacetime. Note: not the curvature of space, but of spacetime. The distinction is crucial". Curved spacetime is a "curved metric", and a metric is to do with measurement. For example, you place optical clocks throughout an equatorial slice through the Earth and the surrounding space, then plot the clock rates. You depict lower slower clocks as lower down in a 3D image, and higher faster clock rates higher up. What your plot looks like, is this:
CCASA image by Johnstone, see Wikipedia
That's a picture from the Wikipedia Riemann curvature tensor article. It's a depiction of curved spacetime. And because it's derived from optical clock rates, it's a plot of the "coordinate" speed of light. Your plot of measurements is curved, space isn't. Instead space is inhomogeneous, and because of this light curves and matter falls down. Note that you need the curvature to get the plot off the flat and level - you need the curvature to get the slant, but the curvature relates to tidal force while the slant relates to the force of gravity.
The analogy with a rubber sheet is popular. In such an analogy, orbiting planets are said to be naturally following "a straight line in a curved space".
The rubber-sheet analogy isn't quite right, but it isn't totally wrong either. The planets move a bit like marbles on a rubber sheet. The slant makes them veer towards the Sun, so they end up going round and round.
Assuming that is not an oversimplification, would it mean that orbits are loops generated in space by massive objects?
No. Orbits are the curved paths of objects moving through inhomogeneous space and so subject to the force of gravity. The central massive object "conditions" the surrounding space, altering its properties, the effect diminishing with distance.
Also, if we consider spacetime as a curved structure, thus lines are not necessarily straight, what would be the meaning of momentum in such a frame?
I'm not sure what to say. Spacetime can be thought of as a "block universe" that models motion through space over time as geodesic worldlines. Worldlines don't move, and objects don't move up them. To get the gist of this, use an old-style movie camera to film a red ball flying across a room. Then develop the film and cut it up into individual frames, then form them into a block. There's a red streak through the block. That's like a worldline. But there's nothing moving in or through the block, nothing at all. Momentum would be where the streak wasn't vertical because the ball was moving. A curved streak would represent acceleration or changing momentum.