Newton's second law tells us that the acceleration of an object is related to the force acting on it by the equation:
$$ \frac{d^2x}{dt^2} = \frac{F}{m} \tag{1} $$
Note that if the force, $F$, is zero then equation (1) reduces to:
$$ \frac{d^2x}{dt^2} = 0 $$
i.e. the acceleration is zero so the velocity of the object doesn't change. If the force is zero a stationary object will remain stationary.
The corresponding equation in general relativity is the geodesic equation:
$$ {d^2 x^\mu \over d\tau^2} = \Gamma^\mu_{\alpha\beta} {dx^\alpha \over d\tau} {dx^\beta \over d\tau} \tag{2} $$
This is a lot more complicated that equation (1), but the left side is basically just an acceleration and the right hand side can be thought of as the gravitational force. The quantity $\Gamma^\mu_{\alpha\beta}$ describes the spacetime curvature in a complicated way that only us nerds need to worry about. What's interesting about this equation is that the right hand side contains the terms $dx^\alpha/d\tau$ and this is sort of a velocity. So if all the $dx^\alpha/d\tau$ terms, i.e. all the velocities, were zero equation (2) would simplify to:
$$ {d^2 x^\mu \over d\tau^2} = 0 $$
and just as for our original Newton equation this would tell us that the velocity of the object is constant i.e. there is no gravitational force - a stationary object wouldn't feel a force, just as you say.
The trouble is that in GR we consider motion in spacetime not just space, and while you can stop moving in space you can't stop moving in time. All of us move through time (normally at one second per second) no matter how much we might wish it otherwise. That means there is no such think as a stationary object, and no way to escape gravity.
You are right to be cautious about the rubber sheet analogies for spacetime curvature. While these have various shortcomings a major one is that they don't portray the curvature in the time dimension.
Response to comment:
Using the geodesic equation to show how movement in time causes an object to move in a gravitational field is straightforward, but I doubt the maths would be very illuminating. So let me attempt an intuitive explanation of why it happens. Nota bene that like any intuitive explanation this will be simplified and potentially misleading if you pursue it too far. Still, let's give it a go.
Let's put you at some distance from a black hole and let go. And just to make it more fun we'll put an opaque shell around you so you can't see out. The key point is that you will feel no force (you'll be in free fall just like the astronauts in the International Space Station) so you're going to assume space is flat. You would draw a spacetime diagram to describe your motion that looks like this:

We're approximating you as a green dot, and in your coordinates you stay still in space but move in time, so you just move up the time axis.
Now suppose I'm watching you from well away from the black hole. Because spacetime is curved, my $r$ and $t$ axes won't match yours. I've drawn my $r$ and $t$ axes as curves, but don't take the shape of the curve too literally as I have just sketched any old curve:

Now let's superimpose my axes and your axes:

Because of the curvature your time axis and my time axis won't match. That means if you're moving along your time axis you're not moving along my time axis. From my perspective you've moved off the time axis by a distance shown by the red arrow, and that means you must have moved in space.
So even though, as far as you're concerned you're just moving in time, as far as I'm concerned you've moved in space as well. And that's why a stationary object falls.