In relativistic particle mechanics we use:
$$ S = \int d\tau = \int g_{\mu,\nu} \frac{dx^\mu}{d\lambda} \frac{dx^\nu}{d\lambda} d\lambda$$
Particles move along the minimal of $S$, which is the minimal proper time, which is the minimal path length along a curve, which is just a "straight" line along your manifold (geodesics are the generalisations of straight lines in flat Euclidean spaces).
One can rewrite this as
$$ S = \int \langle ~ \dot{x}(\lambda) ~, ~ \dot{x}(\lambda) ~ \rangle ~ d\lambda $$
e.g. you integrate over all scalar products of velocities along the path.
If we assume a constant metric $g_{\mu,\nu}$, integrate the first equation by parts and use the Euler-Lagrange equations $\frac{d^2x^\mu}{d\lambda^2} = 0$ we obtain for the minimal action:
$$S_{\textrm{min}} = \langle x_f,\dot{x}_f \rangle - \langle x_i,\dot{x}_i \rangle $$
with $x_f, \dot{x}_f$ the final position and velocity and with $x_i, \dot{x}_i$ the initial position and velocity. The important thing to note here is that the minimal action is a difference between terms that are evaluated at different times, e.g. belong to distinct places on the manifold. This is natural, since the action is an integral over some function along a path on the manifold (e.g. it is the antiderivative evaluated at the beginning and the end of the path).
The minimal action you propose has not that property, e.g. it is not a simple difference of two terms evaluated at different points on the manifold. This is not entirely impossible, however to achieve this your integrand has to be nonlocal, e.g. the particle at position $x$ has two know somehow what will be at $y$ or some finite intermediate point along the path. Then the antiderivative can be some function that "senses" properties at both $x$ and $y$ and in turn lead to some expression that is not necessarily a simple (local) difference.
Nonlocal theories are consider to be unphysical however, since they incorporate some instantaneous action at a distance, which we know should not exist when we limit ourself to causal theories.