In the most cases the local inertial frame is definied "physically" but I'm searching for a mathematically meaningful definition of the local inertial frame to solve my problem:
Is the local inertial frame the same as a coordinate chart of the given manifold? But then the local inertial frame must not be very small (as it should be in general relativity) since a chart can be the whole manifold.
in GR, a local inertial frame constructed in a finite neighborhood of a timelike geodesic (i.e. around a free falling body for a certain lapse of time) is a four dimensional local coordinate chart such that, exactly on the geodesic, all the Christoffel symbols $\Gamma^a_{bc}$ vanish. This is the strongest condition one may require in a general curved spacetime. Using these coordinates, with the time coordinate measured along the geodesic and the three spatial coordinates normal to the geodesic, up to third order corrections, the motion of another free falling body crossing the initial one is pictured as a linear curve. In other words the gravitational acceleration is approximately canceled in a spatial neighborhood of the observer at rest with the reference body.
ADDENDUM A practical construction of the said coordinates is the following. Fix a pseudo-orthonormal basis $e_0,e_1,e_2,e_3$ of the tangent space $T_pM$ of the initial point $p= \gamma(0)$ of the timelike geodesic $\gamma : [0,T] \to M$. Assume that $e_0= \dot{\gamma}(0)$ and define $e_0(t),e_1(t),e_2(t),e_3(t)$ as the analogous basis parallely transported along $\gamma$ to each point $\gamma(t)$. Finally, if $t\in [0,T]$ and $x \in \mathbb R^3$ define $$q(t,x)= \exp_{\gamma(t)}\left(\sum_{k=1}^3 x^k e_k(t)\right)\:.$$ It is possible to prove (e.g. see O'Neill's textbook) that, there is $\delta>0$ and an open ball centered on the origin $B\subset \mathbb R^3$ such that the map $$(0,\delta) \times B \ni (t,x) \mapsto q(t,x)$$ is a diffeomorphism onto an open set including the portion of $\gamma$ for $t \in (0, \delta)$. In other words the numbers $(t,x^1,x^2,x^3)$ are local coordinates in a neighborhood of the said portion of $\gamma$. By direct inspection one sees that, computing the Christoffel symbols in these coordinates, one finds $$\Gamma^a_{bc}(t,0,0,0)=0 \quad \mbox{for $t \in (0,\delta)$}\:. \tag{1}$$ The chart covered by coordinates $t,x^2,x^2,x^3$ is a Riemannian normal coordinate chart adapted to $\gamma$ by definition (it is the mathematical corresponding of the more mathematically vague notion of local inertial frame used by physicists). You see that it is defined in a finite domain (not infinitesimal) though its relevant property (1) holds exactly along the (portion) of the considered curve.
With some computations, (1) entails that the equation of another geodesics $\gamma_1$ crossing $\gamma$ in, say $\gamma(t_0)$ has the form, for three constants $v^1,v^2,v^3$ $$x^\alpha(t) = v^\alpha \cdot (t-t_0) + O((t-t_0)^3)\quad \alpha =1,2,3\:.\tag{2}$$ In other words, confining ourselves to the domain of the coordinates $t,x^2,x^2,x^3$, the motion of another free falling body is a constant-velocity motion as soon as the body is close to the centre of the coordinates where the initial reference free falling body $\gamma(t)$ stays (it happens for $t\to t_0$). The equivalence principle is mathematically apparent in the absence of the second order term, proportional to $(t-t_0)^2$, in (2).
Summing up, a locally inertial reference frame is mathematically pictured in terms of a quite particular coordinate chart adapted to a portion of a given timelike geodesic representing a free falling body. This type of local coordinates are called Riemannian normal coordinates. Generally speaking, this coordinate patch does not cover the whole manifold (however it is true in Minkowski spacetime) but only a small region. Its relevance however just concerns what happens exactly on the reference geodesics.
