How do we physically choose a co-ordinate system for making astronomical observations?
In a special relativistic system, the definition of relative velocity, clock synchronization is well understood and can be measured between observers located at arbitrarily separated points (this has been discussed in an earlier post). However, for a curved space-time, this construction is only true locally. Say, if two observers are far away from one another and they want to synchronize their clocks, they would need to measure the distance between them and also their relative velocities. The only unambiguous way to measure relative velocity between two observers is when their velocity vectors are compared on the same tangent plane. If these observers are at different locations, then there is no unique way one can define their relative velocity (discussed in this post). Further, to determine the distance between these observers, one would ideally need to know the metric (at least, in the neighborhood containing the two observers). This can be determined to some extent from indirect observations. For example, in a cosmological setting, one could cook up a metric in the vicinity of observer's past light cone from independent astronomical observables, like observed redshift, number count density, luminosity etc (see Ellis et al., 1985). However, these observables do not completely determine all of the metric components (but they completely determine the FRW geometry).
Given these uncertainties in the definition of relative velocities and metric of our observable surrounding, are there ways by which one could physically construct a co-ordinate system? If yes, are these co-ordinate system reliable ? And to what extent (length scale)?