Where to measure?

Question

In all my experiments (at the university) so far, I nearly always took my measurements on an equispaced grid over the whole measurement space. And now I asked myself, why? Is there a theory of where to place your measurements for max efficiency? And by max efficiency I mean, lower the standard deviation of a result as low as possible, for a limitted amount of measurements.

To give you a better idea of what I mean: Suppose I have a model function $y = m(\beta,x)$ of $x$ with parameter $\beta$. I can only take $n$ measurements of $(y_i,x_i)$. I'd like to determin $\beta$ as accurate as possible, so get the standard deviation $\Delta\beta$ as low as possible. What values of $x_i$ should I choose to do so?

Answer 1

This is a really good and important question; I am not a mathematician, and my answer will just point to a few things to consider. The general answer depends on the form of $m(\beta,x)$. Specifically, if you want to determine $\beta$ with the greatest accuracy, you need to choose points which contain "the most information" about $\beta$. You don't always know ahead of time where these points will be; if you don't, equal spacing may be appropriate -- but if you have a chance to improve your sampling during the experiment, you may do even better.

For example, if the function is a Heaviside step function, $\beta$ is the position of the step, and a priori you don't know where the step is, then equal spacing is a good approach. However, if you can take data one point at a time, it would be better to bisect the interval; determine if you are to the left or right of the step, and choose your next bisecting point accordingly. That approach will allow you to determine $\beta$ with much greater accuracy than linear spacing. For $N$ samples of an interval $L$, your error would be on the order of $\frac{L}{2^N}$ instead of $\frac{L}{N}$.

Generalizing this approach has spawned a whole branch of mathematics called "optimal sampling" which is usually addressed with maximum likelihood (ML) methods. An introduction to some of the issues can be found in Pagendam et al . Key concept is the Fisher information:

The Fisher information is a way of measuring the amount of information that an observable random variable X carries about an unknown parameter θ upon which the probability of X depends.

As the above Wiki link shows, the Fisher information gives information about the curvature of the support curve near the maximum likelihood estimate of $\beta$. This is another way of saying that you will get the most accurate estimate by using a sampling that maximizes the Fisher information. But since calculating the Fisher information may require you to know the form of the function, and therefore the value of $\beta$, it is not always possible to determine optimal sampling a priori.

As I said - I am not a mathematician and the above is a little bit fuzzy. But your intuition that regular sampling is often not the best strategy is spot-on. I hope this answer will spur some additional answers / improvements.

Answer 2

To get the best possible estimate, obviously you would sample with an infinitely fine grid.

In reality this is not possible or practical in most situations, as you want to finish you measurements in a sensible time frame and other considerations like fluctuations over time and damage to the sample/equipment will become an issue.

Therefore you need to sample your measurement space using some sampling method, of which uniform sampling is one of many.

If we consider the problem in the frequency domain Nyquist-Shannon sampling theory says that in order to accurately reconstruct the underlying function $m$ you must sample at least twice the maximum frequency in $m$.

This is not a problem for linear or periodic functions by if you sample below this frequency it can result in spurious effects such as aliasing. A good example of this is if you try and measure the amplitude of a periodic signal at the same frequency as the underlying signal you will conclude its amplitude is zero!

However, functions with discontinuities have infinite range of spatial frequencies. Therefore, there will always be some error due to the sampling. In many cases these errors will be minor and are generally decreased by increasing sampling frequency. As a side note similar issues can occur for any non-periodic signals or periodic signals when the number of periods in the measurement window is not a whole number - generally this is not a big concern, though.

On specific sampling methods, uniform sampling is often favored as it is very easy to do and requires little calculation. However, as mentioned above it has a finite frequency response which can cause issues when measuring high frequency data.

The other commonly used sampling method is simple random sampling where positions are selected randomly throughout the measurement space. This approach has a theoretically infinite bandwidth and so aliasing is less of a problem and the data is easy to treat mathematically. However, depending on your set-up it can be more difficult to perform as you must generate random positions. Additionally, there are often large areas of the measurement space with sparse coverage, which can result in some poor estimates, especially if discontinuities lie in those areas.

There are many other advanced sampling methods which are designed to help deal with these issues such as adaptive sampling. These approaches are designed to give better results for difficult to measure surfaces, but are often mathematically complex and difficult to implement in practice.