The change of the overall phase of the wave function, $\left|\Phi \right> \to e^{i\phi}\left|\Phi \right>$ is unphysical since they give the same expectation value for any observable operator. So, they represent the same physical state: indeed the space of physical states is not the Hilbert space, but rather it is the space of equivalence classes of Hilbert space vectors (this is the so-called projective Hilbert space).
Yet, the geometric phase is a physically relevant (and experimentally observable) quantity in the following sense:
Suppose you prepare your system in a certain quantum state, then split it iton 2 components. For example this is what is done when you pass a beam of spin-polarized atoms through a Stern-Gerlach apparatus. You can then let one of the 2 components acquire a geometric phase, and then recombine them. The resulting quantum superposition depends on the geometric phase, and this can be detected experimentally. This should be thought of as an interference experiment, in which the pase of one of the components is modified, whereas the other one only serves as a reference phase. So, strictly speaking, one can never measure a geometric phase, but only a difference of geometric phases.