As a supplement to Marco Ocram's excellent answer: we are all moving not only in space, but also in time. We have no choice about that: even if we think we are "at rest" in space, we'll be moving forward through time. But different observers may be moving in different directions in spacetime. If we assign $(x, t)$ coordinates to the path of a watch, with the beginning of the path at $(0,0)$, then after one of our seconds a stationary watch will have coordinates $(0, 1)$ whereas a moving watch will have coordinates $(v, 1)$. The vectors $(v, 1)$ and $(0, 1)$ clearly point in different directions and have different lengths. If you mark out 1 unit intervals along the $(v, 1)$ (moving) line, they won't have the same time or space coordinates as they would along the $(0, 1)$ line.
The only complication to all of this is that time and space are not the same thing, and so in practice the "distance" is calculated with $x^2 - t^2$ rather than $x^2 + t^2$. Time comes into it with a "negative" sign (actually the choice of signs for time and space are arbitrary, but they have to be opposite).
This also explains why when you bring the moving watch back it will show a shorter time. The "moving" watch goes around two sides of a triangle, where the "resting" moves along the third side (only in time, not in space). The moving watch travels a longer spatial distance. Time and space have opposite signs, so this corresponds to a shorter temporal distance, i.e. the watch that moved will show less elapsed time.