Representations represent group (algebra) elements as linear operators on a vector space, in physics the vector spaces are also usually Hilbert and the operators are often orthogonal or unitary (self-adjoint), so they do indeed introduce quite a bit of extra structure. The simplest piece to see is that the representation does not have to be faithful, i.e. the image of the group in the group of operators does not have to be isomorphic to the group itself. In descriptive terms, if we think of representation as describing symmetries of a physical system, only part (or better to say reduced version) of allowed symmetries may be present. But even if representations are faithful, and their images are isomorphic to the group, they may still implement symmetries differently. For example, one representation may be reducible, operators move vectors from some proper subspaces within them, and the other have no such subspaces, be irreducible (this is also described as having a cyclic vector that "generates" entire space). Breaking up reducible representations into irreducible ones can then be used to break it up into simpler components (often called superselection sectors), whose behavior is easier to analyze.
In infinite dimensional systems it is also possible that representations are not unitarily equivalent, there is no unitary map between their vector spaces that corresponds their operators accordingly. Since unitary equivalence reflects physical equivalence this means that different representations correspond to physically distinct systems. Wallace discusses this issue in detail in his Lagrangian QFT paper (Sec. 4), and gives a simple example:"To see what these sectors are, suppose we start with all components having spin up. Then the action of any element of the algebra can, at most, cause finitely many components to have spin down. So no amount of algebraic action can transform such a state into one in which, say, every second component has spin up. This state, in turn, can be transformed into other states differing from it in finitely many places, but not into a state in which all components are spin down..." In QFT itself "infrared"-inequivalent representations may reflect different mass-densities at infinity, different total charges, or inequivalent vacua for the corresponding systems of quantum fields. So the answer to the boldfaced question is no, differences in representations reflect differences in physics.