In addition to what has already been said I would like to add some more comments. I completely understand your suspicion that the passage from unbounded operators to bounded ones is at least tricky. For the canonical commutation relations of position and momentum operators this can be solved in a reasonable and also physically acceptable way by passing to the Weyl algebra (OK; there are zillions of Weyl algebras around in math, but I'm refering here to the $C^*$-algebra generated by the exponentials of $Q$'s and $P$'s subject to the heuristic commutation relations arising from $[Q, P] = \mathrm{i} \hbar$).
However, there are other situations in physics where this is much more complicated: when one tries to quantize a classical mechanical system which has a more complicated phase space than just $\mathbb{R}^{2n}$ then one can not hope to get some easy commutation relations which allow for a Weyl algebra like construction. To be more specific, for general symplectic of Poisson manifolds the beast quantization scheme one can get in this generality is probably formal deformation quantization. Here the classical observable algebra (a Poisson algebra) is deformed into a noncommutative algebra in a $\hbar$-dependent way such that the new product, the so-called star product, depends on $\hbar$ in such a way that for $\hbar = 0$ one recovers the classical mutliplication and in first order of $\hbar$ one gets the Poisson bracket in the commutator.
Now two difficulties arise: the most severe one is that in this generality one only can hope for formal power series in $\hbar$. Thus one has even changed the underlying ring of scalars from $\mathbb{C}$ to $\mathbb{C}[[\hbar]]$. So there is of course no notion of a $C^\ast$-algebra whatsoever on this ring. Nevertheless, there is a good notion of states in the sense of positive functionals already at this stage. Second, even if one succeeds to find a convergent subalgebra one does usually not end up with a $C^\ast$-algebra on the nose. Worse: in most of the explicit exampes one knows (and there are not really many of them...) the commutation relations one obtains are very complicated. In particular, it is not clear at all how one can affiliate a $C^\ast$-algebra to them.
Moreover, it is not clear which of the previous states survive this condition of convergence and yield reasonable representations by the GNS construction.
It takes quite some effort to first represent the observables by typically very unbounded operators in a reasonable way and then show that they give rise to some self-adjoint operators still obying the relevant commutation relations. This is far from being obvious.
To get a flavour for the difficulties it is quite illustrative to take a look at the book of Klimyk and Schmüdgen on Quantum Groups adn their Representations. Note that in these examples one still has a lot of structure around which helps to understand the analysis.
However, physics usually requires still much more general situations. Most important here are systems with gauge degrees of freedoms where one has to pass to a reduced phase space. Even if one starts with a geometrically nice phase space the reduced one can be horribly complicated. This problem is present in any contemporary QFT really relevant to physics :(
For other quantization schemes things are similar, even though I'm not quite the expert to say something more substantial :)
So one may ask the question why one should actually insist on $C^\ast$-algebras and this strong analytic background. There is indeed a physical reason and this is that quantum physics predicts not only expectation values of observables in given states (here the notion of a ${}^\ast$-algebra and a positive functional is sufficient) but also the possible outcomes of a measurement: they are given by particular numbers called the physical spectrum of the observable. To get a good description with predictive power the (as far as I know) only way to achieve this is to say that the physical spectrum is givebn by the mathematical spectrum of a self-adjoint operator in a Hilbert space. If one is here at this point, then the passage from a (unbounded) self-adjoint operator to a $C^\ast$-algebra is comparably easy: one has the spectral projections and takes e.g. the von Neumann algebra generated by them...
So the point I would like to make is that it is very desirable from a physical point of view to have the strong analytic framework for observables as either self-adjoint operators or Hermitian elements in a $C^\ast$-algebra. But the quantum theory of many nontrivial systems requires a long long and nontrivial way before one ends up in this nice heavenly situation.
