We know that the momentum operator must be Hermitian since its eigenvalue gives the momentum which is measurable and hence must be real.
Now, when the momentum operator is written in the form
$$\hat{p}_x = -i \hbar \frac{\partial}{\partial x},$$
then when I perform the Hermitian conjugation, it becomes
$$\hat{p}^\dagger_x = i \hbar \frac{\partial}{\partial x} = -\hat{p}_x$$
which makes the operator skew-Hermitian. What's gone wrong?
Remember that when you write $p_x$ in that form, you are implicitely treating it as an operator on the function space $L^2$, where the scalar product is defined by
$$ \langle\psi,\phi\rangle=\int dx \psi(x)^* \phi(x)$$
To take the adjoint, you are just taking the complex conjugate of the multiplicative constant $i\hbar$, but you have to actually take the adjoint of the operator $\partial_x$ in this space. The adjoint of an operator $A$ is defined by the relation $$ \langle\psi,A\phi\rangle=\langle A^\dagger\psi,\phi\rangle$$
we have by integration by parts
$$ \langle\psi,A\phi\rangle=\int dx \psi(x)^* i\hbar\partial_x\phi(x)=i\hbar\left(\psi^*\phi|_{-\infty}^\infty-\int dx \partial_x \psi^*\phi\right)$$
we use that wavefunctions must vanish at infinity to get rid of the boundary term
$$ \langle\psi,p_x\phi\rangle=\int dx \psi(x)^* i\hbar\partial_x\phi(x)=-i\hbar\int dx \partial_x \psi^*\phi=\int dx(i\hbar\partial_x\psi)^*\phi=\langle p_x\psi,\phi\rangle$$
hence $p_x^\dagger=p_x$
EDIT: I feel I should add that this is a rather sloppy derivation, from a mathematician's perspective (some mathematicians might say it's plain wrong), but it does convey a crude way to get the adjoint operator and delivers the correct result. In reality, the operator $\partial_x$ is unbounded hence one should be careful about the definition of the adjoint and where the adjoint is actually defined. See this question for a more rigorous treatment.
