One gets there by noting that $\langle x | p \rangle = e^{i p x/\hbar}$ is a plane wave, and you have to throw on the test wave function to talk about the derivative operation. So, you are worried about
\begin{equation}
\langle x| P | \Psi \rangle = \int dp ~p~ e^{i p x/\hbar} ~\langle p | \Psi \rangle
\end{equation}
From there you note that you can get the $p$ inside the integral by taking an $x$-derivative and after some algebra and removing the identity operator for $|p \rangle \langle p|$ you find that
\begin{equation}
\langle x| P | \Psi \rangle = - i \hbar \partial_x \langle x | \Psi \rangle
\end{equation}
The point is that you really need to be careful about an operator and its basis-dependent representation. $\hat{P} = - i \hbar \partial_x$ in the x basis. It has other representations in other basis sets, and you have to specify that when you are writing down a concrete representation. It's similar to how the position vector $\vec{r}$ exists, but then the individual components have a basis representation in Cartesian, or cylindrical, or hyperbolic, or …, coordinates.
Edit: just worked through the derivation
You want to know the matrix element $\langle x|P|\Psi \rangle$. Then you have, clearly, that
\begin{equation}
\langle x | P | \Psi \rangle = \int dx' \langle x | P | x' \rangle \langle x' | \Psi \rangle
\end{equation}
From the commutator relation matrix element, you get that
\begin{equation}
\langle x'| \left [ X, P \right ]| x \rangle = - i \hbar \langle x' | x \rangle
\end{equation}
which leads to
\begin{equation}
- i \hbar \frac{\langle x' | x \rangle}{x' - x} = \langle x' | P | x \rangle
\end{equation}
so then we get that
\begin{equation}
\langle x | P | \Psi \rangle = \lim_{x' \rightarrow x} - i \hbar \frac{\langle x' | \Psi \rangle}{x' - x} = - i \hbar \partial_x \langle x | \Psi \rangle
\end{equation}
Make $|\Psi \rangle = | p \rangle$ and you get the plane wave solution, with the caveat that there may be a sign error or two in here.