Solving the Kepler problem requires a bit of vector calculus, but fortunately
the calculations are fairly straightforward. Let's start by defining polar unit
vectors
\begin{align}
\boldsymbol{e}_r &= \cos\theta \boldsymbol{e}_x + \sin\theta \boldsymbol{e}_y,\\
\boldsymbol{e}_\theta &= -\sin\theta \boldsymbol{e}_x + \cos\theta \boldsymbol{e}_y,
\tag{1}
\end{align}
with inverse
\begin{align}
\boldsymbol{e}_x &= \cos\theta \boldsymbol{e}_r - \sin\theta \boldsymbol{e}_\theta,\\
\boldsymbol{e}_y &= \sin\theta \boldsymbol{e}_r + \cos\theta \boldsymbol{e}_\theta.
\tag{2}
\end{align}
We will also need the time derivatives of $\boldsymbol{e}_r$ and $\boldsymbol{e}_\theta$:
\begin{align}
\dot{\boldsymbol{e}}_r &= -\sin\theta\,\dot{\theta} \boldsymbol{e}_x
+ \cos\theta\,\dot{\theta} \boldsymbol{e}_y = \dot{\theta}\boldsymbol{e}_\theta,\\
\dot{\boldsymbol{e}}_\theta &= -\cos\theta\,\dot{\theta} \boldsymbol{e}_x -
\sin\theta\,\dot{\theta} \boldsymbol{e}_y = -\dot{\theta}\boldsymbol{e}_r.
\tag{3}
\end{align}
The position and velocity of the object are then given by
\begin{align}
\boldsymbol{r} &= r\boldsymbol{e}_r,\\
\boldsymbol{v} &= \dot{r}\boldsymbol{e}_r + r\dot{\theta}\boldsymbol{e}_\theta,
\tag{4}
\end{align}
and the acceleration $\boldsymbol{a}$ is of the form
\begin{equation}
\boldsymbol{a} = -\frac{k}{r^2}\boldsymbol{e}_r,
\tag{5}
\end{equation}
for some constant $k$. From this, it follows that the energy $E$ and angular
momentum $\boldsymbol{h}$ per unit mass are conserved:
\begin{align}
E &= \frac{1}{2}v^2 - \frac{k}{r} = \frac{1}{2}\dot{r}^2 + \frac{h^2}{2r^2}
- \frac{k}{r},\\
\boldsymbol{h} &= \boldsymbol{r} \times \boldsymbol{v} = r\boldsymbol{e}_r \times r\dot{\theta}\boldsymbol{e}_\theta =
r^2\dot{\theta}\boldsymbol{e}_z.
\tag{6}
\end{align}
We can now use the magnitude of the angular momentum $h=r^2\dot{\theta}$ to
eliminate $r^2$ in the equation of motion $(5)$:
\begin{equation}
\boldsymbol{a} = -\frac{k}{h}\dot{\theta}\boldsymbol{e}_r =
\frac{k}{h}\dot{\boldsymbol{e}}_\theta,
\tag{7}
\end{equation}
using $(3)$. If we integrate this, we obtain
\begin{equation}
\boldsymbol{v} = \frac{k}{h}\boldsymbol{e}_\theta + \boldsymbol{v}_\varepsilon,
\tag{8}
\end{equation}
where the integration constant is a constant vector $\boldsymbol{v}_\varepsilon$.
Notice that the first term is a velocity with constant magnitude $k/h$, rotating with
$\boldsymbol{e}_\theta$. In other words, this is a velocity on a circle:
\begin{equation}
\boldsymbol{v}_\text{c} = \frac{k}{h}\boldsymbol{e}_\theta.
\tag{9}
\end{equation}
The total velocity is therefore the sum of a circular velocity and a constant boost:
\begin{equation}
\boldsymbol{v} = \boldsymbol{v}_\text{c} + \boldsymbol{v}_\varepsilon.
\tag{10}
\end{equation}
We can freely choose the direction of $\boldsymbol{v}_\varepsilon$. Note that
for $\theta=0$, we have $\boldsymbol{v}_\text{c} = (h/k)\boldsymbol{e}_y$. It's therefore
convenient to choose $\boldsymbol{v}_\varepsilon$ in the $y$-direction as well, so
that $\boldsymbol{v}$ is also in the $y$-direction for $\theta=0$. We write
\begin{equation}
\boldsymbol{v}_\varepsilon = \frac{k}{h}\varepsilon\boldsymbol{e}_y,
\tag{11}
\end{equation}
where $\varepsilon\geqslant 0$ is a dimentionless constant, so that
\begin{equation}
\boldsymbol{v} = \frac{k}{h}\left(\boldsymbol{e}_\theta +
\varepsilon\boldsymbol{e}_y\right).
\tag{12}
\end{equation}
Using $(2)$ and $(4)$, we can write everything in polar coordinates, and we get
\begin{equation}
\dot{r}\boldsymbol{e}_r + r\dot{\theta}\boldsymbol{e}_\theta =
\frac{k}{h}\varepsilon\sin\theta \boldsymbol{e}_r +
\frac{k}{h}\left(1 + \varepsilon\cos\theta \right)\boldsymbol{e}_\theta.
\tag{13}
\end{equation}
Equating the terms in $\boldsymbol{e}_\theta$ on both sides then leads to
\begin{equation}
r\dot{\theta} = \frac{h}{r} = \frac{k}{h}\left(1 + \varepsilon\cos\theta \right),
\tag{14}
\end{equation}
so that finally we arrive at the equation of the orbit in polar coordinates:
\begin{equation}
r = \frac{h^2}{k}\frac{1}{1 + \varepsilon\cos\theta}.
\tag{15}
\end{equation}
The radial distance has a minimum $(\dot{r}=0)$ at $\theta=0$:
\begin{equation}
r_\text{m} = \frac{h^2}{k}\frac{1}{1 + \varepsilon}.
\tag{16}
\end{equation}
We can plug this into the equation of the energy per unit mass $(6)$:
\begin{align}
E &= \frac{h^2}{2r_\text{m}^2} - \frac{k}{r_\text{m}}
= \frac{k^2}{2h^2}(\varepsilon^2 -1).
\tag{17}
\end{align}
For $0\leqslant \varepsilon < 1$, the energy is negative and the orbit is bound;
for $\varepsilon > 1$, the energy is positve and the orbit is unbound. We can see
what happens if we write the velocity $(12)$ in cartesian coordinates:
\begin{equation}
\boldsymbol{v} = -\frac{k}{h}\sin\theta \boldsymbol{e}_x +
\frac{k}{h}\left(\cos\theta + \varepsilon\right)\boldsymbol{e}_y.
\tag{18}
\end{equation}
If $\varepsilon = 0$, the orbit is circular. If we increase the energy for fixed
angular momentum, we increase $\varepsilon$. The object gets an extra boost in
the $y$-direction and will deviate more and more from a circle. For $\theta > 0$
and $\cos\theta > \varepsilon$, we have $v_y > 0$ and the object will move away
from the $x$-axis. If we want the object to return, $v_y$ needs to become negative
at some point. This will happen if $\cos\theta < \varepsilon$ for some values of $\theta$, which is only possible when $\varepsilon < 1$.
Therefore, if $\varepsilon > 1$ the object will escape. At infinity, we find from $(15)$ and $(18)$
\begin{align}
\cos\theta_{\pm\infty} &= -\dfrac{1}{\varepsilon}\\
\boldsymbol{v}_{\pm\infty} &= -\frac{k}{h}\sin\theta_{\pm\infty} \boldsymbol{e}_x +
\frac{k}{h}\left(\cos\theta_{\pm\infty} + \varepsilon\right)\boldsymbol{e}_y.
\tag{19}
\end{align}
Why are these orbits conic sections? I can't think of an intuitive answer. You
can do the algebra starting from $(15)$; see SuperCiocia's answer (note that his version of the equation
has a minus sign in the denominator, because he defined
$\boldsymbol{v}_\varepsilon = -(k/h)\varepsilon\boldsymbol{e}_y$). I will use
a different approach and offer a geometric proof. Richard Feynman famously did
this for elliptical orbits in his Lost Lecture (see also this post); I will
use a similar idea for hyperbolic orbits.
First, let us examine the orbits in velocity space. That is, we plot the
velocity vectors in a $(v_x, v_y)$-coordinate system. From $(10)$, it follows
that the velocity vectors must lie on a circle that is offset in the
$v_y$-direction. The plots below show the orbits for ellipses and hyperbolae,
both in ordinary coordinate space and in velocity space.
There's a certain irony here: ancient philosophers were convinced that the planets
were manifestations of the divine and had to move across the heavens on perfect circles; when observations
showed that they didn't, people desperately tried to remedy the situation by
adding epicycles, which over time became a mess. It turns out that they had been looking in the wrong
place: if they had been able to map out the planets' velocities, they
would've found their divine circles!


The crucial difference between elliptical orbits and hyperbolic orbits is that
in the former case the origin of the $(v_x, v_y)$-coordinates $O$ lies inside the
circle ($v_\varepsilon < v_\text{c}$), and in the latter case outside ($v_\varepsilon > v_\text{c}$). Also note that for hyperbolic orbits
the velocities don't sweep out a full circle, but a circle segment between
$\theta_{\pm\infty}$ (the purple arc on the velocity diagrams).
On to the proof. First, we rotate the velocity diagram clockwise $90^\circ$, so that
it looks like the figure below:

Take a point $A$ on the circle. The line $OA$ intersects the circle in a second
point $B$. Then, we draw a line through $CB$. Finally we draw a line through $O$
parallel to $CA$. This will intersect the line through $CB$ in a point $P$. The
triangle $ABC$ is isosceles. Since $AB$, $BC$, $CA$ are parallel to $OB$, $BP$,
$PO$ respectively, the triangle $OBP$ is also isosceles.
Therefore $|OP|$ = $|BP|$, and $|CP|-|OP| = |CP|-|BP| = |CB| = k/h$, which is
constant, for every point $A$. This means that $P$ traces out a hyperbola with
foci $O$ and $C$.
Furthermore, the line $PQ$ perpendicular to $OA$ bisects the
angle $\widehat{OPB}$. For every other point $P'\neq P$ on $PQ$, we get
$|CP'| - |OP'| < |CB| + |BP'| - |OP'| = |CB|$. So the line through $PQ$
only intersects the hyperbola in one point, $P$, and is therefore a tangent
line to the hyperbola.
But since we rotated our velocity diagram $90^\circ$, the corresponding
velocity vector is now perpendicular to $OA$ and parallel to $PQ$, and thus also
tangent to the hyperbola. And because the velocity vectors are tangent to the
orbit, it follows that the orbit is also a hyperbola. QED.