I think Griffiths means that we assume that
\begin{equation} \lim_{x \rightarrow \infty} V(x) = V_R > E \end{equation}
i.e. that the potential tends to a constant value $V_R$ that is larger than $E$. In this case we can prove it as follows:
As OP writes, this means that for some $X_R$ and some $\epsilon > 0$, we have
\begin{equation} | V(x) - V_R | < \epsilon \quad \forall x > X_R \end{equation}
Let us choose $X_R$ large enough so that
\begin{equation} V(x) > E \quad \forall x>X_R \end{equation}
and approximate $V(x)$ by $V_R$ in this region. The error will be $\epsilon$, which we can make as small as we want by choosing $X_R$ large enough (but finite).
In this case, the Schrödinger equation reads
\begin{equation} -\frac{\hbar^2}{2m}\psi_E''(x) + V_R \psi_E(x) = E\psi_E(x) \quad \forall x>X_R \end{equation}
which means
\begin{equation} \psi_E''(x) = \frac{2m}{\hbar^2} (V_R - E) \psi_E(x) \quad \forall x>X_R \end{equation}
Since $V_R - E > 0$ by assumption, this is a differential equation with solution
\begin{equation} \psi_E (x) = C_1 \exp \left(x \sqrt{\frac{2m(V_R - E)}{\hbar^2}} \right) + C_2 \left(-x \sqrt{\frac{2m(V_R - E)}{\hbar^2}} \right) \end{equation}
Unless we want something non-normalizable, we have to set $C_1 = 0$. The value of $C_2$ must be determined by the value of $\psi_E$ and $\psi_E'$ for $x<X_R$. Hence the wave function decays exponentially. The same analysis obviously holds for negative $x$, just replace $V_R$ by $V_L$, $X_R$ by $X_L$, let $x < X_L$ etc. This is just the particle in a (reeeeeeeeally wide) finite box.
That being the case, the whole wave function is normalizable because the interval $[X_L, X_R]$ is finite and the wave function decays exponentially for $x<X_L$ and $x>X_R$.
For potentials that don't have a limit when $x \rightarrow \pm \infty$, e.g. periodic potentials, I don't think we can say much. Periodic potentials are solved by Bloch's theorem and they would give non-normalizable wave functions if we did not use periodic boundary conditions to study the bulk of the lattice.