Higher order derivatives (usually?) correspond to non-renormalizable terms in the Lagrangian. Every term in the Lagrangian density needs to have units of $[M^4]$, so that the action is unitless (in units where $\hbar = 1$). For example, take a scalar field $\phi$. This has units $[M]$, or equivalently $[1/D]$. Each derivative adds a factor of $[M] = [1/D]$ (think of it as $\delta / \delta x$).
With no derivatives, you can form the standard mass term
$$
\frac{1}{2} m^2 \phi^2
$$
The coefficient $m$ must thus have units $[M]$, making it a mass.
You can't form a Lorentz scalar with only one derivative, since $\partial_\mu \phi$ is a one-form and without breaking rotational invariance there is no vector to take an inner product with. The next possibility is two derivatives. The only allowable term is
$$
\frac{1}{2} \left( \partial_\mu \phi \right) \left( \partial^\mu \phi \right)
\underset{\text{parts}}{\sim}
- \frac{1}{2} \phi \partial^2 \phi
$$
This term doesn't have a unitful coefficient, because the two $\phi$'s and two $\partial$'s make it have units $[M^4]$ already. A term $m \partial^2 \phi$ is equivalent to $0$ by integration by parts, so with no boundary and fields that die off at infinity they don't matter. A term like $m(x) \partial^2 \phi$ would contribute, but breaks translational invariance.
What about three derivatives? Same as with one - there is no four-vector to contract the third derivative with, so this is impossible without breaking rotational (and possible translational) invariance with some preferred four-vector field $v^\mu(x)$.
The next possibility is four derivatives. A generic possibility is
$$
\frac{1}{\Lambda^2} \left( \partial^2 \phi \right)^2
$$
Because of the four derivatives and two appearances of $\phi$, the term needs a coefficient with units $[M^{-2}]$, which is why I've added $1/\Lambda^2$ where $\Lambda \sim [M]$. An inclusion of this term requires a mass scale $\Lambda$. Like the Fermi four-fermion theory, this would violate unitarity at energies $\sqrt{s} \gg \Lambda$.
You can go through the same exercise for spin-1/2 and spin-1 fields and reach much the same conclusion, although the details will be different.