I have been learning it as the velocity required for an object to
never come back to the planet, however, I don't understand how this
works. Theoretically will it just keep moving forever? Why not for any
other velocities less than the escape velocity will this work?
If you are familiar with work, the object does work on against gravity because its traveling away from Earth. This is the same as if a block is sliding through a surface with friction, it does work against the friction. The amount of energy the object has available do work to against a force is equal to its Kinetic energy. If the block starts off faster, it will take longer to reach a stop. The equation for work is $$W = \int_{\text{the path of the object}}{\bf F}\cdot d{\bf r}$$
and this makes sense because we are adding (hence integral) the amount of work done by the object when it moves an infinitesimal amount $d{\bf r}$ in the presence of force ${\bf F}$. There are sign conventions here that have been skipped over (i.e. what positive and negative work mean).
If you try to find how much kinetic energy you need to be able to do work against gravity all the way to infinty, you need to solve for $v$ here:
$$\frac{1}{2}mv^2 = \int_{r_0}^\infty \frac{GMm}{r^2} dr =\frac{GMm}{r_0}$$
where I am assuming the object takes a path starting at $r_0$, so the surface. This solves to $v = \sqrt{\frac{2GM}{r_0}}$ and thus that is the velocity the particle needs to have enough energy to get to infinity, i.e. it will keep moving forever. If it has a lesser velocity, one can show it follows a parabolic path and falls back down (i.e. a regular person throws a baseball vertically up) and if it has a higher or equal velocity then it will have a hyperbolic trajectory and it will go to infinity.
Along with that, I don't understand why at a distance of infinite the gravitational potential energy is $0$.
This is a choice that we make. The potential energy could be written $-\frac{GMm}{r} + C$ for some constant $C$ and that would result in the same force of gravity (because the force is the derivative of the potential) and thus the same equations of motion for particles. Setting $C=0$ is a convention and the fact that we can change what it is is useful because then we can talk about potential energy relative to a point. Setting it to $0$ means we are comparing the potential energy relative to infinity. Setting $ C = +\frac{GMm}{r_0}$ would mean we are comparing potential energy relative to $r_0$. This is the same concept as how the formula $U = mgh$ (gravitational potential energy close to the earth surface) is dependent on where we set $h=0$. This is all because what matters is the energy difference (with respect to some point) and not the absolute energy.
And what about with respect to the planet? Say we considered the gravitational potential energy inside of a planet to be $0$, then what would it be at a distance of infinite?
Depends where. If we set it to $0$ at a point near the surface $r_0$, then $U(\infty) = +\frac{GMm}{r_0}$. The only place we cannot choose to set the potential to be $0$ is at $r=0$, the very center. This would not be useful because that would mean we would have to set $C=\infty$ which is not allowed or useful.