You can differentiate certain types of functions an infinite number of times and they never will become zero. See this related question: https://math.stackexchange.com/questions/1210819/infinite-number-of-derivatives.
After reading your edit of the question, I'm thinking this may be more a philosophy than physics question. What comes to mind is Zeno's paradox.
Zeno's teacher Parmenides taught that existence is a motionless continuum; that all movement is illusion. He illustrated this with the paradox of Achilles racing a turtle. As the Turtle moved, Achilles had to move also. As Achilles caught up with the Turtle, he covered half the remaining distance to the turtle. And then half of that distance, and half of that, and so on. But because half the distance always remained, Achilles never would catch the turtle. The distance between Achilles and the turtle is asymptotic, if the distance function is smooth and continuous.
You seem to me to be asking for a resolution of something like Zeno's paradox with regard to time. Is there a quantized unit of time that would make the rate of acceleration discontinuous?
The Planck time is the time duration for a photon to travel one Planck length in a vacuum. At the Planck length quantum mechanical effects become important. Does this mean the Planck time is the closest to instantaneous one can get? That may be what your question is about. According to the Wikipedia article linked above, a theory of quantum gravity is needed to determine a time scale shorter than the Planck time. So, your question is not resolvable currently.