Lots of questions involved here, but I don't think you will get a complete derivation of all that you list without some digging through textbooks and on the net, on your part.
in many fields of physics, we usually begin with an ordinary formulation (e.g Newton's Laws in classical mechanics), and then move on the Lagrangian, then Hamiltonion, and finally the action formulation. However, I don't understand how this chain of fomulations are derived, one step at a time
For this part of your question, depending on the specific problem, for example Quantum Field Theory, it may be easier to obtain the Lagrangian first, and then explore the implications of that. It does not depend on how it is taught to us, the equations exist, that's all that matters and we use them to suit the problem at hand.
I am confused as to how the different fomulations in physics are derived.In many fields of physics, we usually begin with an ordinary formulation (e.g Newton's Laws in classical mechanics), and then move on the Lagrangian, then Hamiltonion, and finally the action formulation. However, I don't understand how this chain of fomulations are derived, one step at a time.
Three important words, wiki, wiki, wiki. You will find the derivations of all of them there or on Google "derivation of Lagrangian, etc". lots of college notes on classical mechanics cover their derivation and probably lectures on YouTube by this time.
There is no space on this forum to go through the derivation of all you ask, but could I recommend a book by Mary Boas: Mathematical Methods in the Physical Sciences, which, imo, goes through a lot of what you will need later on, or just the derivations you ask about. Also try amazon and the Schuaum books.
Are there any ways to derive the Lagrangian and action, without just guessing or being given a specific Lagrangian?
The Lagrangian is usually given as a density, and the action is the integral of the density with respect to time. No offence intended, but the more you read and practice the problems, the easier it is to follow what the action represents. It's practice and more practice and then ask questions here on specific questions.
Best of luck with it