I understand that, at the more fundamental physical level, waves are phenomena of Fields. Like electromagnetic waves of the electromagnetic field.
However, I also know that we can have waves in Mediums. Like water waves in ocean water.
So, are those mediums, a kind of field? Is the ocean a field?
I'm a bit confused as to how to properly use both terms in physics.
No.
It is incorrect to say that a medium "is" a field. A medium is often conveniently described by properties that are fields. Like density, pressure, displacement, concentration of certain chemicals etc.
Also note that the description of a medium by a field may be just one among others. For example, you can study the deformation of a solid using its displacement field and the continuum theory of elasticity, while there also is the description in terms of lattice models. The latter would be discrete and not involve a field to describe the deformation of the medium.
In physics, a field is a physical quantity, represented by a scalar, vector, or tensor, that has a value for each point in space and time.
So yes, indeed, (continuous) distributions of density, pressure, etc. in continuous media are fields. Moreover, in hydrodynamics and elasticity they obey well-known field equations. In condensed matter physics some of these the fields are even quantized - like phonons (vibration quanta), plasmons (quanta of hydrodynamic plasma oscillations), etc.
