Power is, in words, the rate at which work is done and work done equals the amount of energy converted from one form to another.
For electric circuits, the power associated with a circuit element is the product of the voltage across and the current through.
We can verify this via dimensional analysis:
$$v \cdot i = \frac{J}{C}\cdot \frac{C}{s} = \frac{J}{s}$$
Where we use the passive sign convention, the power is positive if power is delivered to the circuit element and negative when power is supplied by the circuit element.
For an inductor, we have
$$v_L = L \frac{di_L}{dt}$$
thus, the power associated with an inductor is
$$p_L = v_Li_L = Li_L\frac{di_L}{dt}$$
The work, due to the current through and voltage across is then
$$w(t) = \frac{1}{2}Li(t)$$
But this is just the equation for the energy stored in the magnetic field (as it must be).
Additional insight can be gained by looking at mechanical-electrical analogies.
For example, the product of force and velocity is mechanical power.
We are free to choose which mechanical variable is analogous to which electrical variable. So, if we choose velocity analogous to current (thus, force analogous to voltage) then an inductor is analogous to a mass.
$$F = m\dfrac{dv}{dt} \Leftrightarrow v = L \frac{di}{dt}$$
Then, the mass's kinetic energy (equal to the work done on the mass) is analogous to the inductor's magnetic energy (equal to the work done on the inductor):
$$\frac{1}{2}mv^2 \Leftrightarrow \frac{1}{2}Li^2$$