As far as I'm aware, the most popular/rigorous way of deriving $E$ field energy density is starting with the potential energy of a continuous charge distribution (which itself has some flaws that I will also discuss)
From this expression, $$ 1/2 \int \phi \rho dv \tag{1}$$
it can be shown that in terms of the fields this becomes
$$ \epsilon /2( \int E^2 dv + \int \phi E.da)\tag{2} $$
now... this surface integral, as $v$ goes the infinity you are left with
$$ \epsilon /2\int E^2 dv $$
People then say that $ \epsilon /2 E^2 $ can be interpreted as the energy density of the electric field
This energy density is used in the Poynting vector and Poynting's theorem which is fundamental to electromagnetism.
Issues I have with this:
Why is this energy density used to calculate things where the volume interested in is not infinite? We know from (2) that this is only true if the volume is infinity, so how can this be used in e.g. Poynting's theorem for non infinite volumes
This leads me on to my second issue. Deriving this formula, as far as I'm aware, requires you to start from the electrostatic potential energy of charges. So how can this be used to determine the energy of electrodynamical phenomena such as the energy of an electromagnetic wave? This second point is actually in line with my first issue as this surface integral contains the potential function, which only makes sense in the electrostatic case.
As stated in first line l, I also have issues with the potential energy of a continuous charge distribution:
This is because, in going from a discrete set of charges to a continuous set, you actually include the potential from the charge that you are "placing" in space in the presence of other charge distributions. As also said in Griffiths introduction to electrodynamics. Now this doesn't matter when working with an average continuous charge distribution as the contribution from a single charge density is zero, however by adding the contribution from this charge that you "place" in position you get problems like the infinite potential energy of a single point charge as you in theory should exclude the contribution of the charge you place but because a point charge exists in 1 single point, the otherwise "infinitesimal contribution" is actually in fact the only contribution so you get a false result of infinite energy of a point charge when in reality it should be zero? So why do we use this broken formula in the first place? (I'm no expert but if Griffiths even has stated the jump from discrete to continuous isn't sound, then I don't know why we use it.)
Edit: Griffiths also states that integrating (1) over any volume will give the same result since $p=0$ for the empty space he also applies this logic to (2). So does this mean integrating (2) over a region of space where there is no charge density will give 0 for the answer? Then how can we say energy is stored in the field if this specific case it is zero? Which circles back to the point where, why is this valid for non infinite volumes.