The Lagrangian for ED without Gauge fixing term is given by $$\mathcal{L}=-\frac{1}{4}F^{\mu\nu}F_{\mu\nu},\quad \text{where}\quad F_{\mu\nu}:=\partial_\mu A_\nu-\partial_\nu A_\mu.$$
I was wondering if this step, defining $F_{\mu\nu}$ over the $4$-potential $A_\mu$, is necessary. Can't we just formulate electrodynamics in terms of the tensor $F_{\mu\nu}$? That is, set $$F_{\mu\nu}:=\begin{bmatrix}0&E_{x}&E_{y}&E_{z}\\-E_{x}&0&-B_{z}&B_{y}\\-E_{y}&B_{z}&0&-B_{x}\\-E_{z}&-B_{y}&B_{x}&0\end{bmatrix}$$ and then derive the Maxwell equations directly from here, without going over the $4$-potential? If this doesn't work, what exactly is the problem?