Do we require full Lagrangian to be gauge invariant or only equations of motion?
With new potential $A'^\mu$ also a new Lagrangian density $\mathcal{L}'$ is implied which is the same function of its parameters (potential), but is to be used in the new gauge with new arguments (quantities that are put in to evaluate the function). Namely $A'^\mu$ is used instead of $A^\mu$. So it is the same function, but may have different value for a particular spacetime point $\mathbf x,t$ because different potential is used.
With the gauge transformation
$$
A^\mu \rightarrow A'^\mu = A^\mu + \partial^\mu f
$$
the value of the new Lagrangian density for the point $\mathbf x,t$ is
$$
\mathcal{L}' = -\frac{1}{4}F'^{\mu\nu}F'_{\mu\nu} + j^\mu A'_\mu
$$
while the value of the old Lagrangian density for the same point is
$$
\mathcal{L} = -\frac{1}{4}F^{\mu\nu}F_{\mu\nu} + j^\mu A_\mu.
$$
The FF terms have the same value since FF term is a function of gauge-invariant fields $\mathbf E,\mathbf B$, but the jA terms have different value. The new Lagrangian density has higher value by the amount
$$
j^\mu\partial_\mu f.
$$
However, this difference in value among the two Lagrangian densities does not lead to any difference in values of actions
$$
S[A^\mu] =\int_V\, d^3 \mathbf x \int_{t_1}^{t_2}\, dt \mathcal{L}(A^\mu)
$$
$$
S'[A'^\mu] =\int_V\, d^3 \mathbf x \int_{t_1}^{t_2}\, dt \mathcal{L}'(A'^\mu)
$$
if $\mathbf j$ or $\nabla f$ vanish on the boundary of the region $V$ all the time and $\rho$ or $\partial_tf$ vanish at times $t_1,t_2$ everywhere. This is because the difference
$$
\int_V\, d^3\mathbf x \int_{t_1}^{t_2}\,dt~ j^\mu\partial_\mu f.
$$
can be transformed into
$$
\int_V\, d^3\mathbf x \int_{t_1}^{t_2}\,dt~ \partial_\mu(j^\mu f) -\int_V\, d^3\mathbf x \int_{t_1}^{t_2}\,dt~ \partial_\mu j^\mu f.
$$
The second integral is zero because of the equation
$$
\partial_\mu j^\mu = 0
$$
that current density obeys and the first integral can be transformed into surface integral (via Gauss theorem) and is zero if above conditions hold.
Summary: both Lagrangian density and action are gauge-independent as functions of their parameters - potentials. Lagrangian density value, however, is not gauge-independent, because the parameter has different value - $A'^\mu$ instead of $A^\mu$. Action value is gauge-independent, as difference in Lagrangian density integrates to give zero contribution.