I think it is useful to complement @Matteo's answer (for the part related to the surface charge density) with a discussion of the physical conditions justifying the existence of a discontinuous electric field.
It is essential to distinguish between microscopic and macroscopic fields. The microscopic fields are controlled by all the physical point-like sources in the system and may vary quite a lot with time and over microscopic distances. Still, there are always continuous (of course, they are not defined at each point where a charge is located, but that is not a discontinuity, according to the mathematical definitions).
Macroscopic fields are a different story. They can be obtained from the microscopic fields through time and spatial averages, over times and spatial scales large with respect to the atomic scale.
Such averaging process has profound mathematical consequences. Spatial and time variations generally become smoother but with an important exception. The interfaces between different homogeneous media are usually a few atomic layers wide. Suppose there is some pile-up of charge in such interfacial region at the macroscopic level. In that case, it has to be described as a charge distribution confined to the separation surface between the two media.
It is a consequence of Gauss law that such two-dimensional charge density introduces a real discontinuity in the normal values of the electric displacement field at the interface. I.e., introducing the field on the two sides of the surface (${\bf D}_1$ and ${\bf D}_2$)
$$
({\bf D}_1 - {\bf D}_2)\cdot {\bf \hat n} = \sigma.
$$
Notice that a similar discontinuity appears in the tangential component of the magnetic field ${\bf H}$ at a surface with a confined surface current density.
In both cases, the origin of the macroscopic discontinuities can be traced back to the need to describe sources at the interfaces in terms of surface densities.
In terms of field lines, the discontinuities do not introduce any inconsistency. Field lines start or end at a surface density, not only at point-like charges. However, in correspondence with the surface charge, the field has a finite value (does not diverge), but it is different on the two sides of the surface.