In general, assume that $\mathcal{H}$ is a separable Hilbert space and that $A:\mathcal{H}\rightarrow\mathcal{H}$ is a linear operator. The hilbert space $\mathcal{H}$ being separable, admits a countable basis, which we can write as $\eta_i$ with $i\in\mathbb{Z}$ (these are not necessarily the $\delta$ functions that you speak of, but $\mathcal{H}$ being separable is isomorphic to $\ell^2(\mathbb{Z})$, in which case each $\eta_i$ can be mapped onto $\delta_j\in\ell^2(\mathbb{Z})$, where $\delta_j$, $j\in \mathbb{Z}$, are honest $\delta$ functions.
Now assume that $A$ admits eigenvectors $e_j\in \mathcal{H}$, $j\in\mathbb{Z}$, such that the following is satisfied.
1) $\{e_j\}_{j\in\mathbb{Z}}$ are linearly independent (in the sense that any finite subset forms a set of linearly independent vectors.
2) $\{e_j\}_{j\in\mathbb{Z}}$ is complete in the sense that the set of all finite linear combinations of elements from $\{e_j\}$ forms a dense subset of $\mathcal{H}$.
Then we see that $\{e_j\}$ forms a complete bases of $\mathcal{H}$. After normalizing each $e_j$ (or we can assume that they have already been normalized) we can now unitarily map everything into $\ell^2(\mathbb{Z})$, by mapping $e_j$ onto $\delta_j$. This is possible if and only if conditions (1) and (2) are satisfied.
In other words: every linear operator on a separable Hilbert space can be unitarily mapped to an operator on $\ell^2(\mathbb{Z})$. In this case, if the operator admits a set of eigenvectors that forms a complete basis, then this set can be mapped onto honest delta-functions of $\ell^2(\mathbb{Z})$.
Of course, if condition (2) is relaxed, then one can instead consider a subspace of the original Hilbert space which is the closure of the set of all finite linear combinations of the eigenvectors. The operator would also have to be bounded (equivalently, continuous) to pass from a dense subset to its closure.