Fundamental particles are identical.
If you have two electrons, one from the big bang and the other freshly minted from the LHC, there is no experiment you can do to determine which one is which. And if there was an experiment (even in principle) that could distinguish the electrons then they would actually behave differently.
Electrons tend to the lowest energy state, which is the innermost shell of an atom. If I could get a pen and write names on all of my electrons then they would all fall down into this state. However since we can't tell one electron from another only a single (well actually two since there are two spins states of an electron) electron will fit in the lowest energy state, every other electron has to fit in a unique higher energy level.
Edit: people are making a lot of comments on the above paragraph and what I meant by making electrons distinguishable, so I will give a concrete example: If we have a neutral carbon atom it will have six electrons in orbitals 1s2 2s2 2p2. Muons and tauons are fundamental particles with very similar properties to the electron but different masses. Muons are ~200 times more massive than electrons and tauons are ~3477 times more massive than an electron.
If we replace two of the electrons with muons and two of the electrons with tauons all of the particles would fall into the lowest energy shell (which can fit two of each kind because of spin). If in theory these particles only differed in mass by 1% or even 0.0000001% they would still be distinguishable and so all fit on the lowest energy level.
Now atoms are not fundamental particles they are composite, I.E. composed of "smaller" particles, electrons, protons and neutrons.
Protons and neutrons are themselves composed of quarks. But because of the way that quarks combine, they tend to always be in the lowest energy level so all protons can be considered identical, and similarly with neutrons.
To take the example of carbon, there are several different isotopes, different number of neutrons, of carbon (mostly 12C but also ~1% 13C and ~0.0000000001% 14C {the latter which decays with a half life of ~5,730 years [carbon dating] but is replaced by reactions with the sun's rays in the upper atmosphere}).
If we take two 12C atoms, and force all of the spins to be the same. This is not too difficult for the electrons of the atom since the inner electrons do not have a choice of spin because every spin in every level is already full. So only outer electrons matter. The nucleons also have spin.
With our two 12C atoms with all of the same spins, we now have two indistinguishable particles which if you set up an appropriate experiment (similar in principle to the electrons not being able to occupy the same state) we will be able to experimentally prove that these two atoms is indistinguishable.
Answer time:
Are atoms unique?
No.
Do atoms have any uniquely identifying characteristic besides their history?
Their history of a particle does not affect it*. No particles are unique. Atoms may have isotopes or spin to identify one from another, but these are not unique from another particle with the same properties.
would it contain information with which we could positively identify that they two are the same?
Yes only because we could positively identify that this carbon atom is the same as almost every other carbon atom in existence.
*Unless it does, in which case it may be considered a different particle with different properties.