Attracting something involves energy
"Involves energy", true, "require an energy input", false. Gravity for instance can also be attractive without requiring any energy input. Gravitational interactions may involve potential energy, however. I think you're just confusing the two concepts.
Moreover, the term "permanent magnet" is very misleading; they are not permanent at all. Permanent magnets are made by magnetizing ferromagnetic material, a process that normally requires a substantial energy input. It is true that their magnetized state is a method for storing potential energy. This energy can be converted into, for example, kinetic energy (when they attract objects), which usually dissipates to the surroundings. If you then release the stuck object again, that energy is simply lost.
Note that no energy is required to cause a force (for example, when objects "stick" to the magnet, or you leave a fridge magnet stuck to the fridge).
Over time, any permanent magnet will demagnetize. The more you use it, the faster it will demagnetize. They will even lose strength when you don't use them at all.
If you do the calculation, the energy that was put in in order to magnetize it, is far more than the energy you finally get out of it. In that sense, it isn't even a particularly efficient storage device.