Will the atmospheric pressure exerted on the upper surface of it be equal to that of the pressure exerted on its sides?
No, the pressure on the sides will be slightly greater than the pressure on the top and slightly less than the pressure on the bottom. The pressure difference is given by $$\Delta P = \rho \ g \ \Delta h$$ where $\rho$ is the density of the fluid, $g$ is the local acceleration of gravity, and $h$ is the average height of the surface. This pressure difference produces a net upward force called buoyancy.
This formula applies only when the fluid is static. If the fluid is flowing then different formulas must be used.
Now, I suspect that the question that you wanted to ask is not about the pressure on different sides of the same object (since those will be at different heights) but about the pressure on the faces of different objects at the same average height, where the faces of the different objects are at different angles with respect to the vertical direction.
Those will be the same. Notice that in the formula for the pressure there is no angle, only the height. There is no change in the based on the angle of the surface.
Pressure is the isotropic part of the stress tensor. So if the force did depend on the angle then it would be described by a non-isotropic stress tensor rather than a pressure. Now, recall the formula above, that it assumed a static fluid. For a fluid to be static it must have no shear stress, if there is any shear stress then the fluid will deform. It can be shown that the only stress tensor that has no shear stress is one that is isotropic.
So the bottom line is that the fact that the fluid is static implies that there is no shear stress, which implies that the stress tensor is isotropic, which implies that the pressure force is the same in every direction.