The color of sky is no different than the color of grass, milk, trees etc.
Two basic kinds of light we perceive: direct light and reflected light. Direct light is only seen when it is perceived directly by our eyes. All other light is reflected from something. In the case of grass, milk, trees, it's obvious.
In the case of the sky, we perceive the light after it has been reflected from things that are present in the atmosphere such as gases, vapor, dust ... and, of course, the air itself.
And all of those things serve to diffuse the light as well. Nothing has a completely clean edge, it's always got a blurry boundary, if you look closely enough at it.
Where there is nothing to reflect from, there is no color. No light. No blurry edges.
In the case of sky, our atmosphere does not suddenly stop as if it has hit a brick wall, it becomes thinner and thinner and eventually fades away and becomes space.
If you look at images from the space station or other vehicles out there in space, you'll not be able to see the sky. That's because, for the purposes of our seeing light, it's essentially a vacuum. There's nothing for the light to be reflected from in the same way that it is in the atmosphere.
And every image has a sharp edge, because there's no atmosphere to diffuse the light gradually.
Similarly, if there aren't many things for the light to reflect from, or too few to be registered as reflections by our eyes, we don't see so much light.
When we see the sky, we must be somewhere. Usually (and always for people on Earth) the person doing the seeing is located in the atmosphere. So some light will always be reflected by the atmosphere that surrounds the person.
The amount of pure black, or "no-reflections", that you see depends upon your proximity to the vacuum of space. Unless you are really close to the vacuum, you will always see evidence of light being reflected in some degree from the atmosphere.
We see fewer and fewer reflections depending upon the amount of atmosphere between us and space in the direction we look. The darkest part will usually be directly above us, partly because that's usually where the atmosphere is thinnest, and mostly because the reflected light is reflected back up and away from us.