The greenhouse effect is due to two mechanisms, neither of which would cause it independently.
Mechanism 1 – insulation
Greenhouse gases are opaque to infrared light. They absorb infrared, then re-radiate it. To pass through a layer of gas like this, a unit of infrared energy does something like a random walk. It's enters the layer, is absorbed by gas, and is then re-radiated, either up or down. Up, and it escapes the layer (in to space, or back to Earth). Down and it continues on, further in to the layer, where it may be caught by another opaque molecule and need to do another up/down trial.
In this way, the gas layer acts as an insulator. As you increase the density or thickness of the layer, units of infrared energy take longer to get from one end to the other, because they need to perform more up down trials to pass through it. Infrared energy will persist where it is for longer, either at the Earth's surface, or in space.
Mechanism 2 – "creation" of infrared at the surface
The atmospheric insulating layer is only trapping infrared. It is largely transparent to visible light and ultraviolet, so visible light shines through to reach Earth's surface.
Some of the visible light that makes it to Earth is reflected back as visible light and radiates away unhindered in to space. However, much of it is absorbed by Earth, warming it up.
Earth then re-radiates the absorbed energy back out, but not with the original spectrum. Instead, the radiated energy is mostly in the infrared portion of the spectrum *.
Because the atmospheric layer is an infrared insulator, a unit of infrared light tends not to escape out in to space, but instead, persists at the Earth's surface being re-abrobed and re-emitted. I should note: heat will, with a random walk, eventually exit the atmosphere. Insulation just means it takes longer to escape. The better the insulation, the longer each unit of warmth persists at the surface.
Summary
The atmosphere is an insulating blanket. This would keep Earth as cool as it keeps it warm. However, because the blanket only works on infrared energy, visible light shining on Earth gets to "beam through" the blanket and "make" heat at the Earth's surface on the "inside" of the blanket.
Increasing CO2 (and other infrared opaque gases) in the atmosphere thickens the blanket. The thicker the blanket, the longer a unit of heat persists at the surface, leading to a rise in temperature.
Caution
The mechanism of the greenhouse effect was first speculated on in 1896.
Modern climate science considers many more factors and mechanisms than this kind of extremely simplified model, but I am not familiar with them. Deniers / sceptics frequently seize on potential first order approximation mechanisms to try to reject the climate change hypotheses. Sceptical Science is a great source for intensely detailed analysis of common misconceptions.
* The reason that Earth radiates infrared, where as the sun's radiation has a significant visible component, is that while both can be thought of as black body radiators, Earth is very much cooler than the sun, so has a low frequency dominated spectrum.