If we want to know what the solution is, we first have to figure out what the problem is.
Say we have a bucket of water in the middle of space, with nothing else in the universe. If the bucket isn't spinning, the water will rest in the bucket. If the bucket is spinning, the water will be pushed up against the edge.
Here is the problem: who is to say whether the bucket is spinning or not? It seems as though if there is absolutely nothing else in the universe, the two situations are completely the same. That is the "problem."
But in a certain sense, there is no problem. There are a certain set of special frames in the universe called "inertial frames" in which there are no fictitious forces. If we look at the bucket when we are in an inertial frame, and it spinning, we can deduce that the water should be pushed to the sides. If the bucket is not spinning when we are in an inertial frame, then we can deduce that the water should not be pushed to the sides.
No single inertial reference frame is privileged. If we move at a constant velocity with respect to an inertial frame, we are ourselves also at the center of some other inertial frame. However, if we rotate a frame by a time dependent angle, we will no longer be in an inertial reference frame, and it will take fictitious forces to explain the phenomena we see.
This is the "solution" to the "problem," but perhaps it isn't satisfying. You might object that there is nothing physically different happening if the bucket is spinning or not. I would counter, "who are you to demand what is physical or not?"
Einstein was bothered by the bucket problem. He felt as though a theory of space, time, and gravity would solve it. Somehow, he imagined, perhaps a set of stationary stars define what an "inertial" frame is, but if there were no stars very far away, the water would never be pushed to the sides.
While this was one of his motivations for developing general relativity, the actual theory he came up with doesn't actually work the way he hoped. The problem is that the distant stars really don't have an effect on the bucket. They're irrelevant to the problem at hand.
However, Einstein's formulation of general relativity does "explain" the bucket problem in its own way.
Say we take space to be flat, described by the regular Minkowski metric in an inertial frame, and a non-spinning bucket is sitting there in space. We can change coordinates to a rotating frame of reference. In this new frame of reference, with the new coordinates we use, the Minkowski metric will look different. Obviously, all we did is choose different coordinates, but if we ourselves are now rotating with respect to the original inertial frame, these are the coordinates we would use.
With our new metric, things will move on geodesics/straight lines if they are not acted on by forces. (This includes the water particles in the bucket, which would leave the bucket if the bucket's walls were not forcing them to stay inside.) However, these "straight lines" would not look straight to us when we are spinning. So the water particles, which are sitting still in the inertial frame, are now moving in circles.
I should note that this isn't really that different from the Newtonian explanation. I've just changed "moving without fictitious forces" with "moving in a straight line/geodesic." But that is what general relativity has to say about it.
Now that I've explained that GR doesn't really have anything new to say about the bucket problem, I should mention that there's actually more to the story.
It's not enough to have a few "distant stars" out at infinity, but say instead you have a lot of distant stars evenly distributed throughout space. Furthermore, say that the entire mass of stars are orbiting around some central point for no reason, all with the exact same angular velocity. (Who knows why they're doing that, just imagine that they are.) If we then use Einstein's equations to solve for the spacetime metric, we find that there actually is a centrifugal force that would act on a bucket placed in the middle of the space time! In other words, if we placed a non-spinning bucket of water in the middle of this spinning universe, the water actually would be forced to the edge! This is called the "Lens Thirring" effect.
Aha! It seems as though general relativity then DOES solve the Newton bucket problem! Well... not quite. This centrifugal force depends on the mass density of the distant, evenly distributed stars. From a Newtonian analysis, we know that the centrifugal force on the water from changing frames should have absolutely nothing to do with the density of distant stars. Furthermore, the centrifugal force produced by the Lens Thirring effect will always be less than the centrifugal force you would normally expect from changing reference frames.
There is some confusion about the Lens Thirring effect. I have seen some physicists claim that it solves the Newton bucket problem. While it might seem like it does at first glance, it actually doesn't. There is a fundamental difference between the universe where all the distant stars are stationary as described by a rotating coordinate system and the universe where all the distant stars are somehow rotating together in an inertial frame.
