Whilst I completely agree with the accepted answer, I feel it hasn't quite addressed OP's final question: "if at the start the two coordinate systems are shifted in space (the accelerating one to the right), then it would look like the accelerating observer should see the inertial clock running backwards in time. What am I imagining wrong?". I believe the answer to that question is that you are not imagining anything wrong apart from "should see". If you substitute "seeing" for "simultaneous with", you have in fact correctly understood special relativity. You are correct that before accelerating the two twins are in the same frame (time is t for both). You are also correct that once accelerated, the accelerating twin is then simultaneous with the past of the stationary twin (time is now earlier than t). However the accelerating twin cannot see this - if he had a telescope he wouldn't see the other twin's clock run backwards *.
Here is a Minkowski diagram of what you have asked. As you have described one twin as accelerating, I have started the diagram prior to the acceleration, so that it can be seen, and also finished the diagram after arrival so the the deceleration at the end can also be seen. The green lines show simultaneity when both twins are in the same frame (the rest frame). The blue and red lines are the lines of simultaneity for the outbound and return legs of the journey respectively. The grey dashed line shows the point of origin of the travelling twin.

Remember, t' depends not only on t but also x.
\begin{align}
t' &= \gamma \ (t - vx/c^2)
\end{align}
In the classic twins diagram, x is zero so t and t' are both the same (also zero) when the accelerating twin departs. Because they are not separated in space, they are not separated in time either.
However, if the accelerating observer is somewhere to the right, then they are separated in space, x is not zero, and so when the accelerating observer accelerates and v goes from 0 to 0.5c (or whatever non-zero speed), the separation in space also causes a separation in time. Actually, even in the standard twins diagram, we see a similar thing happening (but in reverse) when the accelerating twin turns around at the planet: because they are separated in space (non zero x), the change in v causes a change in t'.
In summary, well thought out, that is exactly what the Minkowski diagram shows you in that case, and is of course correct (albeit counter-intuitive and mind blowing, but then much of relativity is)!
*As the accepted answer says, the accelerating twin will not see the clock of the stationary twin move backwards in time. Simultaneity is by definition faster than light, so you can never see anything that is simultaneous with you. You can only see light, and light takes time to travel, so you are always seeing the past. Although the simultaneity changes due to the acceleration, that doesn't change the light arriving at your position from the other position. Like the accepted answer says, it just slows it down. In an updated diagram below I've added light beams in yellow, and shown time from a little earlier. You can see that the accelerating twin is 2 light units away (in the typical twin diagram that's 2 light years of course), and that's how far behind the accelerating twin will see the stationary twin's clock, even when they are both in the same reference frame... and at the point when the accelerating twin accelerates, although simultaneity has changed, they will still be looking at their twin's clock two year's out of date:
