I'm answering my own question to jot some thoughts of my own down. I'm not completely sure I have the right understanding, so the info here shouldn't be taken authoritatively. @Dale is largely right in saying,
This is not actually a physics question, but a question of semantics. What do people mean when they use the word "absolute" in this context?
In physics, when one says "$x$ is absolute" what is meant is "quantity $x$ is invariant under such-and-such class of transformations." I'm trying to be as general as possible so I'm not necessarily limiting myself to coordinate transformations (for example we can consider gauge transformations as well), but coordinates is what I mainly have in mind.
The key here is that generally speaking we want to restrict ourselves to transformations that keep the form of the laws/formulas of physics the same. Even if we find a transformation that changes quantity $x$, that's not enough to conclude that $x$ is relative (because those transformations that change $x$ can be accompanied by changes in the form of the laws/formulas of physics).
Short Summary
To those who will find the text below too much to read, I'll just put my conclusions here. To say it as briefly as possible,
- The spacetime of Galilean relativity alone does not discriminate between inertial and non-inertial reference frames. You need to go to the level of Newtonian mechanics and appeal to Newton's third law: Inertial reference frames are the unique frames in which Newton's third law holds.
- The spacetime of special relativity does allow you to discriminate between inertial and non-inertial reference frames.
In classical mechanics, relativity of acceleration breaks at the level of mechanics. In special relativity, relativity of acceleration breaks at the level of spacetime.
Newtonian Mechanics / Galilean Relativity
As far as I can tell, there is nothing in the structure of Galilean spacetime that rules out the notion that "acceleration is relative." You can in principle formulate a theory where acceleration is relative that exhibits Galilean invariance with no trouble (this is in contrast to special relativity where the spacetime structure does prevent acceleration from being relative). However, it turns out that Newtonian mechanics does not allow acceleration to be relative for different reasons.
In Newtonian mechanics, if you try to frame acceleration as relative, then you run into a contradiction with Newton's three laws (which one exactly depends on how you want to describe and phrase your model of the universe). In particular, my thought experiment where the entire universe with every particle is accelerating in one direction violates Newton's first law. If I tried to salvage my thought experiment by saying there is some force causing this uniform acceleration, then this violates Newton's third law. Daniel Kleppner says something to this effect in An Introduction to Mechanics in Section 2.6 when he writes,
Newton's third law is essential if the second law is to be meaningful: without it, there would be no way to know whether an acceleration results from a real force, or is merely an artifact of being in a non-inertial system. If the acceleration is due to a force, then somewhere in the universe there must be an equal and opposite force acting on some other body.
Really, this is the key to distinguishing inertial frames from uniformly accelerated frames. Newton's first law postulates the existence of an inertial frame, then Newton's second law describes the relationship between force and acceleration that ought to hold, and Newton's third law places a constraint on what kinds of "real forces" there can be. This in turn determines what is the class of reference frame transformations that can affect the three laws.
If the class of transformations we are considering is the class of Galilean boosts $\vec{x}\,' = \vec{x} - \vec{V}t$, then acceleration is invariant. If the class of transformations is the class of changes to linearly accelerated reference frames $\vec{x}\,' = \vec{x} - \frac{1}{2}\vec{A}t^{2}$, then acceleration is not invariant. We want to restrict ourselves to transformations that keep the form of the laws/formulas of physics the same. The transformations in the former class preserve Newton's three laws whereas the transformations in the latter class do not, because in an accelerating reference frame we either have to modify Newton's first law (as particles would accelerate without any force acting upon them) or we have to modify Newton's third law (as we would have a fictitious force that is unpaired with any other equal and opposite force). In this precise sense, it does seem that "acceleration is absolute."
Special Relativity
In special relativity, we find that acceleration fails to be relative due to the structure of spacetime (which is surprisingly different than what we found in Galilean relativity). The fact is that particles that are undergoing constant proper acceleration attain a Rindler horizon from the point of view of Rindler coordinates, so for each particle there will be a region of Minkowski space that will be rendered inaccessible to the particle as long as it is in acceleration. This is enough to distinguish whether every particle in the universe is being uniformly accelerated (like in the thought experiment) or not.
Consider an inertial reference frame with coordinates $(x^{0}, x^{1})$. If we had two objects that are at rest but at different positions, then their worldlines would be straight vertical lines in a Minkowski diagram for $x^{\mu}$-coordinates, and the particles are able to send light signals to each other without any problems. If, however, the two objects were undergoing constant proper acceleration $A\ne 0$, their worldlines would be hyperbolas in $x^{\mu}$-coordinates. If the two hyperbolas don't intersect each other, then past a certain time there will be a point where one object is no longer able to send messages to the other even though it is able to receive messages from the other. This is enough to tell whether the objects are accelerating or not.
Indeed, even though coordinate acceleration is not invariant under general spacetime coordinate transformations (apart from e.g. Lorentz transformations and translations), in special relativity you can define proper acceleration, which turns out to be invariant under all possible coordinate transformations.
Also, you don't even need to appeal to Rindler horizons. If two comoving observers are uniformly accelerating, they would observe Doppler redshifting/blueshifting when they send light to each other as they accelerate. So you can't use light to tell "absolute velocity" but you can use it to tell "absolute acceleration," so to speak.
No Global Accelerating Reference Frame in SR?
Lastly, I want to point out something else that is interesting with special relativity. As far as I understand, there is no notion of an accelerated reference frame at least in the following sense:
Given an inertial reference frame $(x^{0}, x^{1})$ there is no coordinate transformation $(x^{0}, x^{1})\rightarrow(\xi^{0}, \xi^{1})$ such that the following conditions hold:
- The coordinate transformation is global, i.e. every point in Minkowski space is assigned a unique coordinate $(\xi^{0}, \xi^{1})$.
- (a) It sends worldlines of the form $x^{\mu}(t) = (t, vt+x_{0}^{1})$ with $v\in(-1, 1)$ to worldlines of the form
$$ \xi^{\mu}(t) = (\tfrac{1}{A}\sinh\lambda(t, v, x_{0}^{1}) - \xi_{0}^{0}, \tfrac{1}{A}\cosh\lambda(t, v, x_{0}^{1}) - \xi_{0}^{1}), $$
i.e. it sends constant velocity worldlines to hyperbolic motion worldlines. (b) Also, $\lambda(t, v, x_{0}^{1})$ is strictly increasing with respect to $t$ if $v$ and $x_{0}^{1}$ are kept constant (this is just to fix an orientation of our $\xi^{\mu}$-coordinates relative to the $x^{\mu}$-coordinates).
These two conditions are already contradictory.
Proof. Assume $(x^{0}, x^{1})\rightarrow (\xi^{0}, \xi^{1})$ satisfies these two conditions $(1)$ and $(2)$. Without loss of generality, we may assume it sends $(0, 0)\rightarrow (0, 0)$. We can use condition $(2)$ to deduce the form of the transformation on the quadrant $Q = \{(x^{0}, x^{1}) \,|\, 0 < x^{0},\, |x^{1}|<|x^{0}|\}$. Given a worldline $x^{\mu}(\tau) = (\tau\cosh\phi, \tau\sinh\phi)$, this ought to transform to
$$ \xi^{\mu}(\lambda) = (\tfrac{1}{A}\sinh\lambda - \xi_{0}^{0}, \tfrac{1}{A}\cosh\lambda - \xi_{0}^{1}). $$
By using the fact that $(0, 0)\rightarrow(0, 0)$, rescaling the worldline parameter to normalize the derivative, shifting the parameter, and taking condition $(\text{2b})$ into account, our worldline ends up in the form
$$ \xi^{\mu}(\tau) = (\tfrac{1}{A}\sinh(A\tau+\lambda_{0}(\phi)) - \tfrac{1}{A}\sinh\lambda_{0}(\phi), \tfrac{1}{A}\cosh(A\tau+\lambda_{0}(\phi)) - \tfrac{1}{A}\cosh\lambda_{0}(\phi)) $$
for some strictly increasing or decreasing function $\lambda_{0} = \lambda_{0}(\phi)$.
Using this info, we know that our transformation (if it exists) maps points $(x^{0}, x^{1})\in Q$ to $(\xi^{0}, \xi^{1})$ by
\begin{align*}
\xi^{0} &= \frac{1}{A}\left[ \sinh\left(A\sqrt{(x^{0})^{2} - (x^{1})^{2}} + \lambda_{0}\left(\operatorname{arctanh}\frac{x^{1}}{x^{0}}\right)\right) - \sinh\left(\lambda_{0}\left(\operatorname{arctanh}\frac{x^{1}}{x^{0}}\right)\right) \right], \\
\xi^{1} &= \frac{1}{A}\left[ \cosh\left(A\sqrt{(x^{0})^{2} - (x^{1})^{2}} + \lambda_{0}\left(\operatorname{arctanh}\frac{x^{1}}{x^{0}}\right)\right) - \cosh\left(\lambda_{0}\left(\operatorname{arctanh}\frac{x^{1}}{x^{0}}\right)\right) \right].
\end{align*}
Consider the worldline $x^{\mu}(t) = (t, b)\in Q$ for $t\in(|b|, \infty)$ where $b$ is a constant. Plugging this into what we have so far gives
\begin{align}\tag{$*$}
\xi^{0}(t) &= \frac{1}{A}\left[ \sinh\left(A\sqrt{t^{2} - b^{2}} + \lambda_{0}\left(\operatorname{arctanh}\frac{b}{t}\right)\right) - \sinh\left(\lambda_{0}\left(\operatorname{arctanh}\frac{b}{t}\right)\right) \right], \\
\xi^{1}(t) &= \frac{1}{A}\left[ \cosh\left(A\sqrt{t^{2} - b^{2}} + \lambda_{0}\left(\operatorname{arctanh}\frac{b}{t}\right)\right) - \cosh\left(\lambda_{0}\left(\operatorname{arctanh}\frac{b}{t}\right)\right) \right].
\end{align}
By condition $(2)$, this worldline $x^{\mu}(t)$ must be sent to (a portion of) a hyperbolic path, so
\begin{align}\tag{$\dagger$}
\xi^{0}(t) &= \frac{1}{A}\sinh\lambda(t) - \xi_{0}^{0}, \\
\xi^{1}(t) &= \frac{1}{A}\cosh\lambda(t) - \xi_{0}^{1}.
\end{align}
We set the corresponding expressions to be equal. Then we consider the expression $(A\xi^{1}+A\xi_{0}^{1})^{2}-(A\xi^{0}+A\xi_{0}^{0})^{2}$. By $(\dagger)$ and the $\cosh^{2}-\sinh^{2}=1$ identity, we conclude that our expression is constant. However, using the expressions in $(*)$, if $b\ne 0$ one can show that our expression is not constant for any choice of $\xi_{0}^{0}, \xi_{0}^{1}$. This is a contradiction and so no coordinate transformation can satisfy conditions $(1)$ and $(2)$ simultaneously. $\blacksquare$
If we can't have a global uniformly accelerating reference frame in SR, then we can't say there is any symmetry between inertial frames and accelerating frames to begin with. This is somewhat surprising, because this doesn't happen in Newtonian mechanics / Galilean relativity. Rindler coordinates are often invoked to stand in for what we mean by an accelerated reference frame in SR, but they have two undesirable properties for the purposes of my question. First, they only cover a quadrant of Minkowski space, so they fail condition $(1)$. Second, each hyperbolic worldline that they describe has a different proper acceleration, so they fail condition $(2)$.
"Of we imagine the entire universe with every particle accelerating in one direction, then the accelerometer will not be able to distinguish this from a universe that is not accelerating. " This is a (famous) speculation that can't be checked.
– Ján Lalinský Jan 17 '23 at 13:05"an inertial observer will also observe a changing Doppler shift in the light from an accelerating source. You won't be able to determine whether you are in the non-inertial frame or the source is." Yes, for one source and one observer it is hard to decide between the two. But we have billions of sources (distant stars), and rate of change of their Doppler shift in any frame can be tracked as function of spherical angles. When this function in observer's frame looks like "clear dipole pattern" , it's clear the observer is accelerating.
– Ján Lalinský Jan 18 '23 at 15:39