Computing as applied semiotics
Actually the point of computing in general is that the symbols are abstract and intrinsically meaningless and devoid of physics; this lets us fill in any physical realization of those symbols that we want: as long as we can connect "this symbol over here is represented by this set of those other symbols over there" appropriately, we can realize these symbols (which may be 0 or 1 but may be whatever we want as long as there's more than one of 'em) in any medium. So in your computer, bits are sometimes stored as voltages, sometimes as whether current will pass through a junction or not, sometimes as magnetic spins pointed in various directions on spinning metal platters, sometimes as notches in a spiral groove on a disk that need to be read out with a laser, sometimes as a partial regulation of how much light is coming out of a single color from a single pixel on your LCD (or if you're reading this with e-paper, whether the black or white side of a little ball that's painted half-black, half-white is facing the outside of the display). This fundamental ambiguity about what the "symbols" really map to in the world allows us to use anything as long as we can connect the symbols over here to the symbols over there by some concrete rule.
That is why you're seeing such ambiguous answers in the context of quantum computing; it's because we intentionally defined "quantum" computing in a similarly abstract way and then we are allowed to implement it however.
A simple model system
Now just like you have "voltage" bits that you understand very well, let's take one particular quantum system to learn what qubits work like. And that is the spin-$\frac12$ system.
A particle with spin-$\frac12$, like an electron, has some intrinsic angular momentum which is deeply, unfailingly, quantum. On the one hand it is always "on average" spinning in a certain direction; on the other hand it is never "purely" spinning in any direction. This is because the spin measured along any given axis must take the form $\pm \frac 12 \hbar$ but the total angular momentum squared must take the form $L^2 = \hbar^2 \ell (\ell + 1)$ where $\ell=\frac12$ is what makes it a spin-$\frac12$ particle in the first place. So in some sense when we say that the electron is spinning in one particular direction, "up" for instance, we're really talking about the case where we've measured its spin along that axis as $+\hbar/2$, but we know that since it has all of this extra total-angular-momentum it has a distribution of spin in the "left" and "right" directions and "back" and "forward" directions that have been forced, essentially by the uncertainty principle, to be deeply uncertain, precisely because we're certain about the way that it's spinning.
In fact we know that the mathematics for the spin-$\frac12$ particle has these three matrices called the "Pauli matrices", $$\sigma_x = \begin{bmatrix}0&1\\1&0\end{bmatrix},~~~\sigma_y = \begin{bmatrix}0&-i\\i&0\end{bmatrix},~~~\sigma_z = \begin{bmatrix}1&0\\0&-1\end{bmatrix},$$ and these two complex numbers $[\psi_0, \psi_1],$ and you can find the average value of any measurement by finding the corresponding matrix $\hat M$ to that measurement and computing:$$\langle M \rangle = \begin{bmatrix} \psi_0^* & \psi_1^* \end{bmatrix} \hat M \begin{bmatrix} \psi_0 \\ \psi_1 \end{bmatrix}. $$
The identity matrix $I = \begin{bmatrix}1&0\\0&1\end{bmatrix}$ corresponds to measuring that the constant number 1 is still 1, which means that we must ensure that $|\psi_0|^2 + |\psi_1|^2 = 1.$ The Pauli matrices represent measuring the spin along the $x$, $y$ or $z$ axes, getting either $+1$ (spin angular momentum $+\hbar/2$) or $-1$ (spin angular momentum $-\hbar/2$) along that axis.
As you can see, the vector $|0\rangle = \begin{bmatrix}1\\0\end{bmatrix}$ where $\psi_0=1$ corresponds cleanly to a qubit-zero as well as to "spin-up" if the z-direction points upward. Similarly the vector $|1\rangle = \begin{bmatrix}0\\1\end{bmatrix}$ corresponds cleanly to a qubit-one as well as to "spin down". These are eigenvectors of the $\sigma_z$ operator, and they have no average spin in either the $x$ or $y$ directions, but again, remember that they do have components in those directions, as you can see by looking at $\sigma_x^2 = I$ for example.
The Bloch sphere and Hamiltonian evolution
In fact there is a famous "Bloch sphere" representation of a single qubit which represents the qubit in terms of three angles $\Phi, \theta, \varphi$ as: $$\psi_0 |0\rangle + \psi_1 |1\rangle = e^{i\Phi}\left( \cos\Big(\frac\theta2\Big) |0\rangle + \sin\Big(\frac\theta2\Big)e^{i\varphi} |1\rangle \right),$$ then notices that $\Phi$ has no effect on any expectation value above and can be discarded: the claim is that $(\theta, \varphi)$ are spherical coordinate angles locating an axis that the particle "most spins along." So that's the interpretation of the qubit as a point on a sphere, with $|0\rangle$ at the north pole and $|1\rangle$ at the south pole. (More on this: Wikipedia, A previous answer I wrote.)
Now the most simple single qubit operation would be to fire a charged spin-$\frac12$ particle through a magnetic field pointed in the $z$-direction. The Hamiltonian for this is going to look like $\epsilon ~ \sigma_z$ for some energy $\epsilon$ and the state always evolves like $\exp(-i \hat H t/\hbar)$ for a given Hamiltonian $\hat H$ acting on a time t, therefore defining $\alpha = \epsilon t / \hbar$ if this magnetic field applies to the particle for a time $t$ then it causes $$|0\rangle \mapsto e^{-i\alpha} |0\rangle, \\ |1\rangle \mapsto e^{+i\alpha} |1\rangle.$$Or, looking at the Bloch sphere we see that this maps $\varphi \mapsto \varphi + 2\alpha,$ with one of the $\alpha$ coming from the shift in $\Phi.$ You might prefer to simply do this dividing-out-$\Phi$ to the gates as well, in which case $|0\rangle \mapsto |0\rangle, |1\rangle \mapsto e^{i\alpha} |1\rangle$ is the "phase rotation gate" by $\alpha$, and again, it's just zapping the electron with a magnetic field so that its spin precesses about the $z$-axis.
We might similarly rotate it about the X or Y axes; in particular notice that $\sigma_x$ is basically the classical "NOT" gate mapping $|0\rangle \mapsto |1\rangle$ and $|1\rangle \mapsto |0\rangle.$
Now suppose that you have two such charged spin-1/2 particles, so two qubits, and it needs to be written as $\psi_{00} |00\rangle + \psi_{01} |01\rangle + \psi_{10} |10\rangle + \psi_{11} |11\rangle.$ You can either separate them and do these magnetic rotations to each in particular, e.g. doing the above rotation by $\alpha$ about $z$ to the second qubit effectively maps $|00\rangle \mapsto |00\rangle,$ $|01\rangle \mapsto e^{+i\alpha} |01\rangle,$ $|10\rangle \mapsto |10\rangle,$ and $|11\rangle \mapsto e^{+i\alpha} |11\rangle.$ Or else you can bring them near each other. If you bring them near each other then maybe they prefer to have opposite spins rather than parallel spins or vice versa, this gives you some ability to "entangle" their spin-states with a Hamiltonian $\epsilon \Big(|01\rangle\langle01| + |10\rangle\langle10|\Big),$ causing
$$|00\rangle \mapsto |00\rangle \\
|01\rangle \mapsto e^{i\beta} |01\rangle \\
|10\rangle \mapsto e^{i\beta} |10\rangle \\
|11\rangle \mapsto |11\rangle
$$
Now if we combine these, first off doing this operation, then rotating both qubits along the z axis with $\alpha = -\beta$, we find that $|00\rangle \mapsto |00\rangle$ and likewise for $|01\rangle$ and $|10\rangle$, but $|11\rangle \mapsto e^{-2i\beta} |11\rangle,$ giving us an implementation of the "controlled phase rotation" gate which is a little more well-known. It turns out that you now can express an arbitrary quantum computation in terms of these flying spins: a composition of gently bringing them together pairwise to entangle/disentangle, and rotating them by precession about various arbitrary axes, can be made to simulate any quantum-computing algorithm.