Since the four forces are different, with different force carriers, how are they (seemingly) directly compared? I often read that the weak force, for example, is many orders of magnitude stronger than gravity, and that electromagnetism is several orders of magnitude stronger than the weak.....
I do notice that different sources give different ratios when comparing the four forces' relative strengths, though....
It is the history of how the SU(3)xSU(2)xU(1) standard model of physics was built up making a consistent whole of the grand majority of data up to now, and giving good predictions.
A whole lot of scattering experiments showed that there were different strengths and different lifetimes in the interactions, and these are described with quantum field theory, where each interaction and decay is a sum of Feynman diagrams, starting from the first order,which gives the gross behavior, and going as far as calculations can be made or are needed of higher orders.
Feynman diagrams have vertices, and these vertices in the integrals represented by the Feynman diagrams have different strengths, coupling constants, which are shown in the table here.
The column "strength" is the value used in Feynman diagram vertices.
The first two, electromagnetic and weak, are the ones working well with the series expansion of the interaction into feynman diagrams. The lowest order are the exchanges of the gauge particles, that is why they are identified with the force (first column). QCD the strong force, can only be figuratively shown in Feynman diagrams. The actual calculations need different tools, because the strong coupling constant is 1. Lattice QCD is used for that. Gravity is not yet definitively quantized but there are effective quantizations.
In electromagnetic and gravitational interactions there exist mathematical tools that connect the classical constants with the field theoretical ones. Strong and weak interactions belong only to the study of particle interactions.
So , in a nutshell, it is the fitting of data with a specific standard model that organizes the particle interactions in line with four forces. For example see this summary..
In particle physics, the strength of fundamental forces is specified by either masses of force-carrier particles (e.g. masses of $Z$, $W^{\pm}$ bosons for weak interaction) and/or by a numerical "coupling" coefficient (e.g. electron charge in electromagnetic interaction).
One of the ways to measure the relative strength is by comparing the ratios of particle productions. If during a collision of two hydrogen atoms $K$ particles are produced through the electromagnetic interaction, $L$ particles through the strong interaction, and $M$ through the weak - $K : L : M$ ratio could be used to quantatively compare their strength. This is generally going to be dependent on various factors like angles of detectors, center-of-mass energy of collision, but can be nevertheless used make qualitative statements about the strengths of the forces.
