Well to address your actual question the answer would be ~yes. Since its a scalar and both $a^{\mu}$ and $b^{\mu}$ are 4-vectors $\Gamma_{\mu \nu}$ must be a 2nd rank tensor. In the context of Special Relativity the underlying space is Minkowski space and thats what they mean by 4-tensor (This was already pointed out in the comments by Secavara) and these can be REPRESENTED by a 4x4 matrix.
It might seem a little abstract but that exact configuration is very commonly utilized in SR. For example when you define the inner product which grabs two 4-vectors and gives you a scalar you commonly do this $a \cdot b = a^{\mu} b_{\mu} = a^{\mu} g_{\mu \nu} b^{\nu}$ where $g_{\mu \nu}$ is the Minkowski metric (4x4 representation). You can see that the last term is a special case of the object you have on your question.
As to how you can see that $\Gamma_{\mu \nu}$ is that kind of tensor you can perform a general coordinate transformation. Since a scalar should be invariant under such transformation you can check the only way that it can be satisfied is if Gamma is a 4-tensor.