Taking
$$\beta = \frac{1}{k_B T}$$
is a standard notation (it is just redefining the temperature parameter). This is to say that it is a definition of $\beta$ rather than a result from the third law of the thermodynamics. The answer by @Pieter however makes a good point that one could equally define coldness and derive this relation.
Probability of a particle occupying energy state $i$ is the ratio of the number of particles in this state to the total number of particles, when the letter goes to infinity, i.e.
$$p_i = \lim_{N\rightarrow +\infty}\frac{n_i}{N}.$$
This is actually not a result, but the definition of probability (in the frequentist sense, which is what almost exclusively used in physics).
Update
To address the comments by @NakshatraGangopadhay :
It is important that the probability is defined as a limit for an infinitely big system: both $n_i$ and $N$ become infinitely large, but their ratio remains constant.
If we take a finite system of $N$ particles with the probability $p_i$ of finding each particle in state $i$, the probability of finding $n_i$ particles in this state is given by the binomial distribution (ignoring indistinguishability for simplicity):
$$P(n_i)={N\choose n_i}p_i^{n_i}(1-p_i)^{N-n_i},$$
so that the average number of particles in the state is
$$\langle n_i\rangle = p_i N.$$
We could also calculate the variance as
$$\sigma_{n_i}^2 = Np_i(1-p_i).$$
The relative error in the number of particles is then
$$\frac{\sigma_{n_i}}{n_i}\propto \frac{1}{\sqrt{N}}.$$
In the limit $N\rightarrow +\infty$ we have
$$
\lim_{N\rightarrow +\infty}\frac{n_i}{N}=p_i,\lim_{N\rightarrow +\infty} \frac{\sigma_{n_i}}{n_i}\rightarrow 0.$$
That is, the fluctuations of the particle number around its average become negligible. This thermodynamic limit is crucial for the precise nature of the thermodynamic laws - although we never truly deal with an infinite system of particles, the errors are negligible when applying the laws of thermodynamics to systems with a number of particles of the order of the Avogadro number, $N_A\approx 10^{23}$.
This reasoning very much underlines the Einstein's claim about thermodynamics being beyond doubt.