I'm going to add some mathematical detail to what akhmeteli has said.
Let's restrict the discussion to one dimension with coordinate $x$, then Fourier's law in differential form says
$$
q(x) = -k(x) T'(x)
$$
where $q(x)$ is the local heat flux, $k(x)$ is the conductivity, and $T(x)$ is the temperature gradient. Notice that Fourier's law show that that a given point, the derivative of the temperature is important, but derivatives of a function depend on the value of that function in a neighborhood of that point, not just the value of the function at that particular point. Therefore, the (not entirely explicit) answer to your question is that you need both $k$'s at a point where two materials with different $k$ are in contact. Now let's see the math.
If you are looking at a point $x_0$ at which two materials with different conductivities are joined, (say $k_a$ corresponds to $x<x_0$ and $k_b$ corresponds to $x>x_0$, then $k(x)$ has a jump discontinuity that can be written with the use of the Heaviside step function $\theta(x)$;
$$
k(x) = (k_b-k_a)\theta(x-x_0) + k_a
$$
Which results in the following differential equation:
$$
q(x) = -[(k_b-k_a)\theta(x-x_0) + k_a] T'(x)
$$
Which you can attempt to solve in a given case. For example, let's consider a steady-state system in which $q(x) = q_0$ is a constant and for which we want to determine the temperature gradient. Let's suppose that this system consists of metal bars joined at the point $x_0$ and whose endpoints are located at $x_0-L$ and $x_0+L$ respectively. Additionally, we assume that these other two endpoints are kept at a temperature $T_0$ In this case, the differential equation we would want to solve for $T(x)$ is
$$
q_0 = -[(k_b-k_a)\theta(x-x_0) + k_a] T'(x)
$$
with the boundary data
$$
T(x_0-L) = T_0,\qquad T(x_0+L) = T_0
$$
The differential equation we want to solve can be rewritten as a set of two equations, one for $x<x_0$ and another for $x>x_0$;
$$
q_0 = -k_a T_a'(x), \qquad q_0 = -k_b T_b'(x)
$$
The general solutions are
$$
T_a(x) = T_0-\frac{q_0}{k_a} [x-(x_0-L)], \qquad T_b(x) = T_0-\frac{q_0}{k_b} [x-(x_0+L)]
$$
and the temperature everywhere except at $x=x_0$ can be written as
$$
T(x) = (T_b(x) - T_a(x))\theta(x-x_0) + T_a(x)
$$
In particular, notice that there is a jump discontinuity in the temperature at $x=x_0$;
$$
T_b(x_0) -T_a(x_0) = q_0 L\left(\frac{1}{k_b}-\frac{1}{k_a}\right)
$$
and this discontinuity depends on both $k$ values, not just the value on a particular side. Note further that if $k_a = k_b$, the the discontinuity disappears as you might intuitively expect!
Hope that helps! Let me know of any typos.
Cheers!