Your remark is true, but not very relevant in practice: according to Birkhoff's Theorem (see Misner, Thorne, Wheeler: "Gravitation," 32.2, pages 843-844) the metric outside any spherically symmetric object is the Scwarzschild metric. So a test particle outside that object's mass only sees a Schwarzschild geometry.
Now, such an object has to collapse to a black hole (for general-relativistic reason) if its radius falls below a minimum possible radius $r_{min}:$
$$
r_{min} = \frac {9}{4} \cdot \frac {GM}{c^2} = \frac {9}{8} r_g.
$$
(cfr. Misner, Thorne, Wheeler: "Gravitation," 23.7, page 611). A collapsing star reaches $\ r_{min}\ $ in a finite amount of time: as for the standard expression of the Schwarzschild metric
$$
ds^2 = - \Big( 1 - \frac {2GM}{c^2r}\Big) dt^2 + \dots
$$
it shouldn't take longer than trice the proper collapse time.
Now, as I said before, all the "black hole" dynamics, i.e. particle trajectories (a full discussion of this point is in Misner, Thorne, Wheeler: "Gravitation," 25.3, page 655 ) are present outside the collapsing star, and our telescopes see (and probe) that part of spacetime.
Furthermore, as explained in MTW, chapter 32, electromagnetic signals from the collapsing region die off exponentially with characteristic time $t = 10^{-5}$sec per solar mass.
Quite a black region of space. Now, one could argue: maybe we think we see a black hole, but that's really a very compact neutron star, something like $\ r = (1.15-1.20) r_g. \ $ That doesn't really fly: the neutron stars we know are all much larger than that, around twice the Schwartzschild radius, and their mass is limited to around two solar masses (see https://arxiv.org/abs/1603.02698 or https://arxiv.org/abs/1205.6871 or https://www.slac.stanford.edu/econf/C010815/docs/heiselberg.pdf). Maybe there are quark stars or weak-force stars? That would work only with low-mass black holes. Cygnus X-1 is around 15 solar masses, and we have seen matter falling inside it without hitting anything. (See wikipedia for this).
So, point is: it is true that at spatial infinity the collapse of a star never ends, but in a finite amount of time the object becomes indistinguishable from a genuine black hole.
