Can we say this mathematically and very generally? This would first
require a general definition of what a "physical universe" is, and
then what a "portion of it" would be
I would say the Wikipedia description of a system is fairly general. But with the possible exception of cosmological thermodynamics, it is not necessary to define a "physical universe". In thermodynamics the scope of what is defined to be the system is generally limited and anything not included in the system definition is considered to be part of the systems environment, or surroundings. Then the combination of the defined system and its surroundings is considered the "physical universe".
As a practical matter, in general the only part of the system's surroundings that usually enters into the analysis of the system is that portion of the surroundings in proximity to the boundary of the defined system. That is, the portion of the surroundings that actually exchanges mass and/or energy with the system. In this case, the rest of the "physical universe" becomes irrelevant so that it need not be defined.
It would have to track the state of the "portion" over time, dependent
on the universe state, and also track the effects of the environment
on that portion.
Again, it is only necessary to "track" that portion of the surroundings that actually exchanges mass and/or energy across the boundary between the system and the surroundings. The state of the rest of the universe not in proximity to the boundary need not be defined or tracked.
The manner of mathematically tracking the exchange of mass and/or energy between the system and its immediate surroundings is by means of the first law of thermodynamics. (The direction in which the exchanges can take place is the purview the second law). The more limited version of the first law is that for a closed system, i.e., a system that can only exchange energy with the surroundings and does not exchange mass with the surroundings. The more extensive version is for an open system since the exchange of both energy and mass needs to be tracked.
Hope this helps.