Disentangling Sampling from Loss Weighting in Physics-Informed Neural Networks

Abstract

Interventions on the collocation set enlarging it, or drawing it from a residual-based adaptive distribution are among the most widely adopted means of improving physics-informed neural networks, and the resulting gains are routinely attributed to where the points are placed. We observe that such interventions cannot change the sampling geometry alone: because the residual loss is an empirical mean, the magnitude of its stochastic gradient carries an inflation term tr(Σr)/Nr, so any change to the number or the distribution of collocation points silently rescales the residual term relative to the boundary and initial-condition terms. Reported gains therefore confound two mechanisms that the literature treats as distinct: sampling and loss weighting. We measure the confound directly and find it to be real but moderate, shifting the interior-to-constraint gradient balance by 20 to 40 per cent when the interior batch is enlarged sixteenfold. We then separate the two mechanisms with a matched-control design in which a control arm reproduces, to machine precision, the effective-balance trajectory of an intervention while leaving the collocation set untouched. On the one benchmark where enlarging the collocation set clearly helps viscous Burgers, where it reduces the error five fold the control recovers none of the gain, giving an attribution ratio of A = −0.10 with a bootstrap interval of [−0.90,0.22] that excludes the reweighting explanation. Matching the balance through the weights alone is in fact harmful: across problems the control arms are 1.1 to 1.7 times worse than the baseline they start from. Sampling gains in this setting are therefore geometric, and an alternative explanation that had never been excluded can now be set aside. We also report that seed variance in this regime is large enough to require tens of seeds per arm, and we release a verified implementation with the pre-registered analysis plan.