"A judicial finding attributing an adulterous relationship to a specifically identified person cannot be regarded as an inconsequential observation. Such a finding may have consequences