On Distinguishing Interstellar Objects Like `Oumuamua From Products of Solar System Scattering