Circumbinary castaways: Short-period binary systems can eject orbiting worlds