Billionaire businessman Stephen Cloobeck directly confronted Eric Swalwell about the sexual assault allegations against him, telling the congressman he'd "busted the trust" ... before Cloobeck gave him the boot, TMZ has learned. In a phone…