Scores of ANC members ditched Ramaphosa for the ocean