Concern expressed by John Prescott and Jack Straw in 2004 over possible migration wave later borne out in Brexit vote