Harvey Schwartz and Peter Mandelson quit Bank of London board