Force me to pay extra

I have switched bank accounts this month. Medihelp is of course one of those providers that will not take instruction from the bank just to make your life difficult. However what makes them special is that they expect me to not only keep 2 banks open for a whole extra month just for them, they also expect me to go pay the bank for a stamped statement just for them. This is with more than 2 weeks notice and me actually trying to provide the information they need to keep on taking my money. I am shocked at how bad their attitude is, no you are not doing me a favor, I am entitled to change my bank account as long as I let you know and ensure my payments to you still goes through