There are some "government" bodies who can legally invoke a "deduction order" to recover debt owed, without the consent or knowledge of the account holder, but I've not heard of this from a private company such as Virgin Media.
As The Dr says, direct debits carry safeguards, so if in this case your account has been debited without good reason or sufficient notice, you can legally have recourse the direct debit guarantee, which means your bank are obliged to refund you in full, and then it is a matter for them to take the issue up with the third party to recover "their" money.
However, again as The Dr states, beware "regular card payments", as these differ greatly from direct debits. :
Cancelling a regular card payment
When you give your credit or debit card details to a company and authorise them to take regular payments from your account, such as for a gym membership or magazine subscription, it is known as a ‘recurring transaction’ or ‘continuous payment authority’.
These are often confused with direct debits, but do not offer the same guarantee if the amount or date of the payment changes.
In most cases, regular payments can be cancelled by telling the company taking the payments. However, you have the right to cancel them directly with your bank or card issuer by telling it that you have stopped permission for the payments. Your bank or card issuer must then stop them – it has no right to insist that you agree this first with the company taking the payments.
Be aware though that you will still be responsible for paying any money that you owe.
As these recurring card payments afford less rights than direct debits, as a general point, it is much better to avoid them if possible, but I don't know if that helps you in this instance BurgerChips? Good luck with resolving your problem anyway!