"Literalism breaks down when magic is involved, but whatever the case, you have to admit that smashing the wall really loses some of the subtle intricacies of going through the door. And 'going back to the proper time' is one of those intricacies!"