What he meant was that we have all left the basement without being discovered, so there is nothing to worry about until now.
As a follower of Eternal Blazing Sun, it is normal to come to church to pray.
As for taking a shortcut and taking the side door, that is a trivial matter that can be ignored, and what does it have to do with us if the curate and the servants take too long a lunch break?