You don't have to, but you could house rule it. What you have said makes perfect sense and is a very realistic way to play the Quest action.
Quest time question:
When you complete a quest in under a “day” should you mark a minimum of one day on the calendar?
I figure you headed out that day for the Quest, and if you maybe managed to get it done in just a few hours you might have time for something else that day but more likely you should consider travel and handing in and or going to collect your reward, if applicable. Otherwise the last marked day on the calendar would when my adventurer was doing something else, be it an event or travel or scouting or hunting… also if less than 12 hours, having not eaten food in the dungeon, I’d think I should spend a ration.