Thank you Nick for this. I spent an hour trying to figure this out before I chanced upon this note. I completely forgot (or didn't realize) that the check mark on a completely other setup screen (WAN) had this Automatic DNS checkbox.
I think that there should be a note on the IP Settings > Network about the Temporary DNS vs Automatic and where to find it. Can I log an improvement suggestion somewhere?