It was a while before I figured out that a "cold start" seemed to always work while a "restart" seemed to fail. This lead me to look into the BIOS (a.k.a. Unified Extensible Firmware Interface or "UEFI" these days). Sure enough, I found a couple of entries that seemed to make sense: SecureBoot and QuickBoot. Disable these both and boom hitting ENTER on the power on screen gets you in to BIOS each time.
Note, this really isn't all that much different in terms of boot speeds, so the ability to choose where to boot from, including USB (and not just boot partition which GRUB can handle) is worth it.
OBLIGATORY WARNING: MONKEYING AROUND WITH YOUR BIOS SETTINGS CAN LEAVE YOU WITH A VOID WARRANTY, A VOID SYSTEM, OR BOTH.