It is easily popped back out with a thin tool, or can be avoided by pressing the leftmost edge of the button.
eBay