They're supposed to stay folded until the button is pressed again. Must be an issue with your switch.