Respect closeOnSelect property

This logic was lost in a previous commit, hopefully this new check will
be enough to cover all the cases as well as not ruining any others. I
did not manage to write a test for this as it required props I do not
know how to mock to be able to reproduce.