Is there a way to set one or the other?

Currently you MUST set a User Alert Group even if you have an action set. 

In our case, we use actions to create events in Big Panda but because we also MUST have a User Alert Group set we are doube alerting on the same outage in Site24x7.

Having the option that you MUST have one of the two is fine.