Automation tied to Out of Office app misfiring

My team recently re-installed the Out of Office app, and trained agents to check their "Agent Out?" box in their user profile so that new tickets cannot be assigned to them.

We also have automations that will re-open stale tickets after a certain amount of time, and we added a line to those automations that will also check to make sure the "Agent Out?" box is unchecked before firing and opening the ticket.

However today we found that these automations were still firing, and because tickets are blocked from being assigned to the agent, the ticket becomes unassigned and escapes from view until someone checks the unassigned queue.

Am I misunderstanding how the "Requester: [Out of Office] Agent Out?" line is supposed to work within triggers and automations, or is there something to be fixed on the app end?