Feature Request: Agent Inactive time resetting

My team started using Omnichannel routing with agent workspace and discovered that the inactive time of an agent is now reset even if they miss or decline a call and even if the call is abandoned while ringing at the agent.

After talking to support they informed me this is by design with Omnichannel routing. I believe this design can lead to issues with uneven call routing. 

Say an agent had the call abandoned at them, which now resets their inactive time. Now the following calls that come in will go to other agents first. Leading to those other agents doing 2 calls before this agent did any.
Say they get unlucky and the next call that came to this agent also got abandoned while ringing at them. This would now lead to all other agents doing 3 calls before this agent did any.

It seems like a design flaw in the system with potential for causing uneven call routing to teams. I think it would be beneficial to have an option where an agent's inactive time only resets if they actually serve an inbound or outbound call. Same as when Omnichannel routing is not used.