Does selecting Within their groups prevent agents from searching for other tickets?
Yes, when agents can only access tickets within their groups, search only returns tickets they have access to. All other tickets are hidden.
If agents search for a specific ticket number they cannot access, Support doesn't search the ticket number. For example, if agents go to the URL
https://subdomain.zendesk.com/agent/tickets/### of a ticket they cannot access, Support shows access denied.
Although restricting an agent's access to see only tickets in their group determines what tickets they can see in the search results, this does not affect the visibility of other search results, such as users or organizations. To learn more about further restricting an agent's permission, see the article: Creating custom roles and assigning agents.