The pagination challenge continues, but I've made some good progress!
At a high level, I'm using the following pattern to page through all of my organization's external records:
- GET https://skytapguidetest.zendesk.com/api/v2/guide/external_content/records
- Extend array with "records" JSON objects
- Check "meta" > "has_more" - if true, prepare another request:
- GET https://skytapguidetest.zendesk.com/api/v2/guide/external_content/records?page%5Bafter%5D=AFTER_CURSOR (where AFTER_CURSOR is MTA > MjA > MzA for my current record set)
- Extend array with "records" JSON objects for each result
What I've found is that paginated requests on the final page (MzA) consistently includes duplicate results from previous requests.
These 3 IDs are consistently duplicated between the initial list request against external_content/records and paginated request with cursor MzA:
These 2 IDs are consistently duplicated between MjA and MzA after cursor requests:
After cursor position MzA as of this writing only returns 5 results, so all are duplicated for this cursor position, but without an apparent order/pattern.
Additionally, some records aren't fetched at all, which leads to failures to create articles due to the external_id already existing (an example is external_id f84ee0e6d232876d1f1880c63f57d697 - record ID unknown because I can't seem to fetch this record via API). This external record's link: https://skytapguidetest.zendesk.com/hc/en-us/search/click?data=BAh7CjoHaWRJIh8wMUVFVkFURlpCS000MEY1MFlaQVRNM0g1OAY6BkVUOgl0eXBlSSIcZXh0ZXJuYWxfY29udGVudF9yZWNvcmQGOwZUOgh1cmxJIjVodHRwczovL2hlbHAuc2t5dGFwLmNvbS9NYWludGVuYW5jZV9XaW5kb3dzLmh0bWwGOwZUOg5zZWFyY2hfaWRJIikwMDgyYTY5My1iMmFkLTRlNDItYmQ4Ni1jN2E2Yjg1YmNmNjAGOwZGOglyYW5raQY%3D--ddf91ba1dc8f48a448a2e608ff1b67f5d7b1d2ec
To troubleshoot, I've tried walking backward through paginated requests by specifying the before_cursor value in page[after] calls - while the results are interesting (I can walk backward in a loop forever; eventually the cursor position recycles), it doesn't seem to provide a full return of external record results, and couldn't really be used to populate a full record fetch without a lot of work. My goal with this approach was to see if starting at a different cursor offset would produce different results - so far that hasn't been the case.
I appreciate your review of this issue! Any help is appreciated!
Please sign in to leave a comment.