[FEAT]: Give user control to stop pagination earlier

[jetzhou]

Describe the need

Use case

The default behavior right now is to paginate through all pages always. However, in certain cases users may want to stop the pagination earlier. For example, it could be useful to stop after a certain number of pages or to stop after a particular item shows up in the results. This use case can be solved by using the iterator function/interface, but it then forces the users to do manual merging on their end, which reduces the benefit of using this plugin.

Prior art

The REST plugin provides such a control mechanism via a mapFunction parameter. However, it's called a mapFunction even though it actually serves 2 purposes: mapping/transforming and stopping early. In GraphQL world, mapping/transforming is not necessary nor expected.

Proposal

Add a stopFunction as the third argument to iterator and paginate. After each page is fetched, the function will be invoked with two parameters, first is the response of the most recently fetched page, and the second is a done function that stops the iteration if invoked.

The usage scenarios would be covered like the following snippets.

• To stop after a max number of pages:

const maxPages = 2;
let pages = 0;

await octokit.graphql.paginate(
  `query paginate ($cursor: String) {
    repository(owner: "octokit", name: "rest.js") {
      issues(first: 10, after: $cursor) {
        nodes {
          title
        }
        pageInfo {
          hasNextPage
          endCursor
        }
      }
    }
  }`,
  {},
  (_, done) => {
    pages += 1;
    if (pages >= maxPages) {
      done();
    }
  },
);

• Or, to stop after you find a certain item:

await octokit.graphql.paginate(
  `query paginate ($cursor: String) {
    repository(owner: "octokit", name: "rest.js") {
      issues(first: 10, after: $cursor) {
        nodes {
          title
        }
        pageInfo {
          hasNextPage
          endCursor
        }
      }
    }
  }`,
  {},
  (response, done) => {
    if (response?.repository?.issues?.nodes?.[0].title === "Issue 2") {
      done();
    }
  },
);

Alternative implementation

Instead modeling after the REST plugin's API, we could just add an options object as the third argument and allow users to pass in more explicit "stop conditions". For example:

• for max number of pages:

{ maxPages: 3 }

• or, for stopping after finding an item

{ stopIteration: (response) => response?.repository?.issues?.nodes?.[0].title === "Issue 2" }

This could allow other options to be added in the future so it's more extensible. But this will also mean that there is a divergence in API between REST and GraphQL paginate plugins.

Related PR

#163

SDK Version

No response

API Version

No response

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

👋 Hi! Thank you for this contribution! Just to let you know, our GitHub SDK team does a round of issue and PR reviews twice a week, every Monday and Friday! We have a process in place for prioritizing and responding to your input. Because you are a part of this community please feel free to comment, add to, or pick up any issues/PRs that are labled with Status: Up for grabs. You & others like you are the reason all of this works! So thank you & happy coding! 🚀