I cannot to seem to find any examples on this topic and I would like to know how to do it. Can anyone show me an example or point me to a link on how to do pagination in powershell with Invoke web-request? The challenge I am facing is that I am making API calls to a server that only returns 100 rows at a time. In order to get any more rows, I would have to make a second call to the server. I have no clue how to do it. If it helps, here is the link provided by the Canvas LMS and my code that I have so far.
Pagination
Requests that return multiple items will be paginated to 10 items by default. You can set a custom per-page amount with the ?per_page parameter. There is an unspecified limit to how big you can set per_page to, so be sure to always check for the Link header.
To retrieve additional pages, the returned Link headers should be used. These links should be treated as opaque. They will be absolute urls that include all parameters necessary to retrieve the desired current, next, previous, first, or last page. The one exception is that if an access_token parameter is sent for authentication, it will not be included in the returned links, and must be re-appended.
Pagination information is provided in the Link header:
Link: <https://<canvas>/api/v1/courses/:id/discussion_topics.json?opaqueA>; rel="current", <https://<canvas>/api/v1/courses/:id/discussion_topics.json?opaqueB>;> rel="next", <https://<canvas>/api/v1/courses/:id/discussion_topics.json?opaqueC>;> rel="first", <https://<canvas>/api/v1/courses/:id/discussion_topics.json?opaqueD>;> rel="last"The possible rel values are:
current - link to the current page of results. next - link to the next page of results. prev - link to the previous page of results. first - link to the first page of results. last - link to the last page of results. These will only be included if they are relevant. For example, the first page of results will not contain a rel="prev" link. rel="last" may also be excluded if the total count is too expensive to compute on each request.
The beginning product
$curlly=""
$url_main="https://[instance].instructure.com/api/v1/accounts/1/courses?per_page=1"
$security_token="imhungry"
$header = @{"Authorization"="Bearer "+ $security_token; "rel"="last"}
$curlly=Invoke-WebRequest -Headers $header -Method Get -Uri $url_main
$curlly = ConvertFrom-Json $curlly.Content
foreach($course in $curlly)
{
$course.name
}
$curlly.Count
The final product
##This is an example on how to use pagination in powershell
$url_main="https://[instance].instructure.com/api/v1/accounts/1/courses?per_page=100"
$security_token="boyimhungry"
$header = @{"Authorization"="Bearer "+ $security_token}
$purlly=Invoke-WebRequest -Headers $header -Method Get -Uri $url_main
$curlly = ConvertFrom-Json $purlly.Content
$url_main = $purlly.Headers.Link.Split(",")[1].Replace("<","").Replace(">","") ## you can get away with just doing one replace("<","") but it looks neater this way
while( !$url_main.Contains("prev"))
{
$purlly=Invoke-WebRequest -Headers $header -Method Get -Uri $url_main
$curlly += ConvertFrom-Json $purlly.Content
$url_main = $purlly.Headers.Link.Split(",")[1].Replace("<","").Replace(">","")
cls
$curlly.Count
$url_main
}
foreach($course in $curlly)
{
$course.name
}
$curlly.Count
This looks like a real pain.
On each paginated request, you're going to get back a Link header that contains one or more of the links they describe.
For your purposes (a sequential read through of every result), you only really need to be concerned with the link rel=next link. You would keep calling that one until there is no rel=next anymore, which is how you'll know that you're on the last page.
You don't provide the rel part; it's not a header. It's something that lets you identify which link to use, and then use that link as is.
So the basic overview of what you need to do:
Link header, and find the one that corresponds to rel=next (if it's not there, you're done).Link header again, repeat.