I am trying to query the World Bank Microdata API from within R, using the httr package. Unfortunately, I couldn't find much guidance on how to do this. According to Microdata REST API documentation, the path to search in the catalog is http://microdata.worldbank.org/index.php/api/catalog/search, and the relevant path parameters are
{
"idno": "string",
"title": "string",
"nation": "string",
"created": "2022-06-16T06:24:40Z",
"changed": "2022-06-16T06:24:40Z"
}
The minimum working example below attempts to extract information on items from Malawi only. To this end, I set nation = "malawi" and leave the other arguments empty. I also tried nation = "Malawi" (with capital M) and nation = "mwi" (the iso country code). Either way, status_code(r) yields 200, which indicates that the request was successful. Yet, the output from str() clearly shows that we have items from Afghanistan, not Malawi. My hunch is that the nation parameter is not specified correctly and the response from the API yields the first few of ALL its entries, starting with the country that's ranked first alphabetically - Afghanistan.
library(httr)
r <- GET("http://microdata.worldbank.org/index.php/api/catalog/search",
query = list(idno = "", title = "", nation = "malawi", created = "", changed = ""))
status_code(r)
#> [1] 200
str(content(r))
#> List of 1
#> $ result:List of 7
#> ..$ rows :List of 15
#> .. ..$ :List of 19
#> .. .. ..$ id : int 4302
#> .. .. ..$ idno : chr "AFG_2020_VOLREP_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Voluntary Repatriation 2020"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "UNHCR"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2020
#> .. .. ..$ year_end : int 2020
#> .. .. ..$ repositoryid : chr "unhcr"
#> .. .. ..$ repo_title : chr "United Nations Refugee Agency (UNHCR)"
#> .. .. ..$ created : chr "2022-03-14T12:05:12-04:00"
#> .. .. ..$ changed : chr "2022-03-14T12:31:03-04:00"
#> .. .. ..$ total_views : int 1050
#> .. .. ..$ total_downloads : int 42
#> .. .. ..$ varcount : int 11
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/4302"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 4018
#> .. .. ..$ idno : chr "AFG_2021_WBCS_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "World Bank Group Country Survey 2021"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "Public Opinion Research Group"
#> .. .. ..$ form_model : chr "direct"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2020
#> .. .. ..$ year_end : int 2021
#> .. .. ..$ repositoryid : chr "COS"
#> .. .. ..$ repo_title : chr "The World Bank Group Country Opinion Survey Program (COS)"
#> .. .. ..$ created : chr "2021-06-01T15:41:49-04:00"
#> .. .. ..$ changed : chr "2021-06-01T15:47:09-04:00"
#> .. .. ..$ total_views : int 11245
#> .. .. ..$ total_downloads : int 907
#> .. .. ..$ varcount : int 418
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/4018"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 4346
#> .. .. ..$ idno : chr "AFG_2019_LIS_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Livelihoods Programme Monitoring Beneficiary Survey 2019"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "UN Refugee Agency (UNHCR)"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2019
#> .. .. ..$ year_end : int 2019
#> .. .. ..$ repositoryid : chr "unhcr"
#> .. .. ..$ repo_title : chr "United Nations Refugee Agency (UNHCR)"
#> .. .. ..$ created : chr "2022-03-22T11:52:01-04:00"
#> .. .. ..$ changed : chr "2022-04-13T11:24:04-04:00"
#> .. .. ..$ total_views : int 848
#> .. .. ..$ total_downloads : int 18
#> .. .. ..$ varcount : int 134
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/4346"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 4303
#> .. .. ..$ idno : chr "AFG_2019_VOLREP_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Voluntary Repatriation 2019"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "UNHCR"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2019
#> .. .. ..$ year_end : int 2019
#> .. .. ..$ repositoryid : chr "unhcr"
#> .. .. ..$ repo_title : chr "United Nations Refugee Agency (UNHCR)"
#> .. .. ..$ created : chr "2022-03-14T12:29:41-04:00"
#> .. .. ..$ changed : chr "2022-03-14T12:31:04-04:00"
#> .. .. ..$ total_views : int 370
#> .. .. ..$ total_downloads : int 34
#> .. .. ..$ varcount : int 11
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/4303"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 3645
#> .. .. ..$ idno : chr "AFG_2018_SPA_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Service Provision Assessment Survey 2018-2019"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "Ministry of Public Health"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2018
#> .. .. ..$ year_end : int 2019
#> .. .. ..$ repositoryid : chr "dhs"
#> .. .. ..$ repo_title : chr "MEASURE DHS: Demographic and Health Surveys"
#> .. .. ..$ created : chr "2020-03-17T15:38:39-04:00"
#> .. .. ..$ changed : chr "2020-03-17T15:44:27-04:00"
#> .. .. ..$ total_views : int 7132
#> .. .. ..$ total_downloads : int 0
#> .. .. ..$ varcount : int 0
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/3645"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 4304
#> .. .. ..$ idno : chr "AFG_2018_VOLREP_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Voluntary Repatriation 2018"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "UNHCR"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2018
#> .. .. ..$ year_end : int 2018
#> .. .. ..$ repositoryid : chr "unhcr"
#> .. .. ..$ repo_title : chr "United Nations Refugee Agency (UNHCR)"
#> .. .. ..$ created : chr "2022-03-14T12:31:05-04:00"
#> .. .. ..$ changed : chr "2022-03-14T12:31:05-04:00"
#> .. .. ..$ total_views : int 325
#> .. .. ..$ total_downloads : int 39
#> .. .. ..$ varcount : int 11
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/4304"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 3192
#> .. .. ..$ idno : chr "AFG_2018_WBCS_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "World Bank Group Country Survey 2018"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "Public Opinion Research Group"
#> .. .. ..$ form_model : chr "direct"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2018
#> .. .. ..$ year_end : int 2018
#> .. .. ..$ repositoryid : chr "COS"
#> .. .. ..$ repo_title : chr "The World Bank Group Country Opinion Survey Program (COS)"
#> .. .. ..$ created : chr "2018-09-12T10:42:16-04:00"
#> .. .. ..$ changed : chr "2019-08-16T10:48:25-04:00"
#> .. .. ..$ total_views : int 76106
#> .. .. ..$ total_downloads : int 21800
#> .. .. ..$ varcount : int 442
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/3192"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 3246
#> .. .. ..$ idno : chr "AFG_2017_FINDEX_v02_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Global Financial Inclusion (Global Findex) Database 2017"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "Development Research Group, Finance and Private Sector Development Unit"
#> .. .. ..$ form_model : chr "public"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2017
#> .. .. ..$ year_end : int 2017
#> .. .. ..$ repositoryid : chr "global-findex"
#> .. .. ..$ repo_title : chr "Global Financial Inclusion (Global Findex) Database"
#> .. .. ..$ created : chr "2018-10-18T17:06:58-04:00"
#> .. .. ..$ changed : chr "2018-10-31T13:28:32-04:00"
#> .. .. ..$ total_views : int 32264
#> .. .. ..$ total_downloads : int 4092
#> .. .. ..$ varcount : int 103
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/3246"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 3414
#> .. .. ..$ idno : chr "AFG_2017_SABER-SD_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "SABER Service Delivery 2017, Measuring Education Service Delivery"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "World Bank"
#> .. .. ..$ form_model : chr "public"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2017
#> .. .. ..$ year_end : int 2017
#> .. .. ..$ repositoryid : chr "central"
#> .. .. ..$ repo_title : NULL
#> .. .. ..$ created : chr "2019-03-04T13:28:59-05:00"
#> .. .. ..$ changed : chr "2019-06-11T10:43:31-04:00"
#> .. .. ..$ total_views : int 51844
#> .. .. ..$ total_downloads : int 12549
#> .. .. ..$ varcount : int 1424
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/3414"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 3998
#> .. .. ..$ idno : chr "AFG_2017_SEA-HER_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Socio-Economic Assessment of IDPS and Returnees in the Five Most Populated Settlements of Afghanistan's Herat Province 2017"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "United Nations High Commissioner for Refugees (UNHCR)"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2017
#> .. .. ..$ year_end : int 2017
#> .. .. ..$ repositoryid : chr "unhcr"
#> .. .. ..$ repo_title : chr "United Nations Refugee Agency (UNHCR)"
#> .. .. ..$ created : chr "2021-05-21T18:44:10-04:00"
#> .. .. ..$ changed : chr "2021-05-21T18:44:37-04:00"
#> .. .. ..$ total_views : int 3388
#> .. .. ..$ total_downloads : int 0
#> .. .. ..$ varcount : int 185
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/3998"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 3997
#> .. .. ..$ idno : chr "AFG_2017_SEA-KHO_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Socio-Economic Assessment of Pakistani Refugees in Afghanistan's Khost and Paktika Provinces 2017"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "United Nations High Commissioner for Refugees (UNHCR)"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2017
#> .. .. ..$ year_end : int 2017
#> .. .. ..$ repositoryid : chr "unhcr"
#> .. .. ..$ repo_title : chr "United Nations Refugee Agency (UNHCR)"
#> .. .. ..$ created : chr "2021-05-21T18:40:58-04:00"
#> .. .. ..$ changed : chr "2021-05-21T18:41:30-04:00"
#> .. .. ..$ total_views : int 2965
#> .. .. ..$ total_downloads : int 0
#> .. .. ..$ varcount : int 186
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/3997"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 4305
#> .. .. ..$ idno : chr "AFG_2016_VOLREP_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Voluntary Repatriation 2016"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "UNHCR"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2016
#> .. .. ..$ year_end : int 2016
#> .. .. ..$ repositoryid : chr "unhcr"
#> .. .. ..$ repo_title : chr "United Nations Refugee Agency (UNHCR)"
#> .. .. ..$ created : chr "2022-03-14T12:31:05-04:00"
#> .. .. ..$ changed : chr "2022-03-14T12:31:05-04:00"
#> .. .. ..$ total_views : int 318
#> .. .. ..$ total_downloads : int 32
#> .. .. ..$ varcount : int 11
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/4305"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 2786
#> .. .. ..$ idno : chr "AFG_2015_DHS_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Demographic and Health Survey 2015"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "Central Statistics Organization (CSO), Ministry of Public Health (MoPH)"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2015
#> .. .. ..$ year_end : int 2016
#> .. .. ..$ repositoryid : chr "dhs"
#> .. .. ..$ repo_title : chr "MEASURE DHS: Demographic and Health Surveys"
#> .. .. ..$ created : chr "2017-02-27T13:40:30-05:00"
#> .. .. ..$ changed : chr "2017-05-23T13:37:01-04:00"
#> .. .. ..$ total_views : int 191840
#> .. .. ..$ total_downloads : int 3375
#> .. .. ..$ varcount : int 2429
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/2786"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 3651
#> .. .. ..$ idno : chr "AFG_2015_DHS_v01_M_v01_A_IPUMS"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Demographic and Health Survey 2015 - IPUMS Subset"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "Central Statistics Organization (CSO) [Afghanistan], Ministry of Public Health (MoPH) [Afghanistan], and ICF., "| __truncated__
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2015
#> .. .. ..$ year_end : int 2015
#> .. .. ..$ repositoryid : chr "ipums"
#> .. .. ..$ repo_title : chr "Integrated Public Use Microdata Series (IPUMS)"
#> .. .. ..$ created : chr "2020-05-01T13:29:51-04:00"
#> .. .. ..$ changed : chr "2020-05-14T13:38:12-04:00"
#> .. .. ..$ total_views : int 50559
#> .. .. ..$ total_downloads : int 0
#> .. .. ..$ varcount : int 3829
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/3651"
#> .. ..$ :List of 19
#> .. .. ..$ id : int 4306
#> .. .. ..$ idno : chr "AFG_2015_VOLREP_v01_M"
#> .. .. ..$ type : chr "survey"
#> .. .. ..$ title : chr "Voluntary Repatriation 2015"
#> .. .. ..$ nation : chr "Afghanistan"
#> .. .. ..$ authoring_entity: chr "UNHCR"
#> .. .. ..$ form_model : chr "remote"
#> .. .. ..$ data_class_id : NULL
#> .. .. ..$ year_start : int 2015
#> .. .. ..$ year_end : int 2015
#> .. .. ..$ repositoryid : chr "unhcr"
#> .. .. ..$ repo_title : chr "United Nations Refugee Agency (UNHCR)"
#> .. .. ..$ created : chr "2022-03-14T12:31:06-04:00"
#> .. .. ..$ changed : chr "2022-03-14T12:31:06-04:00"
#> .. .. ..$ total_views : int 308
#> .. .. ..$ total_downloads : int 23
#> .. .. ..$ varcount : int 11
#> .. .. ..$ thumbnail : NULL
#> .. .. ..$ url : chr "https://microdata.worldbank.org/index.php/catalog/4306"
#> ..$ found : int 3854
#> ..$ total : int 3854
#> ..$ limit : int 15
#> ..$ offset : int 0
#> ..$ search_counts_by_type:List of 1
#> .. ..$ survey: int 3854
#> ..$ page : int 1
Created on 2022-06-16 by the reprex package (v2.0.1)
Can somebody tell me what I am doing wrong and how these search criteria, specified here in JSON, should be used in R? Cheers in advance...
You are mistakenly using the response sample as a query parameter. If you see search query parameters, it's a country, not a nation. But it will be returned as a nation from the API. It gives you the exact result whether you used country="Malawi" or country="mwi".
library(httr2)
request("http://microdata.worldbank.org/index.php/api/catalog/search") |>
req_url_query(country = "Malawi") |>
req_perform() |>
resp_body_json() |>
str(max.level = 3)
#> List of 1
#> $ result:List of 7
#> ..$ rows :List of 15
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> .. ..$ :List of 19
#> ..$ found : int 123
#> ..$ total : int 5528
#> ..$ limit : int 15
#> ..$ offset : int 0
#> ..$ search_counts_by_type:List of 1
#> .. ..$ survey: int 123
#> ..$ page : int 1
Created on 2024-04-21 with reprex v2.1.0
See it $found : int 123 out of 5528 studies. If you go and filter the web result to Malawi, you will get exactly the same result.