[SOLVED] Getting all records in a set using the Sickle package

Getting all records in a set using the Sickle package

How can I access all the records in each set using Sickle?

I can access sets like this, but I don't know how to go from here and download each record from every set:

from sickle import Sickle

sickle = Sickle('http://www.duo.uio.no/oai/request')
    sets = sickle.ListSets()
    for s in sets:
        print s

The print prints out every set like this:

<set xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><setSpec>com_10852_1</setSpec><setName>Det matematisk-naturvitenskapelige fakultet</setName></set>

I can also iterate through the sets to go deeper:

for s in sets:
    for rec in sets:
        print rec

This prints all the sub-sets, so it's probably from here I can get access to the individual records, but the API is hard to understand, and I have not be able to access the records.

Solution

Be sure to read the short and sweet Tutorial.

For harvesting an entire OAI-PMH repository, you do not need to iterate over sets. Here is the complete code:

from sickle import Sickle

sickle = Sickle('http://www.duo.uio.no/oai/request')
recs = sickle.ListRecords(metadataPrefix="oai_dc")
for r in recs:
    print r

If for some reason you really wish to harvest records set by set, you can certainly do so. Here is the complete code again:

from sickle import Sickle

sickle = Sickle('http://www.duo.uio.no/oai/request')
sets = sickle.ListSets()
for s in sets:
    recs = sickle.ListRecords(metadataPrefix="oai_dc", set=s.setSpec)
    for r in recs:
        print r