pythonsocratasoda

Socrata Metadata API Not Updating Dataset


I have been fussing with this for two days now and am still unsure what I am doing wrong. I have tried using both urllib2 using Python 2.7 and now Requests using Python 3.6. I have even abandoned the API and attempted to use sodapy to update the metadata to no avail.

This snippet is using requests for 3.6. I am passing values through the data arg in the request.get() method, but the values are not being updated with the PATCH lambda.

Here's my code:

#!/usr/bin/python
#coding: utf-8
import base64
import json
import requests

username = '{REDACTED}'
password = '{REDACTED}'
string = '%s:%s' % (username, password)
base64string = base64.b64encode(string.encode())
app_token = '{REDACTED}'

values = """
{
  "metadata" : {
    "id" : "TEST",
    "name" : "Business Entities_TEST"
  }
}
"""

headers = {
  'Accept': '*/*',  
  'X-Socrata-Host': 'data.colorado.gov',
  'Content-Type': 'application/json',
  'Authorization': 'Basic {REDACTED}',
  'X-App-Token': app_token
}

request = requests.get('https://data.colorado.gov/api/views/metadata/v1/4ykn-tg5h', data=values, headers=headers)
request.get_method = lambda: 'PATCH'

response_body = request.json()
print(response_body)

My print response is the current metadata for the dataset and does not include the updates I passed in the values variable. Any help would be greatly appreciated!

EDIT: Here's my code for running python 2.7 with urllib2 with the same results

#!/usr/bin/python
#coding: utf-8
import base64
import json
import requests
from urllib2 import Request, urlopen

username = '{REDACTED}'
password = '{REDACTED}'
auth = base64.encodestring('%s:%s' % (username, password)).replace('\n', '')
app_token = '{REDACTED}'

values = """{
  "metadata" : {
    "name" : "TESTING"
  }
}"""


headers = {
  'Accept': '*/*',  
  'X-Socrata-Host': 'data.colorado.gov',
  'Content-Type': 'application/json',
  'Authorization': 'Basic %s' % auth,
  'X-App-Token': app_token
}

request = Request('https://data.colorado.gov/api/views/metadata/v1/4ykn-tg5h ', data=values, headers=headers)
request.get_method = lambda: 'PATCH'

response_body = urlopen(request).read()
print response_body

Same issue here where the current metadata will print to the console, but my updates in values aren't reflected. I've double checked all of my parameters to ensure there isn't a type in my username, pw, app-token etc. No luck.


Solution

  • I'm Katie, a developer at Socrata, and one of the people that built this API. There are a couple of possible issues here.

    1) The data should not be under a metadata key, just as-is, top-level. That is to say, name should be a top-level key in the JSON you pass to the API, like {"name": "A new name!"}

    2) You won't be able to change the id field. Unless you are using strict input checking (see this section of the docs), the request will still go through, just the ID won't be updated.

    3) You can use the built-in requests.patch method to do this, however, you might be missing the required requests SSL setup, which you can install with pip install "requests[security]"

    Here's some code to get you started. Let me know if you have other questions about this API! The documentation, in case you hadn't discovered it (or for others who stumble across this question) is here: https://socratametadataapi.docs.apiary.io/

    import json
    import requests
    
    # This is the full path to the metadata API on the domain that you care about
    url = 'https://data.colorado.gov/api/views/metadata/v1'
    
    # This is the unique identifier of the dataset you care about
    uid = '4ykn-tg5h'
    
    # And this is your login information
    username = '{your socrata login}'
    password = '{your socrata password}'
    
    headers = {'Content-Type': 'application/json'}
    
    # These are the fields you want to update
    data = {'name': 'Testing 123!'}
    
    response = requests.patch('{}/{}'.format(url, uid),
                              auth=(username, password),
                              headers=headers,
                              # the data has to be passed as a string
                              data=json.dumps(data))
    
    print(response.json())