pythonconfluenceconfluence-rest-api

In Confluence, how to replicate manual search with API search?

I am following the Confluence API search documentation to implement text search with CQL (confluence query language) in the company Confluence pages. Here is my code for a search query:

import requests
from requests.auth import HTTPBasicAuth
import urllib.parse

# Replace with your Confluence credentials and base URL
base_url = 'https://your-domain.atlassian.net/wiki'
username = 'your-email@example.com'
api_token = CONFLUENCE_API_TOKEN

search_term = 'What is the per diem costs allowed to be reimbursed for business trips to Paris?'
encoded_search_term = urllib.parse.quote(search_term)

# Construct the search URL with encoded search term
search_url = f'{base_url}/rest/api/content/search?cql=text~"{encoded_search_term}"&limit=10'

# Send the request
response = requests.get(search_url, auth=HTTPBasicAuth(username, api_token))

# Check for successful response
if response.status_code == 200:
    search_results = response.json()
    print(search_results)
else:
    print(f'Error: {response.status_code}, {response.text}')

Here is the result returned:

{'results': [], 'start': 0, 'limit': 10, 'size': 0, '_links': {'base': 'https://your-domain.atlassian.net/wiki', 'context': '/wiki', 'self': 'https://your-domain.atlassian.net/wiki/rest/api/content/search?cql=text~%22What%20is%20the%20per%20diem%20costs%20allowed%20to%20be%20reimbursed%20for%20business%20trips%20to%20Paris%3F%22'}}

So zero documents returned during the search.

Whereas if I do the search manually in the Confluence page, here is the result:

enter image description here

There are loads of documents retrieved with manual search.

When I tried shorter search queries with the API, I do get results. Like, for "What is the per diem costs", I get 14 confluence documents retrieved; for "What is the per diem costs allowed to be reimbursed", I get 7 results; and 4 results for "What is the per diem costs allowed to be reimbursed for business trips". But zero results for "What is the per diem costs allowed to be reimbursed for business trips to Paris?"

But these are all nowhere close to what I get with manual search (thousands of documents retrieved).

So, how do I replicate this manual search with the API? What is the search algorithm used in the "simple" manual search?

Here is the same search with the Atlassian API. The result is the same, zero documents returned:

enter image description here

Solution

  • I used the Network tab of developer console to see what exactly Confluence's "Simple" search is doing.

    Turns out it constructs a cql as follows:

    siteSearch ~ "SEARCH TEXT HERE" AND type in ("space","user","com.atlassian.confluence.extra.team-calendars:calendar-content-type","attachment","page","com.atlassian.confluence.extra.team-calendars:space-calendars-view-content-type","blogpost")

    So try changing text to siteSearch:

    cql_query = f'siteSearch ~ "{search_term}"'
confluence.cql(cql_query, limit=1000)  # default limit is 25

    Note: Using this method I get a maximum of 500 results. Confluence API documentation does not mention this limit; but the documentation of the Confluence class of atlassian-python-api mentions that fixed system limits may be imposed, which I presume is what's happening here. If you need more than 500 results, you can do repeated calls using the start parameter of cql()