pythondjangologgingairbrake

Airbrake notifications disabled in Django tests?


I've configured a Django app called lucy_web to log errors to Airbrake using pybrake. In a module in the lucy_web hierarchy, lucy_web.lib.session_recommendations, I've defined a testing function:

import logging

logger = logging.getLogger(__name__)

def log_something():
    logger.error("Logging something...")

If I call this function from the Django shell, like so:

(venv) Kurts-MacBook-Pro-2:lucy-web kurtpeek$ python manage.py shell
Python 3.6.4 (v3.6.4:d48ecebad5, Dec 18 2017, 21:07:28) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.3.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from lucy_web.lib.session_recommendation import *

In [2]: log_something()

I see an error email appear:

enter image description here

However, if I define a test, and try to call it from there, I don't see any new instances of the error appear:

from django.test import TestCase
from django.core import mail


class SessionRecommendationTestCase(TestCase):
    def test_airbrake_notification_if_session_type_does_not_exist(self):
        from lucy_web.lib.session_recommendation import log_something
        log_something()
        import ipdb; ipdb.set_trace()

Running this did not cause any new instances of the error appear in the Airbrake dashboard. Also, mail.outbox is empty:

(venv) Kurts-MacBook-Pro-2:lucy-web kurtpeek$ python manage.py test lucy_web.tests.test_session_recommendation.SessionRecommendationTestCase.test_airbrake_notification_if_session_type_does_not_exist
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
--Return--
None
> /Users/kurtpeek/Documents/Dev/lucy2/lucy-web/lucy_web/tests/test_session_recommendation.py(377)test_airbrake_notification_if_session_type_does_not_exist()
    375         from lucy_web.lib.session_recommendation import log_something
    376         log_something()
--> 377         import ipdb; ipdb.set_trace()

ipdb> mail.outbox
[]

As I understand from https://docs.djangoproject.com/en/2.0/topics/testing/tools/#email-services, within a test, Django redirects all email sent by Django to a dummy outbox, mail.outbox. I would not expect this to the case for Airbrake, though, since the emails are probably sent by their backend - and if it were the case, I wouldn't expect mail.outbox to be empty.

Why am I not getting any Airbrake notification for the test? Is it something more general, like that all logging is disabled/captured during the test?


Solution

  • I verified by setting a trace in the the pybrake.Notifier's send_notice method that it does not get called in the unit test, whereas it does when I call the function 'manually' from the Django shell.

    To still get test coverage, I worked around this problem simply asserting that logger.error gets called:

    from django.test import TestCase
    from unittest.mock import patch
    
    class SessionRecommendationTestCase(TestCase):
        @patch('lucy_web.lib.session_recommendation.logger.error')
        def test_create_session_from_non_existent_session_type_title_triggers_error_logging(self, mock_error):
            title = "Being Awesome"    # Title of a non-existent session type
            _create_sessions([title])
            mock_error.assert_called_with(
                f"Tried to create a session of type '{title}', but no such session type was found in the database.")
    

    where this is the actual function I'm trying to test:

    def _create_sessions(title_list, starting_number=1, **kwargs):
        '''Auxiliary function to create sessions from a list of SessionType titles'''
        session_number = starting_number
        for title in title_list:
            session_type = SessionType.objects.filter(title=title).first()
            if session_type:
                Session.objects.create(
                    session_type=session_type,
                    session_number=session_number,
                    **kwargs)
                session_number += 1
            else:
                logger.error(
                    f"Tried to create a session of type "
                    f"'{title}', but no such session type was found in the database.")
    

    This test passes:

    (venv) Kurts-MacBook-Pro-2:lucy-web kurtpeek$ python manage.py test lucy_web.tests.test_session_recommendation.SessionRecommendationTestCase.test_create_session_from_non_existent_session_type_title_triggers_error_logging
    Creating test database for alias 'default'...
    System check identified no issues (0 silenced).
    .
    ----------------------------------------------------------------------
    Ran 1 test in 0.004s
    
    OK
    Destroying test database for alias 'default'...