Python Unit-test Mocking, unable to patch `time` module that's being imported in `__init__.py`

I've the following code in the __init__.py file

from time import sleep
from functools import wraps

def multi_try(func):
    @wraps(func)
    def inner(*args, **kwargs):
        count = 0
        while count < 5:
            resp = func(*args, **kwargs)
            if resp.status_code in [200, 201]:
                return resp
            sleep(1)
            count += 1
    return inner

While writing tests for the above decorator I'm not able to patch the the time.sleep properly.

See the test below, even though I've patched time module, still the sleep function inside the decorator getting called, thereby test case require 5+ seconds to finish.

def test_multi_try_time():
    with patch("time.sleep") as tm: 
       mocker = MagicMock(name="mocker")
       mocker.__name__ = "blah"
       resp_mock = MagicMock()
       resp_mock.status_code=400
       _json = '{"test":"twist"}'
       resp_mock.json=_json
       mocker.return_value = resp_mock
       wrapped = multi_try(mocker)
       resp = wrapped("p", "q")
       assert mocker.call_count == 5
       mocker.assert_called_with('p', 'q')
       assert resp == None

Also I tried this,

with patch("dir.__init__.time" ) as tm:

and

with patch("dir.utils.time" ) as tm:

That resulted in

AttributeError: <module 'dir/__init__.pyc'> does not have the attribute 'time'

All I had to do was

with patch("dir.sleep" ) as tm:

Instead of,

with patch("time.sleep") as tm: