With the following pytest file:

(import pytest [mark])
(import pathlib [Path])
(require oreo [with-cwd])
(defn [mark.with-cwd] test-with-cwd [cookies]
      (let [ cwd (.cwd Path) ]
           (with-cwd cookies (assert (= (.cwd Path) cookies)))
           (assert (= (.cwd Path) cwd))))

I'm getting the following error:

Traceback (most recent call last):
  File "/home/shadowrylander/shadowrylander/sylveon/syvlorg/oreo/tests/test-with-cwd.hy", line 7, in test_with_cwd
    (with-cwd cookies (assert (= (.cwd Path) cookies)))
NameError: name 'os' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/shadowrylander/shadowrylander/sylveon/syvlorg/oreo/tests/test-with-cwd.hy", line 7, in test_with_cwd
    (with-cwd cookies (assert (= (.cwd Path) cookies)))
NameError: name 'os' is not defined

The cookies fixture is as follows:

@pytest.fixture()
def cookies(request):
    from pathlib import Path
    return Path(request.config.rootdir).expanduser().resolve(strict = True) / "cookies"

And finally, with-cwd in the oreo module is basically:

(eval-and-compile (import os hy))
(defmacro with-cwd [dir #* body]
          (setv cwd (hy.gensym))
          `(let [ ~cwd (.cwd Path) ]
                (try (.chdir os ~dir)
                     ~@body
                     (finally (.chdir os ~cwd)))))

I am using hy version 0.25.0. The test works if I import os in the test file itself.

Update

This new one does not seem to work either, giving me the same error:

(defmacro with-cwd [dir #* body]
          (setv cwd (hy.gensym)
                os (hy.gensym))
          `(let [ ~cwd (.cwd Path) ]
                (import os :as ~os)
                (try ((. ~os chdir) ~dir)
                     ~@body
                     (finally ((. ~os chdir) ~cwd)))))

It sounds like you've all but entirely answered your own question, but haven't quite the connected the dots. As usual, it's easier to see what's going on in a minimal example:

(defmacro m []
  (import os)
  `(os.getcwd))

(print (m))

This raises NameError: name 'os' is not defined because os indeed is not defined at runtime in the scope in which m expands. The code is equivalent to simply

(print (os.getcwd))

So either os has to already be in the scope where the macro expands, or you have to include the import in the expansion, like so:

(defmacro m []
  (setv os (hy.gensym))
  `(do
    (import os :as ~os)
    ((. ~os getcwd))))

(print (m))

Or, use hy.I:

(defmacro m []
  `(hy.I.os.getcwd))

(print (m))