Support doctest

[ArneBachmannDLR]

I try to use doctest only in my code (with occasional unittest.DoctestRunner) and it's always a hassle to capture log output, because doctest clones the logger instances transparently (or something).

[etianen]

I think doctest is already supported. Check this out:

import doctest
import logging

logger = logging.getLogger(__name__)


def foo() -> None:
    """
    This is foo.

    >>> foo()
    >>> logot.assert_logged(logged.info("foo"))
    """
    logger.info("foo")


if __name__ == "__main__":
    import doctest

    from logot import Logot, logged

    with Logot().capturing() as logot:
        doctest.testmod(verbose=True)

I don't use doctest myself though. What would you want out of doctest support that isn't already provided here? Best I can think of is a logot.doctes.testmod() wrapper that injected logot and logged into the doctests automatically, so you could just do this:

import doctest
import logging

logger = logging.getLogger(__name__)


def foo() -> None:
    """
    This is foo.

    >>> foo()
    >>> logot.assert_logged(logged.info("foo"))
    """
    logger.info("foo")


if __name__ == "__main__":
    import doctest

    from logot.doctest import testmod

    testmod(verbose=True)

Is that the sort of thing you were thinking of?

[etianen]

Looking at the doctest source, it seems hard/impossible to write an extension for it, as it has no plugin system like pytest, and no way of mixing in a test case subclass like unittest.

So while it would be possible to make a simple testmod wrapper, doing so wouldn't work for doctests run from the command line, (or via pytest, or via doctests own unittest integration!). Very dissapointing.

I think you can do a lot using the existing support in my first example. And that at least demonstrates that log capture workes fine in doctests! If there's anything else you think could be added to make things better, I'm all ears! 🙇

[ArneBachmannDLR]

It works great so far! I had this code before:

def myfun():
  >>> import logging, sys; handler = logging.StreamHandler(sys.stdout)
  >>> from logsetup.logsetup import get_log_object  # a convenience function
  >>> logger = get_log_object(__name__)
  >>> logger.addHandler(handler)
  >>> logger.setLevel(logging.DEBUG)

  >>> # do the stuff
  WHAT WE EXPECT
  AS OUTPUT

  >>> logger.removeHandler(handler)  # mandatory cleanup
  ...

The difficulty comes when running modules not via python -m package.module but as script or from a DoctestRunner. In that case I need to have a logot reference that works across modules, which is a hassle in Python. Something like that:

# Top of file
if __sut__ or '--test' in sys.argv:  # code put here to reuse in tests.py, otherwise keep it at bottom test block
  from logot import Logot, logged
  logot_capture = Logot().capturing
  logot = []

def myfunc():
  >>> logot = logot[0]
  ...

# End of file
if __sut__:
  import doctest
  with logot_capture() as _logot: logot.append(_logot); doctest.testmod(optionflags=doctest.IGNORE_EXCEPTION_DETAIL)

and somewhere else

  dt_runner = doctest.DocTestRunner(optionflags=doctest.IGNORE_EXCEPTION_DETAIL|doctest.ELLIPSIS)
  with logot_capture() as _logot:
    logot.append(_logot)
    for t in tests: failed, attempted = dt_runner.run(t); errors += failed; success += 1 if failed == 0 else 0

This ensures there's a cross-module writable global variable which can be used inside doctest, even when the logot variable is defined in another module. The reason is that I wasn't able to use logot without the context manager (with .... as logot:).

If there is another way to use it, that would simplify it very much.

[etianen]

Hmm. So maybe it would be possible to have an api like:

logot = Logot.get_current()

This would return a reference to the logot instance that's currently capturing logs. You could use this in any doctest.

[etianen]

Alternatively, a LogotDoctestRunner subclass that injected an initialized logot would be possible too. But that would be useless with pytest doctest runners and higher-level doctest APIs.

doctest is frustrating, because subclassing one of the low-level APIs doesn't help with any of the high-level APIs!

[ArneBachmannDLR]

I like the get_current approach(). It would keep the code in the doctest and out of the module code. Thanks for all the support so far!