A Look at Structlog

I spent some time reading through the docs of the logging library in Python. It’s a bit of a pain to use.

Someone on Bluesky pointed me to structlog.

The one thing that convinced me to try it out was the context manager.

You can do something like this:

# Create a logger for User 5
log = logger.bind(user_id=5)

# Now every log entry will include the user_id
log.info("order_started")

This is sick for web applications.

You can bind user parameters to the logger in the middleware and be done with it.

When you have already written your log messages, it will take you a while to migrate to `structlog`.

It is designed to have minimal event names instead of verbose messages.

# Don't do this:
logger.info(f"Order {order_id} processed with total {total}")

# Do this:
logger.info("order_processed", order_id=order_id, total=total)

You add variables to the log messages and don’t insert them into the message string.

This makes it easier to render them in different formats.

You use the configuration to set the format

You can set the format of the log messages in the configuration.

import structlog

structlog.configure(
    processors=[
        structlog.processors.JSONRenderer()
    ]
)

This renders the log messages as JSON.

But you can also use the ConsoleRenderer to render the log messages as text (in colors ;)).

The base configuration I went with in my project is this:

def configure_logging():
     # Configure structlog processors
    processors = [
        # Context & metadata first
        structlog.contextvars.merge_contextvars,
        structlog.processors.add_log_level,

        # Call site info early for visibility
        structlog.processors.CallsiteParameterAdder([
            structlog.processors.CallsiteParameter.PATHNAME,
            structlog.processors.CallsiteParameter.FILENAME,
            structlog.processors.CallsiteParameter.FUNC_NAME,
            structlog.processors.CallsiteParameter.LINENO,
            structlog.processors.CallsiteParameter.MODULE,
        ]),

        # Timestamps and stack info
        structlog.processors.TimeStamper(fmt="iso", utc=False),
        structlog.processors.StackInfoRenderer(),

        # Exception handling
        structlog.dev.set_exc_info,
        structlog.processors.format_exc_info,
        structlog.processors.ExceptionPrettyPrinter(),

        # Cleanup and encoding
        structlog.processors.UnicodeDecoder(),

        # Final rendering (Cloudwatch by AWS automatically formats JSON)
        structlog.dev.ConsoleRenderer(
            colors=True, # probably should be set in the environment
            exception_formatter=structlog.dev.plain_traceback
        )
    ]

    # Configure structlog with numeric level
    structlog.configure(
        processors=processors,
        wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
        context_class=dict,
        logger_factory=structlog.PrintLoggerFactory(),
        cache_logger_on_first_use=True
    )

CallsiteParameterAdder adds the file name, function name, and line number to the log message.
TimeStamper adds the timestamp to the log message.
StackInfoRenderer adds the stack trace to the log message.
ExceptionPrettyPrinter adds the exception to the log message.
UnicodeDecoder decodes the log message to Unicode.
ConsoleRenderer renders the log message to the console.

Good logging is like having a clear conversation with you in a few years when you have forgotten what you did.

When you have already written your log messages, it will take you a while to migrate to structlog.

You use the configuration to set the format

When you have already written your log messages, it will take you a while to migrate to `structlog`.