python-logstash-async-dwn 1.0.0

关于

这个 Python 日志处理程序是 https://github.com/eht16/python-logstash-async的一个分支

它具有以下特点：

• 日志事件的异步传输

• 通过 TCP 和 UDP 传输事件

• TCP 传输可选 SSL 加密

• 准备在 Django 项目中使用的特殊格式化程序

与https://github.com/eht16/python-logstash-async相比，如果无法与 logstash 建立连接，此分支不会尝试保留日志语句。

安装

使用点子：

pip install git+https://github.com/koendeschacht/python-logstash-async.git

用法

AsynchronousLogstashHandler是一个自定义日志处理程序，它使用 UDP 和 TCP 发送 Logstash 消息。例如：

import logging
import sys
from logstash_async.handler import AsynchronousLogstashHandler

host = 'localhost'
port = 5959

test_logger = logging.getLogger('python-logstash-logger')
test_logger.setLevel(logging.INFO)
test_logger.addHandler(AsynchronousLogstashHandler(
    host, port)))
# test_logger.addHandler(AsynchronousLogstashHandler(host, port))

test_logger.error('python-logstash-async: test logstash error message.')
test_logger.info('python-logstash-async: test logstash info message.')
test_logger.warning('python-logstash-async: test logstash warning message.')

# add extra field to logstash message
extra = {
    'test_string': 'python version: ' + repr(sys.version_info),
    'test_boolean': True,
    'test_dict': {'a': 1, 'b': 'c'},
    'test_float': 1.23,
    'test_integer': 123,
    'test_list': [1, 2, '3'],
}
test_logger.info('python-logstash: test extra fields', extra=extra)

使用额外字段时，请确保您不使用保留名称。来自Python 文档：

"The keys in the dictionary passed in extra should not clash
with the keys used by the logging system.
(See the `Formatter <https://docs.python.org/2/library/logging.html#logging.Formatter>`_ documentation
for more information on which keys are used by the logging system.)"

您还可以在日志配置中指定一个额外的额外字典，其中包含应用程序名称、环境等静态值。这些值将与日志调用中传递的任何额外字典项合并到配置的额外前缀中。

与 Django 一起使用

修改您的settings.py以将python-logstash-async与 Django 的日志记录集成：

LOGGING = {
  ...
  'formatters': {
      ...
      'logstash': {
          '()': 'logstash_async.formatter.DjangoLogstashFormatter',
          'message_type': 'python-logstash',
          'fqdn': False, # Fully qualified domain name. Default value: false.
          'extra_prefix': 'dev', #
          'extra': {
              'application': PROJECT_APP,
              'project_path': PROJECT_APP_PATH,
              'environment': 'production'
          }
      },
  },
  'handlers': {
      ...
      'logstash': {
          'level': 'DEBUG',
          'class': 'logstash_async.handler.AsynchronousLogstashHandler',
          'transport': 'logstash_async.transport.TcpTransport',
          'host': 'logstash.host.tld',
          'port': 5959,
          'ssl_enable': True,
          'ssl_verify': True,
          'ca_certs': 'etc/ssl/certs/logstash_ca.crt',
          'certfile': '/etc/ssl/certs/logstash.crt',
          'keyfile': '/etc/ssl/private/logstash.key',
      },
  },
  'loggers': {
      'django.request': {
          'handlers': ['logstash'],
          'level': 'DEBUG',
          'propagate': True,
      },
  },
  ...
}

这将导致如下所示的 Logstash 事件（注意：在某种程度上取决于您的 Logstash 配置）：

{
    "@timestamp": "2016-10-23T15:11:16.853Z",
    "@version": "1",
    "extra": {
        "application": "django_example",
        "django_version": "1.10.2",
        "environment": "production",
        "func_name": "get_response",
        "interpreter": "/home/enrico/example/venv/bin/python",
        "interpreter_version": "2.7.12",
        "line": 152,
        "logger_name": "django.request",
        "path": "/home/enrico/example/venv/lib/python2.7/site-packages/django/core/handlers/base.py",
        "process_name": "MainProcess",
        "project_path": "/home/enrico/example/app",
        "req_host": "localhost",
        "req_method": "GET",
        "req_referer": "",
        "req_remote_address": "127.0.0.1",
        "req_uri": "http://localhost/hosts/nonexistent/",
        "req_user": "enrico",
        "req_useragent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.1",
        "request": "<WSGIRequest: GET '/hosts/nonexistent/'>",
        "status_code": 404,
        "thread_name": "Thread-1"
    },
    "host": "my.host.tld",
    "level": "WARNING",
    "logsource": "endor.l8failed.net",
    "message": "Not Found: /hosts/nonexistent/",
    "pid": 23605,
    "port": 56170,
    "program": "manage.py",
    "type": "python-logstash"
}

配置

示例 Logstash 配置

未加密 TCP 传输的示例logstash.conf ：

input {
    tcp {
        host => "127.0.0.1"
        port => 5959
        mode => server
        codec => json
    }
}

SSL 加密 TCP 传输的示例logstash.conf ：

input {
    tcp {
        host => "127.0.0.1"
        port => 5958
        mode => server
        codec => json

        ssl_enable => true
        ssl_verify => true
        ssl_extra_chain_certs => ["/etc/ssl/certs/logstash_ca.crt"]
        ssl_cert => "/etc/ssl/certs/logstash.crt"
        ssl_key => "/etc/ssl/private/logstash.key"
    }
}