Kafka Logs

Collect log information from request gateways and output it to Kafka to facilitate further operations, such as log analysis.

Features

The ability to output log contents generated during program execution to a specified Kafka cluster queue.

Operation Demonstration

Create a New Kafka Log Configuration

1. Click on Operations and Integration -> Log Configuration -> Kafka Logs in the left navigation bar, then click on Add Kafka Log.

2. Fill in the Kafka log configuration.

Configuration Explanation:

Field Name	Description
Version	Kafka version
Server Address	Addresses of Kafka services, multiple addresses are separated by commas
Topic	Topic information for Kafka service
Partition Type	Method for selecting partitions, default is hash; if partition_key is empty when hash is selected, it defaults to random selection
Partition	Specifies the partition number when Partition Type is set to manual
Partition Key	Specifies the hash value when Partition Type is set to hash
Request Timeout	Timeout period in seconds
Output Format	Format for log content output, supporting single-line and JSON format output
Formatting Configuration	Output format template, configuration tutorial can be found here

Sample Formatting Configuration

{
   "fields": [
      "$time_iso8601",
      "$request_id",
      "@request",
      "@proxy",
      "@response",
      "@status_code",
      "@time"
   ],
   "request": [
      "$request_method",
      "$scheme",
      "$request_uri",
      "$host",
      "$header",
      "$remote_addr"
   ],
   "proxy": [
      "$proxy_method",
      "$proxy_scheme",
      "$proxy_uri",
      "$proxy_host",
      "$proxy_header",
      "$proxy_addr"
   ],
   "response": [
      "$response_header"
   ],
   "status_code": [
      "$status",
      "$proxy_status"
   ],
   "time": [
      "$request_time",
      "$response_time"
   ]
}

Go Live

1. Click the Go Live button next to the configuration that is ready to be deployed.