 |
Kea 3.1.1
|
|
Kea 3.1.1
|
Welcome to Kea Legal Log Hooks Library. This documentation is addressed at developers who are interested in internal operation of the libdhcp_legal_log library. This file provides information needed to understand and perhaps extend this library.
This documentation is stand-alone: you should have read and understood Kea Developer's Guide and in particular its section about hooks: Hooks Developer's Guide.
libdhcp_legal_log is a hooks library which customizes the DHCP lease management provided by Kea DHCP server modules (kea-dhcp4 and kea-dhcp6). Specifically it implements recording a detailed log of lease assignments and renewals as might be required for legal purposes. It is also described in the Kea Administrator Reference Manual instead in the section titled Forensic Logging Hooks.
This library is designed to help with the requirement that in many legal jurisdictions companies, especially ISPs, must record information about the addresses they have leased to DHCP clients. If the information that it records is sufficient it may be used directly. If your jurisdiction requires that you save a different set of information you may use it as a template or example and create your own custom logging hooks.
Legal logs are appended into a text file implemented by the class isc::legal_log::RotatingFile. By default, it is rotated daily, but this can be configured via the time-unit and count parameters.
Additional actions can be performed just before closing the old file and after opening the new file. These actions must point to an external executable or script and are configured by setting prerotate and postrotate parameters. Kea will instantiate the prerotate action with the name of the file that will be closed, but does not wait for the process to finish. Kea will instantiate the postrotate action with the name of the file that had been opened, but does not wait for the process to finish.
Legal file names, if using day, month or year as time unit have the form:
<path>/<base-name>.<CCYYMMDD>.txt
where CC represents century, YY represents current year, MM represents current month and DD represents current day.
Legal file names, if using second as time unit have the form:
<path>/<base-name>.<TXXXXXXXXXXXXXXXXXXXX>.txt
where XXXXXXXXXXXXXXXXXXXX represents time in seconds since epoch.
To note that when using second as time unit, the file will be rotated when the count number of seconds pass. In contrast, when using day, month or year as time unit, the file will be rotated whenever the countth day, month or year starts respectively.
For DHCPv4, the library creates entries based on DHCPREQUEST, DHCPDECLINE, DHCPRELEASE messages et.al. and their responses. The resulting packets and leases are taken into account, intercepted through the following hook points: pkt4_receive leases4_committed pkt4_send lease4_release lease4_decline This is necessary to get the subnet logging information. Since Kea 1.9.8, the logging action is performed only on the pkt4_send hook (for request and renew replies), on the lease4_release hook (for release replies) and on the lease4_decline hook (for declined replies). This guarantees that nothing is logged if the server or any other hook decides to drop the packet between lease update and packet send operations.
An entry is a single string with no embedded end-of-line markers, a prepended timestamp and has the following sections:
Where:
For instance:
2018-01-06 01:02:03 CET Address: 192.2.1.100 has been renewed for 1 hrs 52 min 15 secs to a device with hardware address: hwtype=1 08:00:2b:02:3f:4e, client-id: 17:34:e2:ff:09:92:54 connected via relay at address: 192.2.16.33, identified by circuit-id: 68:6f:77:64:79 (howdy) and remote-id: 87:f6:79:77:ef
or for a release:
2018-01-06 01:02:03 CET Address: 192.2.1.100 has been released from a device with hardware address: hwtype=1 08:00:2b:02:3f:4e, client-id: 17:34:e2:ff:09:92:54 connected via relay at address: 192.2.16.33, identified by circuit-id: 68:6f:77:64:79 (howdy) and remote-id: 87:f6:79:77:ef
The request-parser-format and response-parser-format can be used to extract and log data from the incoming packet and server response packet respectively. The configured value is an evaluated parsed expression returning a string. This can be used to generate custom log entries. Use with caution as this might affect server performance. If any of them is configured, the default logging format is not used. If both of them are configured, the resulting log message is constructed by concatenating the logged data extracted from the request and the logged data extracted from the response.
The timestamp-format parameter can be used to change the timestamp logged at the beginning of each line. Permissible formatting is the one supported by strftime plus the 'Q' extra format which adds the microseconds subunits. The default is: "%Y-%m-%d %H:%M:%S %Z". This parameter has no effect for the database backends, where the timestamp is defined at the schema level.
In addition to logging lease activity driven by DHCPv4 client traffic, it also logs entries for the following lease management control channel commands: lease4-add, lease4-update, and lease4-del. Each entry is a single string with no embedded end-of-line markers and they will typically have the following forms:
lease4-add:
Dependent on the arguments of the add command, it may also include the client-id and duration.
Example:
2018-01-06 01:02:03 CET Administrator added a lease of address: 192.0.2.202 to a device with hardware address: 1a:1b:1c:1d:1e:1f for 1 days 0 hrs 0 mins 0 secs
lease4-update:
Dependent on the arguments of the update command, it may also include the client-id and lease duration.
Example:
2018-01-06 01:02:03 CET Administrator updated information on the lease of address: 192.0.2.202 to a device with hardware address: 1a:1b:1c:1d:1e:1f, client-id: 1234567890
lease4-del: Deletes have two forms, one by address and one by identifier and identifier type:
or
Currently only a type of hw-address (hardware address) is supported.
Examples:
2018-01-06 01:02:03 CET Administrator deleted the lease for address: 192.0.2.202 2018-01-06 01:02:12 CET Administrator deleted a lease for a device identified by: hw-address of 1a:1b:1c:1d:1e:1f
If High availability module is enabled, the partner will periodically send lease commands which have a similar format, the only difference is that the issuer of the command is 'HA partner' instead of 'Administrator'.
Example:
*timestamp* HA partner added ...
or
*timestamp* HA partner updated ...
or
*timestamp* HA partner deleted ...
For DHCPv6, the library creates entries based on REQUEST, RENEW, RELEASE, DECLINE messages et.al. and their responses. The resulting packets and leases are taken into account, intercepted through the following hook points: pkt6_receive leases6_committed pkt6_send lease6_release lease6_decline This is necessary to get the subnet logging information. Since Kea 1.9.8, the logging action is performed only on the pkt6_send hook (for request, renew, rebind, release and declined replies). This guarantees that nothing is logged if the server or any other hook decides to drop the packet between lease update and packet send operations.
Note:
When using custom log format, the lease address and other parameters must be extracted from the information stored in the client request or server response. In the case of IPv6, the packets can contain multiple IA_NA (3) or IA_PD (25) options, each containing multiple options, including OPTION_IAADDR (5) or OPTION_IAPREFIX (25) suboptions. To be able to print the current lease associated with the log entry, the forensic log hook library internally isolates the corresponding IA_NA or IA_PD option and respective suboptions matching the current lease. The hook library will iterate over all new allocated addresses (stored in the leases6 hook argument/context) and all deleted addresses (stored in the deleted_lease6 hook argument/context), making each address available for logging as the current lease for the respective logged entry. It is recommended to extract the data from the server response (if available) because the server can allocate other leases if the ones requested by the client are not available. In this case the server must respond and update lifetimes to 0 for such leases, and they can be filtered out by the parser expression.
An entry is a single string with no embedded end-of-line markers, a prepended timestamp and has the following sections:
Where:
For instance:
2018-01-06 01:02:03 PST Address:2001:db8:1:: has been assigned for 0 hrs 11 mins 53 secs to a device with DUID: 17:34:e2:ff:09:92:54 and hardware address: hwtype=1 08:00:2b:02:3f:4e (from Raw Socket) connected via relay at address: fe80::abcd for client on link address: 3001::1, hop count: 1, identified by remote-id: 01:02:03:04:0a:0b:0c:0d:0e:0f and subscriber-id: 1a:2b:3c:4d:5e:6f
or for a release:
2018-01-06 01:02:03 PST Address:2001:db8:1:: has been released from a device with DUID: 17:34:e2:ff:09:92:54 and hardware address: hwtype=1 08:00:2b:02:3f:4e (from Raw Socket) connected via relay at address: fe80::abcd for client on link address: 3001::1, hop count: 1, identified by remote-id: 01:02:03:04:0a:0b:0c:0d:0e:0f and subscriber-id: 1a:2b:3c:4d:5e:6f
In addition to logging lease activity driven by DHCPv6 client traffic, it also logs entries for the following lease management control channel commands: lease6-add, lease6-update, and lease6-del. Each entry is a single string with no embedded end-of-line markers and they will typically have the following forms:
lease6-add:
Dependent on the arguments of the add command, it may also include the hardware address and duration.
Example:
2018-01-06 01:02:03 PST Administrator added a lease of address: 2001:db8::3 to a device with DUID: 1a:1b:1c:1d:1e:1f:20:21:22:23:24 for 1 days 0 hrs 0 mins 0 secs
lease6-update:
Dependent on the arguments of the update command, it may also include the hardware address and lease duration.
Example:
2018-01-06 01:02:03 PST Administrator updated information on the lease of address: 2001:db8::3 to a device with DUID: 1a:1b:1c:1d:1e:1f:20:21:22:23:24, hardware address: 1a:1b:1c:1d:1e:1f
lease6-del: Deletes have two forms, one by address and one by identifier and identifier type:
or
Currently only a type of DUID is supported.
Examples:
2018-01-06 01:02:03 PST Administrator deleted the lease for address: 2001:db8::3 2018-01-06 01:02:11 PST Administrator deleted a lease for a device identified by: duid of 1a:1b:1c:1d:1e:1f:20:21:22:23:24
If High availability module is enabled, the partner will periodically send lease commands which have a similar format, the only difference is that the issuer of the command is 'HA partner' instead of 'Administrator'.
Example:
*timestamp* HA partner added ...
or
*timestamp* HA partner updated ...
or
*timestamp* HA partner deleted ...
It must be configured as a hook library for the desired DHCP server modules. Note that the legal_log library is installed alongside the Kea libraries in "<install-dir>/lib" where <install-dir> is determined by the –prefix meson setup option. It defaults to "/usr/local". Assuming the default value then, configuring kea-dhcp4 to load the legal_log library could be done with the following Kea4 configuration:
To configure it for kea-dhcp6, the commands are simply as shown below:
Two string Hook Library Parameters are supported:
When running Kea servers for both DHCPv4 and DHCPv6 the log names must be distinct, either the path or the base-name (or both) should be different for DHCP4 vs DHCP6.
Some countries require ISPs to log the connection meta-data for legal purposes. A legal log hook library demonstrating how to log this information is then useful as both an example explaining how to program with hooks and for direct use.
As explained in the tutorial of the Hook Developer's Guide Tutorial a hook library is a Dynamic Shared Object with known entry points for the framework functions and hooks. The DSO is (when not static linked) dynamically loaded by a Kea server module. Some parameters can be passed to configure it and DSO entry points with a well known hook names are auto-magically registered.
The Kea server module invokes the registered callouts during processing and passes some context arguments. The returned status is used to control further processing, e.g., by dropping a message to cancel the current action.
The file version.cc defines the required framework version() function which protects against Hooks API changes. The file load_unload.cc defines a load() function which processes the configuration parameters and creates and opens the legal log file (implemented by class isc::legal_log::RotatingFile in rotating_file.h and rotating_file.cc). The unload() function destroys the object which by side effect closes the legal log file. Note the load() function does not register callouts nor a logger: the Hooks Library Manager takes care of this (triggered for the logger by legal_log_log.h, legal_log_log.cc and legal_log_messages.h files).
For DHCPv4 pkt4_receive(), leases4_committed(), pkt4_send(), lease4_release() and lease4_decline() hook callouts are defined in lease4_callouts.cc file. For DHCPv6 pkt6_receive(), leases6_committed(), pkt6_send(), lease6_release() and lease6_decline() hook callouts are defined in lease6_callouts.cc file.
The legalLog4Handler() or legalLog6Handler() is called with the callout handle (to get the parsed query message and the lease) and an action type which can either be ASSIGN or RELEASE. This requires the receive packet callouts (pkt4_receive and pkt6_receive) to initialize an empty context which can be used in all following hooks.
When something goes wrong callouts return 1 (next step skip) which tells the server to skip processing the next step. So in the server any legal log error is converted to an internal error and the client is denied service.
The hook callout, command_processed() is defined to log lease management control channel commands (lease<4/6>-add, -update, -del). Entries are only generated for commands that execute successfully.
For DHCPv4 the only difficulty is to get the relay agent information option and its two interesting (i.e., to log) circuit ID and remote ID sub-options for any queries that were relayed.
For DHCPv6 the hardware address is not built-in but can be recovered and recorded by kea-dhcp6 in many cases. A DHCPv6 query can be relayed more than once but the current code logs only the first (closest to the client) relay information with first found remote ID and subscriber ID relay options.
The lease client context can only be printed using the default format, as this information is not directly stored in the incoming packet or in the server response packet.
As an alternative to files a database backend is available. The code is based on the lease backend with an automatic timestamp and log text columns in a new logs table.
The configuration reuses the lease database one with for backward (i.e. file) compatibility either no "type" entry or "logfile" value.
Since Kea 1.9.6, the libdhcp_legal_log hooks library supports database connection recovery which can be enabled by setting the on-fail parameter. If not specified, the on-fail parameter defaults to serve-retry-continue as opposed to the case of lease manager, host manager and config backend where it defaults to stop-retry-exit. In this case, the server will continue serving clients and it will not shut down even if the recovery mechanism fails. If the on-fail is set to serve-retry-exit, the server will shut down if the connection to the database backend is not restored according to the max-reconnect-tries and reconnect-wait-time parameters, but it will continue serving clients while this mechanism is activated. During server startup, the inability to connect to any of the configured backends is considered fatal only if retry-on-startup is set to false (the default). A fatal error is logged and the server exits, based on the idea that the configuration should be valid at startup. Exiting to the operating system allows nanny scripts to detect the problem. If retry-on-startup is set to true, the server will start reconnection attempts even at server startup or on reconfigure events, and will honor the action specified in the on-fail parameter. The retry-on-startup parameter is supported since Kea 2.5.5.
Design notes:
Log entries can be inserted into syslog by setting the "type" to "syslog". When syslog type is configured, the "pattern" parameter specifies the details that are used for logging. If not configured, it defaults to: "%-5p [%c.%t] %m\n".
The "facility" parameter specifies the syslog facility and it defaults to "local0".
The libdhcp_legal_log hooks library is compatible with multi-threading.
1.13.2