Guide to Hooks for the Kea Component Developer

Introduction

The hooks framework is a Kea system that simplifies the way that users can write code to modify the behavior of Kea. Instead of altering the Kea source code, they write functions that are compiled and linked into one or more dynamic shared objects, called here (for historical reasons), shared libraries. The library is specified in the Kea configuration and at run time Kea dynamically loads the library into its address space. At various points in the processing, the component "calls out" to functions in the library, passing to them the data is it currently working on. They can examine and modify the data as required.

This guide is aimed at Kea developers who want to write or modify a Kea component to use hooks. It shows how the component should be written to load a shared library at run-time and how to call functions in it.

For information about writing a hooks library containing functions called by Kea during its execution, see the document Hooks Developer's Guide.

Terminology

In the remainder of this guide, the following terminology is used:

• Component - a Kea process, e.g. the DHCPv4 or DHCPv6 server.
• Hook/Hook Point - used interchangeably, this is a point in the code at which a call to user-written functions is made. Each hook has a name and each hook can have any number (including 0) of user-written functions attached to it.
• Callout - a user-written function called by the component at a hook point. This is so-named because the component "calls out" to the library to execute a user-written function.
• User code/user library - non-Kea code that is compiled into a shared library and loaded by Kea into its address space. Multiple user libraries can be loaded at the same time, each containing callouts for the same hooks. The hooks framework calls these libraries one after the other. (See the document Hooks Developer's Guide for more details.)

Languages

The core of Kea is written in C++ with some remaining legacy parts in Python. While it is the intention to provide the hooks framework for all languages, the initial version is for C++. All examples in this guide are in that language.

Basic Ideas

From the point of view of the component author, the basic ideas of the hooks framework are quite simple:

• The location of hook points in the code need to be determined.
• Name the hook points and register them.
• At each hook point, the component needs to complete the following steps to execute callouts registered by the user-library:
  1. copy data into the object used to pass information to the callout.
  2. call the callout.
  3. copy data back from the object used to exchange information.
  4. take action based on information returned.

Of course, to set up the system the libraries need to be loaded in the first place. The component also needs to:

• Define the configuration item that specifies the user libraries for this component.
• Handle configuration changes and load/unload the user libraries.

The following sections will describe these tasks in more detail.

Determining the Hook Points

Before any other action takes place, the location of the hook points in the code need to be determined. This, of course, depends on the component but, as a general guideline, hook locations should be located where a callout is able to obtain useful information from Kea and/or affect processing. Typically this means at the start or end of a major step in the processing of a request, at a point where either useful information can be passed to a callout and/or the callout can affect the processing of the component. The latter is achieved in either or both of the following ways:

• Setting the nest step status. This is an enum that the callout can set and is a quick way of passing information back to the component. It is used to indicate that the component should perform certain actions. Currently there are three statuses defined: CONTINUE (this is the default, the server is expected to continue as usual), SKIP (the server is expected to skip the next processing step, but otherwise continue as usual) and DROP (the server is expected to drop the packet or request completely. The exact action is up to the component.
• Modifying data passed to it. The component should be prepared to continue processing with the data returned by the callout. It is up to the component author whether the data is validated before being used, but doing so will have performance implications.

Naming and Registering the Hooks Points

Once the location of the hook point has been determined, it should be given a name. This name should be unique amongst all hook points and is subject to certain restrictions (see below).

Before the callouts at any hook point are called and any user libraries loaded - so typically during component initialization - the component must register the names of all the hooks. The registration is done using the static method isc::hooks::HooksManager::registerHook():

#include <hooks/hooks_manager.h>
    :
    int example_index = HooksManager::registerHook("lease_allocate");

The name of the hook is passed as the sole argument to the registerHook() method. The value returned is the index of that hook point and should be retained - it is needed to call the callouts attached to that hook.

Note that a hook only needs to be registered once. There is no mechanism for unregistering a hook and there is no need to do so.

Automatic Registration of Hooks

In some components, it may be convenient to set up a single initialization function that registers all hooks. For others, it may be more convenient for each module within the component to perform its own initialization. Since the isc::hooks::HooksManager object is a singleton and is created when first accessed, a useful trick is to automatically register the hooks when the module is loaded.

This technique involves declaring an object outside of any execution unit in the module. When the module is loaded, the object's constructor is run. By placing the hook registration calls in the constructor, the hooks in the module are defined at load time, before any function in the module is run. The code for such an initialization sequence would be similar to:

#include <hooks/hooks_manager.h>
 
namespace {
 
// Declare structure to perform initialization and store the hook indexes.
//
struct MyHooks {
    int pkt_rcvd;   // Index of "packet received" hook
    int pkt_sent;   // Index of "packet sent" hook
 
    // Constructor
    MyHooks() {
        pkt_rcvd = HooksManager::registerHook("pkt_rcvd");
        pkt_sent = HooksManager::registerHook("pkt_sent");
    }
};
 
// Declare a "MyHooks" object.  As this is outside any function or method, it
// will be instantiated (and the constructor run) when the module is loaded.
// As a result, the hook indexes will be defined before any method in this
// module is called.
MyHooks my_hooks;
 
} // Anonymous namespace
 
void Someclass::someFunction() {
    :
    // Check if any callouts are defined on the pkt_rcvd hook.
    if (HooksManager::calloutPresent(my_hooks.pkt_rcvd)) {
          :
    }
    :
}

Hook Names

Hook names are strings and in principle, any string can be used as the name of a hook, even one containing spaces and non-printable characters. However, the following guidelines should be observed:

• The names context_create and context_destroy are reserved to the hooks system and are automatically registered: an attempt to register one of these will lead to a isc::hooks::DuplicateHook exception being thrown.
• The hook name should be a valid "C" function name. If a user gives a callout the same name as one of the hooks, the hooks framework will automatically load that callout and attach it to the hook: the user does not have to explicitly register it.
• The hook name should not conflict with the name of a function in any of the system libraries (e.g. naming a hook "sqrt" could lead to the square-root function in the system's maths library being attached to the hook as a callout).
• Although hook names can be in any case (including mixed case), the Kea convention is that they are lower-case.

Calling Callouts on a Hook

The Callout Handle

Before describing how to call user code at a hook point, we must first consider how to pass data to it.

Each user callout has the signature:

int callout_name(isc::hooks::CalloutHandle& handle);

The isc::hooks::CalloutHandle object is the object used to pass data to and from the callout. This holds the data as a set of name/value pairs, each pair being considered an argument to the callout. If there are multiple callouts attached to a hook, the CalloutHandle is passed to each in turn. Should a callout modify an argument, the updated data is passed subsequent callouts (each of which could also modify it) before being returned to the component.

Two methods are provided to get and set the arguments passed to the callout called (naturally enough) getArgument and setArgument. Their usage is illustrated by the following code snippets.

int count = 10;
boost::shared_ptr<Pkt4> pktptr = ... // Set to appropriate value
 
// Assume that "handle_ptr" has been created and is a pointer to a
// CalloutHandle.
handle_ptr->setArgument("data_count", count);
handle_ptr->setArgument("inpacket", pktptr);
 
// Call the hook code.  lease_assigned_index is the value returned from
// HooksManager::registerHook() when the hook was registered.
HooksManager::callCallouts(lease_assigned_index, *handle_ptr);
 
// Retrieve the modified values
handle_ptr->getArgument("data_count", count);
handle_ptr->getArgument("inpacket", pktptr);

As can be seen getArgument is used to retrieve data from the CalloutHandle, and setArgument used to put data into it. If a callout wishes to alter data and pass it back to the component, it should retrieve the data with getArgument, modify it, and call setArgument to send it back.

There are a couple points to be aware of:

• The data type of the variable in the call to getArgument must match the data type of the variable passed to the corresponding setArgument exactly: using what would normally be considered to be a "compatible" type is not enough. For example, if the callout passed an argument back to the component as an int and the component attempted to retrieve it as a long, an exception would be thrown even though any value that can be stored in an int will fit into a long. This restriction also applies the "const" attribute but only as applied to data pointed to by pointers, e.g. if an argument is defined as a char*, an exception will be thrown if an attempt is made to retrieve it into a variable of type const char*. (However, if an argument is set as a const int, it can be retrieved into an int.) The documentation of a hook point should detail the exact data type of each argument.
• If a pointer to an object is passed to a callout (either a "raw" pointer, or a boost smart pointer (as in the example above), and the underlying object is altered through that pointer, the change will be reflected in the component even if the callout makes no call to setArgument. This can be avoided by passing a pointer to a "const" object.

The Skip Flag (obsolete)

The next step status

Although information is passed back to the component from callouts through CalloutHandle arguments, a common action for callouts is to inform the component that its flow of control should be altered. For example:

• In the DHCP servers, there is a hook at the point at which a lease is about to be assigned. Callouts attached to this hooks may handle the lease assignment in special cases, in which case they set the next step status to SKIP to indicate that the server should not perform lease assignment in this case.
• A server may define a hook just after a packet is received. A callout attached to the hook might inspect the source address and compare it against a blacklist. If the address is on the list, the callout could set the DROP flag to indicate to the server that the packet should be dropped.

For ease of processing, the CalloutHandle contains two methods, isc::hooks::CalloutHandle::getStatus() and isc::hooks::CalloutHandle::setStatus(). It is only meaningful for the component to use the "get" method. The next step status is cleared (set to the default value of CONTINUE) by the hooks framework when the component requests that callouts be executed, so any value set by the component is lost. Callouts can both inspect the status (it might have been set by callouts earlier in the callout list for the hook) and set it. Note that the setting of the status by a callout does not prevent callouts later in the list from being called: the next step status is just an enum value - the only significance comes from its interpretation by the component.

An example of use could be:

// Set up arguments for DHCP lease assignment.
handle->setArgument("query", query);
handle->setArgument("response", response);
HooksManager::callCallouts(lease_hook_index, *handle_ptr);
if (handle_ptr->getStatus() != CalloutHandle::NEXT_STEP_SKIP) {
    // Skip flag not set, do the address allocation
    :
}

Getting the Callout Handle

The CalloutHandle object is linked to the loaded libraries for lifetime reasons (described below). Components should retrieve a isc::hooks::CalloutHandle using isc::hooks::HooksManager::createCalloutHandle():

CalloutHandlePtr handle_ptr = HooksManager::createCalloutHandle();

(isc::hooks::CalloutHandlePtr is a typedef for a Boost shared pointer to a CalloutHandle.) The CalloutHandle so retrieved may be used for as long as the libraries are loaded.

The handle is deleted by resetting the pointer:

handle_ptr.reset();

... or by letting the handle pointer go out of scope. The actual deletion occurs when the CallHandle's reference count goes to zero. (The current version of the hooks framework does not maintain any other pointer to the returned CalloutHandle, so it gets destroyed when the shared pointer to it is cleared or destroyed. However, this may change in a future version.)

When the handle is not created locally it is not destroyed so it can keep ownership on arguments. In such case the code must call isc::hooks::CalloutHandle::deleteAllArguments or simply use the RAII helper isc::hooks::ScopedCalloutHandleState as in:

CalloutHandlePtr handle_ptr = getCalloutHandle(query);
ScopedCalloutHandleState state(handle_ptr);

Calling the Callout

Calling the callout is a simple matter of executing the isc::hooks::HooksManager::callCallouts() method for the hook index in question. For example, with the hook index "pkt_sent" defined as above, the hook can be executed by:

HooksManager::callCallouts(pkt_sent, *handle_ptr);

... where "*handle_ptr" is a reference (note: not a pointer) to the isc::hooks::CalloutHandle object holding the arguments. No status code is returned. If a component needs to get data returned (other than that provided by the next step status), it should define an argument through which the callout can do so.

Conditionally Calling Hook Callouts

Most hooks in a component will not have callouts attached to them. To avoid the overhead of setting up arguments in the CalloutHandle, a component can check for callouts before doing that processing using isc::hooks::HooksManager::calloutsPresent(). Taking the index of a hook as its sole argument, the function returns true if there are any callouts attached to the hook and false otherwise.

With this check, the code in the component for calling a hook would look something like:

if (HooksManager::calloutsPresent(lease_hook_index)) {
    // Set up arguments for lease assignment
    handle->setArgument("query", query);
    handle->setArgument("response", response);
    HooksManager::callCallouts(lease_hook_index, *handle);
    if (handle->getStatus() != CalloutHandle::NEXT_STEP_DROP) {
        // Next step allows us to continue, do the address allocation
        :
    }
}

Loading the User Libraries

Once hooks are defined, all the hooks code described above will work, even if no libraries are loaded (and even if the library loading method is not called). The CalloutHandle returned by isc::hooks::HooksManager::createCalloutHandle() will be valid, isc::hooks::HooksManager::calloutsPresent() will return false for every index, and isc::hooks::HooksManager::callCallouts() will be a no-op.

However, if user libraries are specified in the Kea configuration, the component should load them. (Note the term "libraries": the hooks framework allows multiple user libraries to be loaded.) This should take place after the component's configuration has been read, and is achieved by the isc::hooks::HooksManager::loadLibraries() method. The method is passed a vector of strings, each giving the full file specification of a user library:

std::vector<std::string> libraries = ... // Get array of libraries
bool success = HooksManager::loadLibraries(libraries);

loadLibraries() returns a boolean status which is true if all libraries loaded successfully or false if one or more failed to load. Appropriate error messages will have been logged in the latter case, the status being more to allow the developer to decide whether the execution should proceed in such circumstances.

Before loadLibraries() can be called a second or subsequent time (as a result of a reconfiguration), all existing libraries must be successfully unloaded. If a library stays in memory from a programming bug in Kea (for instance when no libraries were loaded) or in a library (Memory Management Considerations for Hooks Writer) loadLibraries() throws a not recoverable error.

Unloading is done in two phases since Kea version 1.7.10:

• call to isc::hooks::HooksManager::prepareUnloadLibraries() which calls all unload() entry points and deregisters callout points.
• call to isc::hooks::HooksManager::unloadLibraries() even when the prepare failed.

If a failure of unloadLibraries() is ignored any call to loadLibraries() will throw.

Unload and Reload Issues

Unloading a shared library works by unmapping the part of the process's virtual address space in which the library lies. This may lead to problems if there are still references to that address space elsewhere in the process.

In many operating systems, heap storage allowed by a shared library will lie in the virtual address allocated to the library. This has implications in the hooks framework because:

• Argument information stored in a CalloutHandle by a callout in a library may lie in the library's address space.
• Data modified in objects passed as arguments may lie in the address space. For example, it is common for a DHCP callout to add "options" to a packet: the memory allocated for those options will most likely lie in library address space.

The problem really arises because of the extensive use by Kea of boost smart pointers. When the pointer is destroyed, the pointed-to memory is deallocated. If the pointer points to address space that is unmapped because a library has been unloaded, the deletion causes a segmentation fault.

The hooks framework addresses the issue for CalloutHandles by keeping in that object a shared pointer to the object controlling library unloading. Although a library can be unloaded at any time, it is only when all CalloutHandles that could possibly reference address space in the library have been deleted that the library will actually be unloaded and the address space unmapped.

The hooks framework cannot solve the second issue as the objects in question are under control of the component. It is up to the component developer to ensure that all such objects have been destroyed before libraries are reloaded. In extreme cases this may mean the component suspending all processing of incoming requests until all currently executing requests have completed and data object destroyed, reloading the libraries, then resuming processing.

Since Kea 1.7.10 the unload() entry point is called as the first phase of unloading. This gives more chance to hooks writer to perform necessary cleanup actions so the second phase, memory unmapping can safely happen. The isc::hooks::unloadLibraries() function was updated too to return false when at least one active callout handle remained.

Component-Defined Callouts

Previous sections have discussed callout registration by user libraries. It is possible for a component to register its own functions (i.e. within its own address space) as hook callouts. These functions are called in exactly the same way as user callouts, being passed their arguments though a CalloutHandle object. (Guidelines for writing callouts can be found in Hooks Developer's Guide.)

A component can associate with a hook callouts that run either before user-registered callouts or after them. Registration is done via a isc::hooks::LibraryHandle object, a reference to one being obtained through the methods isc::hooks::HooksManager::preCalloutLibraryHandle() (for a handle to register callouts to run before the user library callouts) or isc::hooks::HooksManager::postCalloutLibraryHandle() (for a handle to register callouts to run after the user callouts). Use of the LibraryHandle to register and deregister callouts is described in The LibraryHandle Object.

Finally, it should be noted that callouts registered in this way only remain registered until the next call to isc::hooks::loadLibraries(). It is up to the component to re-register the callouts after this method has been called.