3.3. Algorithms

As mentioned in Section 2.5, an algorithm is registered with the framework as an operator to a higher-order function (HOF). In general, Phlex supports the registration of C++ algorithms with function signatures like (see Section 3.5 for a list of supported HOFs):

return_type function_name(P1, Pn..., Rm...) [qualifiers];

where the types P1, Pn... denote types of data products and the types Rm... indicate resources. Each registered function must accept at least one data product.

The signature of a Python algorithm needs to be available through reflection, either because the function is JITed (e.g. with Numba), bound (e.g. with ctypes), or annotated. The latter is good practice regardless and commonly required by Python coding conventions:

def function_name(p1: P1, pn: Pn..., rm: Rm) -> return_type:

We will first discuss the data-product and resource types in Section 3.3.1, followed by the return types in Section 3.3.2, and then the function name and optional qualifers in Section 3.3.3.

3.3.1. Input Parameters

A data product of type P may be presented to a C++ algorithm if the corresponding input parameter (i.e. the relevant P1, ..., PN type) is one of the following:

  • P const& — read-only access to a data product provided through a reference

  • P const* — read-only access to a data product provided through a pointer

  • P — the data product is copied into an object (assumes data product is copyable) [1]

  • phlex::handle<P> — a lightweight object that provides read-only access to a data product as well as any metadata associated with it

For each of these cases, the data product itself remains immutable. A Python algorithm can receive a phlex::handle or a direct reference to the data product. There is no equivalent language support for read-only access, but it will be enforced where possible.

Whereas data products may to be copied, resources of type R may not. The following types are therefore supported:

  • R const& — read-only access to a resource provided through a reference

  • R const* — read-only access to a resource provided through a pointer

Resources are described in more detail in Section 3.9.

3.3.2. Return Types

The meaning of an algorithm’s return type depends on the HOF and is discussed in the Section 3.5. However, to simplify the discussion we introduce to concept of the created data-product type. For Phlex to appropriately schedule the execution of algorithms and manage the lifetimes of data products, the framework itself must retain ownership of the data products. This means that the data products created by algorithms must have types that connote unique ownership. An algorithm’s returned object must therefore model a created data-product type, which can be:

  • a value of type T, or

  • a std::unique_ptr<T>, where the created object is non-null.

For Python, this means that an algorithm should not retain any external hard references to a returned object.

The following types (or their equivalents) are forbidden as created data-product types because they do not imply unambiguous ownership:

  • bare pointer types, such as T* or T const*

  • reference types, such as T& or T const&

3.3.3. Function Names and Qualifiers

The function_name in Section 3.3 above may be any function name supported by the C++ language. Code authors should aim to implement algorithms as free functions. However, in some cases it may be necessary for class member functions to be used instead. When member functions are required, the qualifier const should be specified to indicate that the class instance remains immutable during the execution of the member function [2].

Footnotes

References