Hwloc C API¶

This page provides the reference of the hwloc C interface. The Python package is built on top of this interface. We created this page for reference purposes only and don’t aim to replace the original doxygen document.

Error reporting in the API¶

group Error reporting in the API

Most functions in the hwloc API return an integer value. Unless documentated differently, they return 0 on success and -1 on error. Functions that return a pointer type return NULL on error.

errno will be set to a meaningful value whenever possible. This includes the usual EINVAL when invalid function parameters are passed or ENOMEM when an internal allocation fails. Some specific errno value are also used, for instance for binding errors as documented in CPU binding.

Some modules describe return values of their functions in their introduction, for instance in The bitmap API.

API version¶

group API version

Defines

HWLOC_API_VERSION¶

Indicate at build time which hwloc API version is being used.

This number is updated to (X<<16)+(Y<<8)+Z when a new release X.Y.Z actually modifies the API.

Users may check for available features at build time using this number (see How do I handle API changes?).

Note

This should not be confused with HWLOC_VERSION, the library version. Two stable releases of the same series usually have the same HWLOC_API_VERSION even if their HWLOC_VERSION are different.

HWLOC_COMPONENT_ABI¶: Current component and plugin ABI version (see hwloc/plugins.h).

Functions

unsigned hwloc_get_api_version(void)¶

Indicate at runtime which hwloc API version was used at build time.

Should be HWLOC_API_VERSION if running on the same version.

Returns:: the build-time version number.

Object Sets (hwloc_cpuset_t and hwloc_nodeset_t)¶

group Object Sets (hwloc_cpuset_t and hwloc_nodeset_t)

Hwloc uses bitmaps to represent two distinct kinds of object sets: CPU sets (hwloc_cpuset_t) and NUMA node sets (hwloc_nodeset_t). These types are both typedefs to a common back end type (hwloc_bitmap_t), and therefore all the hwloc bitmap functions are applicable to both hwloc_cpuset_t and hwloc_nodeset_t (see The bitmap API).

The rationale for having two different types is that even though the actions one wants to perform on these types are the same (e.g., enable and disable individual items in the set/mask), they’re used in very different contexts: one for specifying which processors to use and one for specifying which NUMA nodes to use. Hence, the name difference is really just to reflect the intent of where the type is used.

Typedefs

typedef hwloc_bitmap_t hwloc_cpuset_t¶

A CPU set is a bitmap whose bits are set according to CPU physical OS indexes.

It may be consulted and modified with the bitmap API as any hwloc_bitmap_t (see hwloc/bitmap.h).

Each bit may be converted into a PU object using hwloc_get_pu_obj_by_os_index().

See also

faq_indexes

typedef hwloc_const_bitmap_t hwloc_const_cpuset_t¶: A non-modifiable hwloc_cpuset_t.

typedef hwloc_bitmap_t hwloc_nodeset_t¶

A node set is a bitmap whose bits are set according to NUMA memory node physical OS indexes.

It may be consulted and modified with the bitmap API as any hwloc_bitmap_t (see hwloc/bitmap.h). Each bit may be converted into a NUMA node object using hwloc_get_numanode_obj_by_os_index()

.

When binding memory on a system without any NUMA node, the single main memory bank is considered as NUMA node #0.

See also

faq_indexes

typedef hwloc_const_bitmap_t hwloc_const_nodeset_t¶: A non-modifiable hwloc_nodeset_t.

Object Types¶

group Object Types

Defines

HWLOC_TYPE_UNORDERED¶: Value returned by hwloc_compare_types() when types can not be compared.

Typedefs

typedef enum hwloc_obj_cache_type_e hwloc_obj_cache_type_t¶: Cache type.

typedef enum hwloc_obj_bridge_type_e hwloc_obj_bridge_type_t¶: Type of one side (upstream or downstream) of an I/O bridge.

typedef unsigned long hwloc_obj_osdev_types_t¶: Type(s) of a OS device. OR’ed set of hwloc_obj_osdev_type_e.

Enums

enum hwloc_obj_type_t¶

Type of topology object.

Note

Do not rely on the ordering or completeness of the values as new ones may be defined in the future! If you need to compare types, use hwloc_compare_types() instead.

Values:

enumerator HWLOC_OBJ_MACHINE¶

Machine. A set of processors and memory with cache coherency.

This type is always used for the root object of a topology, and never used anywhere else. Hence its parent is always NULL.

enumerator HWLOC_OBJ_PACKAGE¶: Physical package. The physical package that usually gets inserted into a socket on the motherboard. A processor package usually contains multiple cores, and possibly some dies.

enumerator HWLOC_OBJ_DIE¶

Die within a physical package. A subpart of the physical package, that contains multiple cores.

Some operating systems (e.g. Linux) may expose a single die per package even if the hardware does not support dies at all. To avoid showing such non-existing dies, hwloc will filter them out if all of them are identical to packages. This is functionally equivalent to HWLOC_TYPE_FILTER_KEEP_STRUCTURE being enforced for Dies versus Packages.

enumerator HWLOC_OBJ_CORE¶: Core. A computation unit (may be shared by several PUs, aka logical processors).

enumerator HWLOC_OBJ_PU¶

Processing Unit, or (Logical) Processor. An execution unit (may share a core with some other logical processors, e.g. in the case of an SMT core).

This is the smallest object representing CPU resources, it cannot have any child except Misc objects.

Objects of this kind are always reported and can thus be used as fallback when others are not.

enumerator HWLOC_OBJ_L1CACHE¶: Level 1 Data (or Unified) Cache.

enumerator HWLOC_OBJ_L2CACHE¶: Level 2 Data (or Unified) Cache.

enumerator HWLOC_OBJ_L3CACHE¶: Level 3 Data (or Unified) Cache.

enumerator HWLOC_OBJ_L4CACHE¶: Level 4 Data (or Unified) Cache.

enumerator HWLOC_OBJ_L5CACHE¶: Level 5 Data (or Unified) Cache.

enumerator HWLOC_OBJ_L1ICACHE¶: Level 1 instruction Cache (filtered out by default).

enumerator HWLOC_OBJ_L2ICACHE¶: Level 2 instruction Cache (filtered out by default).

enumerator HWLOC_OBJ_L3ICACHE¶: Level 3 instruction Cache (filtered out by default).

enumerator HWLOC_OBJ_GROUP¶

Group objects. Objects which do not fit in the above but are detected by hwloc and are useful to take into account for affinity. For instance, some operating systems expose their arbitrary processors aggregation this way. And hwloc may insert such objects to group NUMA nodes according to their distances. See also What are these Group objects in my topology?.

These objects are removed when they do not bring any structure (see HWLOC_TYPE_FILTER_KEEP_STRUCTURE).

enumerator HWLOC_OBJ_NUMANODE¶

NUMA node. An object that contains memory that is directly and byte-accessible to the host processors. It is usually close to some cores (the corresponding objects are descendants of the NUMA node object in the hwloc tree).

This is the smallest object representing Memory resources, it cannot have any child except Misc objects. However it may have Memory-side cache parents.

NUMA nodes may correspond to different kinds of memory (DRAM, HBM, CXL-DRAM, etc.). When hwloc is able to guess that kind, it is specified in the subtype field of the object. See also Normal attributes in the main documentation.

There is always at least one such object in the topology even if the machine is not NUMA.

Memory objects are not listed in the main children list, but rather in the dedicated Memory children list.

NUMA nodes have a special depth HWLOC_TYPE_DEPTH_NUMANODE instead of a normal depth just like other objects in the main tree.

enumerator HWLOC_OBJ_MEMCACHE¶

Memory-side cache (filtered out by default). A cache in front of a specific NUMA node.

This object always has at least one NUMA node as a memory child.

Memory objects are not listed in the main children list, but rather in the dedicated Memory children list.

Memory-side cache have a special depth HWLOC_TYPE_DEPTH_MEMCACHE instead of a normal depth just like other objects in the main tree.

enumerator HWLOC_OBJ_BRIDGE¶

Bridge (filtered out by default). Any bridge (or PCI switch) that connects the host or an I/O bus, to another I/O bus.

Bridges are not added to the topology unless their filtering is changed (see hwloc_topology_set_type_filter() and hwloc_topology_set_io_types_filter()).

I/O objects are not listed in the main children list, but rather in the dedicated io children list. I/O objects have NULL CPU and node sets.

enumerator HWLOC_OBJ_PCI_DEVICE¶

PCI device (filtered out by default).

PCI devices are not added to the topology unless their filtering is changed (see hwloc_topology_set_type_filter() and hwloc_topology_set_io_types_filter()).

I/O objects are not listed in the main children list, but rather in the dedicated io children list. I/O objects have NULL CPU and node sets.

enumerator HWLOC_OBJ_OS_DEVICE¶

Operating system device (filtered out by default).

OS devices are not added to the topology unless their filtering is changed (see hwloc_topology_set_type_filter() and hwloc_topology_set_io_types_filter()).

I/O objects are not listed in the main children list, but rather in the dedicated io children list. I/O objects have NULL CPU and node sets.

enumerator HWLOC_OBJ_MISC¶

Miscellaneous objects (filtered out by default). Objects without particular meaning, that can e.g. be added by the application for its own use, or by hwloc for miscellaneous objects such as MemoryModule (DIMMs).

They are not added to the topology unless their filtering is changed (see hwloc_topology_set_type_filter()).

These objects are not listed in the main children list, but rather in the dedicated misc children list. Misc objects may only have Misc objects as children, and those are in the dedicated misc children list as well. Misc objects have NULL CPU and node sets.

enumerator HWLOC_OBJ_TYPE_MAX¶: Sentinel value

enum hwloc_obj_cache_type_e¶

Cache type.

Values:

enumerator HWLOC_OBJ_CACHE_UNIFIED¶: Unified cache.

enumerator HWLOC_OBJ_CACHE_DATA¶: Data cache.

enumerator HWLOC_OBJ_CACHE_INSTRUCTION¶: Instruction cache (filtered out by default).

enum hwloc_obj_bridge_type_e¶

Type of one side (upstream or downstream) of an I/O bridge.

Values:

enumerator HWLOC_OBJ_BRIDGE_HOST¶: Host-side of a bridge, only possible upstream.

enumerator HWLOC_OBJ_BRIDGE_PCI¶: PCI-side of a bridge.

enum hwloc_obj_osdev_type_e¶

Single type of a OS device. Multiple of these may be combined for a single device.

Values:

enumerator HWLOC_OBJ_OSDEV_STORAGE¶: Operating system storage device (e.g. block). For instance “sda” or “pmem0” on Linux, or “dax0.0” if backed by non-volatile memory (may be combined with HWLOC_OBJ_OSDEV_MEMORY).

enumerator HWLOC_OBJ_OSDEV_MEMORY¶: Operating system memory device. For instance “dax2.0” on Linux, either for normal DRAM when not exposed as a NUMA node (e.g. CXL), special-purpose memory (SPM), high-bandwidth memory (HBM), or non-volatile memory (may be combined with HWLOC_OBJ_OSDEV_STORAGE).

enumerator HWLOC_OBJ_OSDEV_GPU¶: Operating system GPU device. For instance “:0.0” for a GL display, “card0” for a Linux DRM device, “cuda0”, “rsmi1”, “ze0.0” for driver-specific GPU handles (may be combined with HWLOC_OBJ_OSDEV_COPROC).

enumerator HWLOC_OBJ_OSDEV_COPROC¶: Operating system co-processor device. For instance “opencl0d0” for a OpenCL device, “cuda0”, “rsmi1”, “ze0.0” for driver-specific GPU handles. (may be combined with HWLOC_OBJ_OSDEV_GPU).

enumerator HWLOC_OBJ_OSDEV_NETWORK¶: Operating system network device. For instance the “eth0” interface, “bxi0” Atos/Bull BXI HCA, as well as OpenFabrics HCA. (may be combined with HWLOC_OBJ_OSDEV_OPENFABRICS).

enumerator HWLOC_OBJ_OSDEV_OPENFABRICS¶: Operating system OpenFabrics device. For instance the “mlx4_0” InfiniBand HCA, or “hfi1_0” Omni-Path interface. (usually combined with HWLOC_OBJ_OSDEV_NETWORK).

enumerator HWLOC_OBJ_OSDEV_DMA¶: Operating system dma engine device. For instance the “dma0chan0” DMA channel on Linux.

Functions

int hwloc_compare_types(hwloc_obj_type_t type1, hwloc_obj_type_t type2)¶

Compare the depth of two object types.

Types shouldn’t be compared as they are, since newer ones may be added in the future.

Note

Object types containing CPUs can always be compared (usually, a machine contains packages, which contain caches, which contain cores, which contain PUs).

Note

HWLOC_OBJ_PU will always be the deepest, while HWLOC_OBJ_MACHINE is always the highest.

Note

This does not mean that the actual topology will respect that order: e.g. as of today cores may also contain caches, and packages may also contain nodes. This is thus just to be seen as a fallback comparison method.

Returns:: A negative integer if type1 objects usually include type2 objects.
Returns:: A positive integer if type1 objects are usually included in type2 objects.
Returns:: 0 if type1 and type2 objects are the same.
Returns:: HWLOC_TYPE_UNORDERED if objects cannot be compared (because neither is usually contained in the other).

Object Structure and Attributes¶

group Object Structure and Attributes

Typedefs

typedef struct hwloc_obj *hwloc_obj_t¶: Convenience typedef; a pointer to a struct hwloc_obj.

struct hwloc_info_s¶

#include <hwloc.h>

Info attribute (name and value strings).

See also

Consulting and Adding Info Attributes

struct hwloc_infos_s¶

#include <hwloc.h>

Array of string info attributes (pairs of name and value).

These structures may only be modified with hwloc_obj_add_info() and hwloc_modify_infos().

See also

Consulting and Adding Info Attributes

struct hwloc_obj¶

#include <hwloc.h>

Structure of a topology object.

Applications must not modify any field except hwloc_obj.userdata.

union hwloc_obj_attr_u¶

#include <hwloc.h>

Object type-specific Attributes.

Public Members

struct hwloc_obj_attr_u.hwloc_numanode_attr_s numanode¶

struct hwloc_obj_attr_u.hwloc_cache_attr_s cache¶

struct hwloc_obj_attr_u.hwloc_group_attr_s group¶

struct hwloc_obj_attr_u.hwloc_pcidev_attr_s pcidev¶

struct hwloc_obj_attr_u.hwloc_bridge_attr_s bridge¶

struct hwloc_obj_attr_u.hwloc_osdev_attr_s osdev¶

struct hwloc_numanode_attr_s¶: #include <hwloc.h>

NUMA node-specific Object Attributes.

struct hwloc_cache_attr_s¶: #include <hwloc.h>

Cache-specific Object Attributes.

struct hwloc_group_attr_s¶: #include <hwloc.h>

Group-specific Object Attributes.

struct hwloc_pcidev_attr_s¶: #include <hwloc.h>

PCI Device specific Object Attributes.

struct hwloc_bridge_attr_s¶: #include <hwloc.h>

Bridge specific Object Attributes.

struct hwloc_osdev_attr_s¶: #include <hwloc.h>

OS Device specific Object Attributes.

Topology Creation and Destruction¶

group Topology Creation and Destruction

Typedefs

typedef struct hwloc_topology *hwloc_topology_t¶

Topology context.

To be initialized with hwloc_topology_init() and built with hwloc_topology_load().

Functions

int hwloc_topology_init(hwloc_topology_t *topologyp)¶

Allocate a topology context.

Parameters:

topologyp – [out] is assigned a pointer to the new allocated context.

Returns:

0 on success, -1 on error.

int hwloc_topology_load(hwloc_topology_t topology)¶

Build the actual topology.

Build the actual topology once initialized with hwloc_topology_init() and tuned with Topology Detection Configuration and Query and Changing the Source of Topology Discovery routines. No other routine may be called earlier using this topology context.

Note

On failure, the topology is reinitialized. It should be either destroyed with hwloc_topology_destroy() or configured and loaded again.

Note

This function may be called only once per topology.

Note

The binding of the current thread or process may temporarily change during this call but it will be restored before it returns.

Parameters:

topology – is the topology to be loaded with objects.

Returns:

0 on success, -1 on error.

void hwloc_topology_destroy(hwloc_topology_t topology)¶

Terminate and free a topology context.

Parameters:

topology – is the topology to be freed

int hwloc_topology_dup(hwloc_topology_t *newtopology, hwloc_topology_t oldtopology)¶

Duplicate a topology.

The entire topology structure as well as its objects are duplicated into a new one.

This is useful for keeping a backup while modifying a topology.

Note

Object userdata is not duplicated since hwloc does not know what it point to. The objects of both old and new topologies will point to the same userdata.

Returns:: 0 on success, -1 on error.

int hwloc_topology_abi_check(hwloc_topology_t topology)¶

Verify that the topology is compatible with the current hwloc library.

This is useful when using the same topology structure (in memory) in different libraries that may use different hwloc installations (for instance if one library embeds a specific version of hwloc, while another library uses a default system-wide hwloc installation).

If all libraries/programs use the same hwloc installation, this function always returns success.

Note

If sharing between processes with hwloc_shmem_topology_write(), the relevant check is already performed inside hwloc_shmem_topology_adopt().

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if incompatible.

void hwloc_topology_check(hwloc_topology_t topology)¶

Run internal checks on a topology structure.

The program aborts if an inconsistency is detected in the given topology.

Note

This routine is only useful to developers.

Note

The input topology should have been previously loaded with hwloc_topology_load().

Parameters:

topology – is the topology to be checked

Object levels, depths and types¶

group Object levels, depths and types

Be sure to see the figure in Terms and Definitions that shows a complete topology tree, including depths, child/sibling/cousin relationships, and an example of an asymmetric topology where one package has fewer caches than its peers.

Enums

enum hwloc_get_type_depth_e¶

Values:

enumerator HWLOC_TYPE_DEPTH_UNKNOWN¶: No object of given type exists in the topology.

enumerator HWLOC_TYPE_DEPTH_MULTIPLE¶: Objects of given type exist at different depth in the topology (only for Groups).

enumerator HWLOC_TYPE_DEPTH_NUMANODE¶: Virtual depth for NUMA nodes.

enumerator HWLOC_TYPE_DEPTH_BRIDGE¶: Virtual depth for bridge object level.

enumerator HWLOC_TYPE_DEPTH_PCI_DEVICE¶: Virtual depth for PCI device object level.

enumerator HWLOC_TYPE_DEPTH_OS_DEVICE¶: Virtual depth for software device object level.

enumerator HWLOC_TYPE_DEPTH_MISC¶: Virtual depth for Misc object.

enumerator HWLOC_TYPE_DEPTH_MEMCACHE¶: Virtual depth for MemCache object.

Functions

int hwloc_topology_get_depth(hwloc_topology_t restrict topology)¶

Get the depth of the hierarchical tree of objects.

This is the depth of HWLOC_OBJ_PU objects plus one.

Note

NUMA nodes, I/O and Misc objects are ignored when computing the depth of the tree (they are placed on special levels).

Returns:: the depth of the object tree.

int hwloc_get_type_depth(hwloc_topology_t topology, hwloc_obj_type_t type)¶

Returns the depth of objects of type type.

See also

hwloc_get_memory_parents_depth() for managing the depth of memory objects.

See also

hwloc_get_type_depth_with_attr() for disambiguating cases where HWLOC_TYPE_DEPTH_MULTIPLE is returned.

Note

If the type is absent but a similar type is acceptable, see also hwloc_get_type_or_below_depth() and hwloc_get_type_or_above_depth().

Returns:: The depth of objects of type type.
Returns:: A negative virtual depth if a NUMA node, I/O or Misc object type is given. These objects are stored in special levels that are not CPU-related. This virtual depth may be passed to other hwloc functions such as hwloc_get_obj_by_depth() but it should not be considered as an actual depth by the application. In particular, it should not be compared with any other object depth or with the entire topology depth.
Returns:: HWLOC_TYPE_DEPTH_UNKNOWN if no object of this type is present on the underlying architecture, or if the OS doesn’t provide this kind of information.
Returns:: HWLOC_TYPE_DEPTH_MULTIPLE if type HWLOC_OBJ_GROUP is given and multiple levels of Groups exist.

int hwloc_get_type_depth_with_attr(hwloc_topology_t topology, hwloc_obj_type_t type, union hwloc_obj_attr_u *attrp, size_t attrsize)¶

Returns the depth of objects of type and attributes attrp.

This is identical to hwloc_get_type_depth() but may disambiguate multiple levels with same type by looking at object attributes given in attrp (the attribute structure size should also be given in attrsize). For instance these attributes may specify the Group depth.

These attributes may for instance have been previously obtained with hwloc_type_sscanf().

If attrp is NULL and attrsize is 0, attributes are ignored and the function behaves as hwloc_get_type_depth().

int hwloc_get_memory_parents_depth(hwloc_topology_t topology)¶

Return the depth of parents where memory objects are attached.

Memory objects have virtual negative depths because they are not part of the main CPU-side hierarchy of objects. This depth should not be compared with other level depths.

If all Memory objects are attached to Normal parents at the same depth, this parent depth may be compared to other as usual, for instance for knowing whether NUMA nodes is attached above or below Packages.

Returns:: The depth of Normal parents of all memory children if all these parents have the same depth. For instance the depth of the Package level if all NUMA nodes are attached to Package objects.
Returns:: HWLOC_TYPE_DEPTH_MULTIPLE if Normal parents of all memory children do not have the same depth. For instance if some NUMA nodes are attached to Packages while others are attached to Groups.

inline int hwloc_get_type_or_below_depth(hwloc_topology_t topology, hwloc_obj_type_t type)¶

Returns the depth of objects of type type or below.

If no object of this type is present on the underlying architecture, the function returns the depth of the first “present” object typically found inside type.

This function is only meaningful for normal object types. If a memory, I/O or Misc object type is given, the corresponding virtual depth is always returned (see hwloc_get_type_depth()).

May return HWLOC_TYPE_DEPTH_MULTIPLE for HWLOC_OBJ_GROUP just like hwloc_get_type_depth().

inline int hwloc_get_type_or_above_depth(hwloc_topology_t topology, hwloc_obj_type_t type)¶

Returns the depth of objects of type type or above.

If no object of this type is present on the underlying architecture, the function returns the depth of the first “present” object typically containing type.

This function is only meaningful for normal object types. If a memory, I/O or Misc object type is given, the corresponding virtual depth is always returned (see hwloc_get_type_depth()).

May return HWLOC_TYPE_DEPTH_MULTIPLE for HWLOC_OBJ_GROUP just like hwloc_get_type_depth().

hwloc_obj_type_t hwloc_get_depth_type(hwloc_topology_t topology, int depth)¶

Returns the type of objects at depth depth.

depth should between 0 and hwloc_topology_get_depth()-1, or a virtual depth such as HWLOC_TYPE_DEPTH_NUMANODE.

Returns:: The type of objects at depth depth.
Returns:: (hwloc_obj_type_t)-1 if depth depth does not exist.

unsigned hwloc_get_nbobjs_by_depth(hwloc_topology_t topology, int depth)¶

Returns the width of level at depth depth.

Returns:: The number of objects at topology depth depth.
Returns:: 0 if there are no objects at depth depth.

inline int hwloc_get_nbobjs_by_type(hwloc_topology_t topology, hwloc_obj_type_t type)¶

Returns the width of level type type.

Returns:: The number of objects of type type.
Returns:: -1 if there are multiple levels with objects of that type, e.g. HWLOC_OBJ_GROUP.
Returns:: 0 if there are no objects at depth depth.

inline hwloc_obj_t hwloc_get_root_obj(hwloc_topology_t topology)¶

Returns the top-object of the topology-tree.

Its type is HWLOC_OBJ_MACHINE.

This function cannot return NULL.

hwloc_obj_t hwloc_get_obj_by_depth(hwloc_topology_t topology, int depth, unsigned idx)¶

Returns the topology object at logical index idx from depth depth.

Returns:: The object if it exists.
Returns:: NULL if there is no object with this index and depth.

inline hwloc_obj_t hwloc_get_obj_by_type(hwloc_topology_t topology, hwloc_obj_type_t type, unsigned idx)¶

Returns the topology object at logical index idx with type type.

Returns:: The object if it exists.
Returns:: NULL if there is no object with this index and type.
Returns:: NULL if there are multiple levels with objects of that type (e.g. HWLOC_OBJ_GROUP), the caller may fallback to hwloc_get_obj_by_depth().

inline hwloc_obj_t hwloc_get_next_obj_by_depth(hwloc_topology_t topology, int depth, hwloc_obj_t prev)¶

Returns the next object at depth depth.

Returns:: The first object at depth depth if prev is NULL.
Returns:: The object after prev at depth depth if prev is not NULL.
Returns:: NULL if there is no such object.

inline hwloc_obj_t hwloc_get_next_obj_by_type(hwloc_topology_t topology, hwloc_obj_type_t type, hwloc_obj_t prev)¶

Returns the next object of type type.

Returns:: The first object of type type if prev is NULL.
Returns:: The object after prev of type type if prev is not NULL.
Returns:: NULL if there is no such object.
Returns:: NULL if there are multiple levels with objects of that type (e.g. HWLOC_OBJ_GROUP), the caller may fallback to hwloc_get_obj_by_depth().

Converting between Object Types and Attributes, and Strings¶

group Converting between Object Types and Attributes, and Strings

Enums

enum hwloc_obj_snprintf_flag_e¶

Flags to be given to hwloc_obj_type_snprintf() and hwloc_obj_attr_snprintf().

Values:

enumerator HWLOC_OBJ_SNPRINTF_FLAG_LONG_NAMES¶: Use long type names such as L2Cache instead of L2.

enumerator HWLOC_OBJ_SNPRINTF_FLAG_SHORT_NAMES¶: Reduce the name even if it may become ambiguous, for instance by removing the OS device prefix and showing a single type. hwloc_type_sscanf() might not be able to parse it back exactly anymore.

enumerator HWLOC_OBJ_SNPRINTF_FLAG_MORE_ATTRS¶: Display additional attributes such as cache associativity, PCI link speed, and total memory.

enumerator HWLOC_OBJ_SNPRINTF_FLAG_NO_UNITS¶: Display memory sizes in bytes without units.

enumerator HWLOC_OBJ_SNPRINTF_FLAG_UNITS_1000¶: Display memory sizes in KB, MB, GB, etc i.e. divide by 1000 instead of 1024 for KiB, MiB, GiB, etc.

enumerator HWLOC_OBJ_SNPRINTF_FLAG_OLD_VERBOSE¶: Backward compatibility with hwloc 2.x verbose mode, shows additional attributes, and memory sizes with KB unit but KiB values.

Functions

const char *hwloc_obj_type_string(hwloc_obj_type_t type)¶

Return a constant stringified object type.

This function is the basic way to convert a generic type into a string. The output string may be parsed back by hwloc_type_sscanf().

hwloc_obj_type_snprintf() may return a more precise output for a specific object, but it requires the caller to provide the output buffer.

Returns:: A constant string containing the object type name or "Unknown".

int hwloc_obj_type_snprintf(char *restrict string, size_t size, hwloc_obj_t obj, unsigned long flags)¶

Stringify the type of a given topology object into a human-readable form.

Contrary to hwloc_obj_type_string(), this function includes object-specific attributes (such as the Group depth, the Bridge type, or OS device type) in the output, and it requires the caller to provide the output buffer.

The output is guaranteed to be the same for all objects of a same topology level.

The output string may be parsed back by hwloc_type_sscanf().

If size is 0, string may safely be NULL.

Flags flags is a OR’ed set of hwloc_obj_snprintf_flag_e. By default, short names are used.

Returns:: the number of characters that were actually written if not truncating, or that would have been written (not including the ending \0).

int hwloc_obj_attr_snprintf(char *restrict string, size_t size, hwloc_obj_t obj, const char *restrict separator, unsigned long flags)¶

Stringify the attributes of a given topology object into a human-readable form.

Attribute values are separated by separator.

Only the major attributes are printed in non-verbose mode.

If size is 0, string may safely be NULL.

Flags flags is a OR’ed set of hwloc_obj_snprintf_flag_e. By default, only important attributes such as memory and cache sizes are shown. Sizes are reported in units such as GiB or KiB.

Note

By default the output string is reasonably short without being ambiguous so that hwloc_type_sscanf() may parse it back.

Returns:: the number of characters that were actually written if not truncating, or that would have been written (not including the ending \0).

int hwloc_type_sscanf(const char *string, hwloc_obj_type_t *typep, union hwloc_obj_attr_u *attrp, size_t attrsize)¶

Return an object type and attributes from a type string.

Convert strings such as “Package” or “L1iCache” into the corresponding types. Matching is case-insensitive, and only the first letters are actually required to match.

The matched object type is set in typep (which cannot be NULL).

Type-specific attributes, for instance Cache type, Cache depth, Group depth, Bridge type or OS Device type may be returned in attrp. Attributes that are not specified in the string (for instance “Group” without a depth, or “L2Cache” without a cache type) are set to -1. The OS-device-specific type attribute is rather set to 0.

attrp is only filled if not NULL and if its size specified in attrsize is large enough. It should be at least as large as union hwloc_obj_attr_u.

Note

This function is guaranteed to match any string returned by hwloc_obj_type_string() or hwloc_obj_type_snprintf() except if HWLOC_OBJ_SNPRINTF_FLAG_SHORT_NAMES was given.

Returns:: 0 if a type was correctly identified, otherwise -1.

Consulting and Adding Info Attributes¶

group Consulting and Adding Info Attributes

Enums

enum hwloc_modify_infos_op_e¶

Operations given to hwloc_modify_infos().

Values:

enumerator HWLOC_MODIFY_INFOS_OP_ADD¶

Add a new info attribute with the given name and value.

Return:: 1 if the pair was successfully added.

enumerator HWLOC_MODIFY_INFOS_OP_ADD_UNIQUE¶

Add a new info attribute with the given name and value only if that pair doesn’t exist yet.

Return:: 1 if the pair was successfully added.
Return:: 0 if the pair was ignored because it already existed.

enumerator HWLOC_MODIFY_INFOS_OP_REPLACE¶

Replace existing info attributes with the given name, with a single attribute with the given name and value. If no existing pair matches, add a new one. If multiple pairs match, only one remains.

Return:: 1 if the pair was added.
Return:: N+1 if N existing pairs were replaced by one.

enumerator HWLOC_MODIFY_INFOS_OP_REMOVE¶

Remove existing info attributes that matches the given name and/or value if not NULL.

Return:: N if N existing pairs were removed.
Return:: 0 if no matching pair was found and removed.

Functions

inline const char *hwloc_obj_get_info_by_name(hwloc_obj_t obj, const char *name)¶

Search the given name in the object array of infos and return the corresponding value.

Identical to hwloc_get_info_by_name() but operates on the infos of the given object.

inline const char *hwloc_get_info_by_name(struct hwloc_infos_s *infos, const char *name)¶

Search the given name in the array of infos and return the corresponding value.

If multiple info attributes match the given name, only the first one is returned.

Note

The string should not be freed by the caller, it belongs to the hwloc library.

Returns:: A pointer to the value string if it exists.
Returns:: NULL if no such info attribute exists.

int hwloc_modify_infos(struct hwloc_infos_s *infos, unsigned long operation, const char *name, const char *value)¶

Modify an array of info attributes.

operation is a single value among hwloc_modify_infos_op_e.

If adding (HWLOC_MODIFY_INFOS_OP_ADD or HWLOC_MODIFY_INFOS_OP_ADD_UNIQUE) or replacing (HWLOC_MODIFY_INFOS_OP_REPLACE) an info attribute, name and value cannot be NULL. The input strings are copied before being added in the object infos.

If removing existing info pairs (HWLOC_MODIFY_INFOS_OP_REMOVE), name and/or value may be non NULL to specify which pair(s) to remove. If both name and value are NULL, all pairs are removed.

Note

If value contains some non-printable characters, they will be dropped when exporting to XML, see hwloc_topology_export_xml() in hwloc/export.h.

Returns:: a positive value if some info attributes were added/modified/removed (see the documentation of each operation).
Returns:: 0 if the request was valid but no change was applied.
Returns:: -1 on error.

inline int hwloc_obj_add_info(hwloc_obj_t obj, const char *name, const char *value)¶

Add the given name and value pair to the given object info attributes.

The info pair is appended to the existing info array even if another pair with the same name already exists. See hwloc_modify_infos() for a more flexible variant.

The input strings are copied before being added in the object infos.

name and value must be non NULL.

Note

This function may be used to enforce object colors in the lstopo graphical output by adding “lstopoStyle” as a name and “Background=#rrggbb” as a value. See CUSTOM COLORS in the lstopo(1) manpage for details.

Note

If name or value contain some non-printable characters, they will be dropped when exporting to XML, see hwloc_topology_export_xml() in hwloc/export.h.

Returns:: 0 on success, -1 on error.

struct hwloc_infos_s *hwloc_topology_get_infos(hwloc_topology_t topology)¶

Get the structure of info attributes attached to the topology.

These attributes contain information about the topology discovery (backends) and about the operating system.

Note

The function cannot return NULL.

Note

The returned pointer belongs to the hwloc library, it should not be freed or modified.

int hwloc_obj_set_subtype(hwloc_topology_t topology, hwloc_obj_t obj, const char *subtype)¶

Set (or replace) the subtype of an object.

The given subtype is copied internally, the caller is responsible for freeing the original subtype if needed.

If another subtype already exists in object, it is replaced. The given subtype may be NULL to remove the existing subtype.

Note

This function is mostly meant to initialize the subtype of user-added objects such as groups with hwloc_topology_alloc_group_object().

Returns:: 0 on success.
Returns:: -1 with errno set to ENOMEM on failure to allocate memory.

CPU Binding¶

group CPU binding

Some operating systems only support binding threads or processes to a single PU. Others allow binding to larger sets such as entire Cores or Packages or even random sets of individual PUs. In such operating system, the scheduler is free to run the task on one of these PU, then migrate it to another PU, etc. It is often useful to call hwloc_bitmap_singlify() on the target CPU set before passing it to the binding function to avoid these expensive migrations. See the documentation of hwloc_bitmap_singlify() for details.

Some operating systems do not provide all hwloc-supported mechanisms to bind processes, threads, etc. hwloc_topology_get_support() may be used to query about the actual CPU binding support in the currently used operating system.

When the requested binding operation is not available and the HWLOC_CPUBIND_STRICT flag was passed, the function returns -1. errno is set to ENOSYS when it is not possible to bind the requested kind of object processes/threads. errno is set to EXDEV when the requested cpuset can not be enforced (e.g. some systems only allow one CPU, and some other systems only allow one NUMA node).

If HWLOC_CPUBIND_STRICT was not passed, the function may fail as well, or the operating system may use a slightly different operation (with side-effects, smaller binding set, etc.) when the requested operation is not exactly supported.

The most portable version that should be preferred over the others, whenever possible, is the following one which just binds the current program, assuming it is single-threaded:

hwloc_set_cpubind(topology, set, 0),

If the program may be multithreaded, the following one should be preferred to only bind the current thread:

hwloc_set_cpubind(topology, set, HWLOC_CPUBIND_THREAD),

See also

Some example codes are available under doc/examples/ in the source tree.

Note

To unbind, just call the binding function with either a full cpuset or a cpuset equal to the system cpuset.

Note

On some operating systems, CPU binding may have effects on memory binding, see HWLOC_CPUBIND_NOMEMBIND

Note

Running lstopo –top or hwloc-ps can be a very convenient tool to check how binding actually happened.

Enums

enum hwloc_cpubind_flags_t¶

Process/Thread binding flags.

These bit flags can be used to refine the binding policy.

The default (0) is to bind the current process, assumed to be single-threaded, in a non-strict way. This is the most portable way to bind as all operating systems usually provide it.

Note

Not all systems support all kinds of binding. See the “Detailed Description” section of CPU binding for a description of errors that can occur.

Values:

enumerator HWLOC_CPUBIND_PROCESS¶: Bind all threads of the current (possibly) multithreaded process.

enumerator HWLOC_CPUBIND_THREAD¶: Bind current thread of current process.

enumerator HWLOC_CPUBIND_STRICT¶

Request for strict binding from the OS.

By default, when the designated CPUs are all busy while other CPUs are idle, operating systems may execute the thread/process on those other CPUs instead of the designated CPUs, to let them progress anyway. Strict binding means that the thread/process will _never_ execute on other CPUs than the designated CPUs, even when those are busy with other tasks and other CPUs are idle.

When retrieving the binding of a process, this flag checks whether all its threads actually have the same binding. If the flag is not given, the binding of each thread will be accumulated.

Note

Depending on the operating system, strict binding may not be possible (e.g., the OS does not implement it) or not allowed (e.g., for an administrative reasons), and the function will fail in that case.

Note

This flag is meaningless when retrieving the binding of a thread.

enumerator HWLOC_CPUBIND_NOMEMBIND¶

Avoid any effect on memory binding.

On some operating systems, some CPU binding function would also bind the memory on the corresponding NUMA node. It is often not a problem for the application, but if it is, setting this flag will make hwloc avoid using OS functions that would also bind memory. This will however reduce the support of CPU bindings, i.e. potentially return -1 with errno set to ENOSYS in some cases.

This flag is only meaningful when used with functions that set the CPU binding. It is ignored when used with functions that get CPU binding information.

Functions

int hwloc_set_cpubind(hwloc_topology_t topology, hwloc_const_cpuset_t set, int flags)¶

Bind current process or thread on CPUs given in physical bitmap set.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOSYS if the action is not supported.
Returns:: -1 with errno set to EXDEV if the binding cannot be enforced.

int hwloc_get_cpubind(hwloc_topology_t topology, hwloc_cpuset_t set, int flags)¶

Get current process or thread binding.

The CPU-set set (previously allocated by the caller) is filled with the list of PUs which the process or thread (according to flags) was last bound to.

Returns:: 0 on success, -1 on error.

int hwloc_set_proc_cpubind(hwloc_topology_t topology, hwloc_pid_t pid, hwloc_const_cpuset_t set, int flags)¶

Bind a process pid on CPUs given in physical bitmap set.

Note

hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native Windows platforms.

Note

As a special case on Linux, if a tid (thread ID) is supplied instead of a pid (process ID) and HWLOC_CPUBIND_THREAD is passed in flags, the binding is applied to that specific thread.

Note

On non-Linux systems, HWLOC_CPUBIND_THREAD can not be used in flags.

Returns:: 0 on success, -1 on error.

int hwloc_get_proc_cpubind(hwloc_topology_t topology, hwloc_pid_t pid, hwloc_cpuset_t set, int flags)¶

Get the current physical binding of process pid.

The CPU-set set (previously allocated by the caller) is filled with the list of PUs which the process was last bound to.

Note

hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native Windows platforms.

Note

As a special case on Linux, if a tid (thread ID) is supplied instead of a pid (process ID) and HWLOC_CPUBIND_THREAD is passed in flags, the binding for that specific thread is returned.

Note

On non-Linux systems, HWLOC_CPUBIND_THREAD can not be used in flags.

Returns:: 0 on success, -1 on error.

int hwloc_set_thread_cpubind(hwloc_topology_t topology, hwloc_thread_t thread, hwloc_const_cpuset_t set, int flags)¶

Bind a thread thread on CPUs given in physical bitmap set.

Note

hwloc_thread_t is pthread_t on Unix platforms, and HANDLE on native Windows platforms.

Note

HWLOC_CPUBIND_PROCESS can not be used in flags.

Returns:: 0 on success, -1 on error.

int hwloc_get_thread_cpubind(hwloc_topology_t topology, hwloc_thread_t thread, hwloc_cpuset_t set, int flags)¶

Get the current physical binding of thread tid.

The CPU-set set (previously allocated by the caller) is filled with the list of PUs which the thread was last bound to.

Note

hwloc_thread_t is pthread_t on Unix platforms, and HANDLE on native Windows platforms.

Note

HWLOC_CPUBIND_PROCESS can not be used in flags.

Returns:: 0 on success, -1 on error.

int hwloc_get_last_cpu_location(hwloc_topology_t topology, hwloc_cpuset_t set, int flags)¶

Get the last physical CPU where the current process or thread ran.

The CPU-set set (previously allocated by the caller) is filled with the list of PUs which the process or thread (according to flags) last ran on.

The operating system may move some tasks from one processor to another at any time according to their binding, so this function may return something that is already outdated.

flags can include either HWLOC_CPUBIND_PROCESS or HWLOC_CPUBIND_THREAD to specify whether the query should be for the whole process (union of all CPUs on which all threads are running), or only the current thread. If the process is single-threaded, flags can be set to zero to let hwloc use whichever method is available on the underlying OS.

Returns:: 0 on success, -1 on error.

int hwloc_get_proc_last_cpu_location(hwloc_topology_t topology, hwloc_pid_t pid, hwloc_cpuset_t set, int flags)¶

Get the last physical CPU where a process ran.

The CPU-set set (previously allocated by the caller) is filled with the list of PUs which the process last ran on.

The operating system may move some tasks from one processor to another at any time according to their binding, so this function may return something that is already outdated.

Note

hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native Windows platforms.

Note

As a special case on Linux, if a tid (thread ID) is supplied instead of a pid (process ID) and HWLOC_CPUBIND_THREAD is passed in flags, the last CPU location of that specific thread is returned.

Note

On non-Linux systems, HWLOC_CPUBIND_THREAD can not be used in flags.

Returns:: 0 on success, -1 on error.

Memory Binding¶

group Memory binding

Memory binding can be done three ways:

explicit memory allocation thanks to hwloc_alloc_membind() and friends: the binding will have effect on the memory allocated by these functions.
implicit memory binding through binding policy: hwloc_set_membind() and friends only define the current policy of the process, which will be applied to the subsequent calls to malloc() and friends.
migration of existing memory ranges, thanks to hwloc_set_area_membind() and friends, which move already-allocated data.

Not all operating systems support all three ways. hwloc_topology_get_support() may be used to query about the actual memory binding support in the currently used operating system.

When the requested binding operation is not available and the HWLOC_MEMBIND_STRICT flag was passed, the function returns -1. errno will be set to ENOSYS when the system does support the specified action or policy (e.g., some systems only allow binding memory on a per-thread basis, whereas other systems only allow binding memory for all threads in a process). errno will be set to EXDEV when the requested set can not be enforced (e.g., some systems only allow binding memory to a single NUMA node).

If HWLOC_MEMBIND_STRICT was not passed, the function may fail as well, or the operating system may use a slightly different operation (with side-effects, smaller binding set, etc.) when the requested operation is not exactly supported.

The most portable form that should be preferred over the others whenever possible is as follows. It allocates some memory hopefully bound to the specified set. To do so, hwloc will possibly have to change the current memory binding policy in order to actually get the memory bound, if the OS does not provide any other way to simply allocate bound memory without changing the policy for all allocations. That is the difference with hwloc_alloc_membind(), which will never change the current memory binding policy.

hwloc_alloc_membind_policy(topology, size, set,
                           HWLOC_MEMBIND_BIND, 0);

Each hwloc memory binding function takes a bitmap argument that is a CPU set by default, or a NUMA memory node set if the flag HWLOC_MEMBIND_BYNODESET is specified. See Object Sets (hwloc_cpuset_t and hwloc_nodeset_t) and The bitmap API for a discussion of CPU sets and NUMA memory node sets. It is also possible to convert between CPU set and node set using hwloc_cpuset_to_nodeset() or hwloc_cpuset_from_nodeset().

Memory binding by CPU set cannot work for CPU-less NUMA memory nodes. Binding by nodeset should therefore be preferred whenever possible.

See also

Some example codes are available under doc/examples/ in the source tree.

Note

On some operating systems, memory binding affects the CPU binding; see HWLOC_MEMBIND_NOCPUBIND

Enums

enum hwloc_membind_policy_t¶

Memory binding policy.

These constants can be used to choose the binding policy. Only one policy can be used at a time (i.e., the values cannot be OR’ed together).

Not all systems support all kinds of binding. hwloc_topology_get_support() may be used to query about the actual memory binding policy support in the currently used operating system. See the “Detailed Description” section of Memory binding for a description of errors that can occur.

Values:

enumerator HWLOC_MEMBIND_DEFAULT¶: Reset the memory allocation policy to the system default. Depending on the operating system, this may correspond to HWLOC_MEMBIND_FIRSTTOUCH (Linux, FreeBSD), or HWLOC_MEMBIND_BIND (AIX, HP-UX, Solaris, Windows). This policy is never returned by get membind functions. The nodeset argument is ignored.

enumerator HWLOC_MEMBIND_FIRSTTOUCH¶

Allocate each memory page individually on the local NUMA node of the thread that touches it.

The given nodeset should usually be hwloc_topology_get_topology_nodeset() so that the touching thread may run and allocate on any node in the system.

On AIX, if the nodeset is smaller, pages are allocated locally (if the local node is in the nodeset) or from a random non-local node (otherwise).

enumerator HWLOC_MEMBIND_BIND¶

Allocate memory on the specified nodes.

The actual behavior may slightly vary between operating systems, especially when (some of) the requested nodes are full. On Linux, by default, the MPOL_PREFERRED_MANY (or MPOL_PREFERRED) policy is used. However, if the hwloc strict flag is also given, the Linux MPOL_BIND policy is rather used.

enumerator HWLOC_MEMBIND_INTERLEAVE¶: Allocate memory on the given nodes in an interleaved / round-robin manner. The precise layout of the memory across multiple NUMA nodes is OS/system specific. Interleaving can be useful when threads distributed across the specified NUMA nodes will all be accessing the whole memory range concurrently, since the interleave will then balance the memory references.

enumerator HWLOC_MEMBIND_WEIGHTED_INTERLEAVE¶: Allocate memory on the given nodes in an interleaved / weighted manner. The precise layout of the memory across multiple NUMA nodes is OS/system specific. Weighted interleaving can be useful when threads distributed across the specified NUMA nodes with different bandwidth capabilities will all be accessing the whole memory range concurrently, since the interleave will then balance the memory references.

enumerator HWLOC_MEMBIND_NEXTTOUCH¶: For each page bound with this policy, by next time it is touched (and next time only), it is moved from its current location to the local NUMA node of the thread where the memory reference occurred (if it needs to be moved at all).

enumerator HWLOC_MEMBIND_MIXED¶: Returned by get_membind() functions when multiple threads or parts of a memory area have differing memory binding policies. Also returned when binding is unknown because binding hooks are empty when the topology is loaded from XML without HWLOC_THISSYSTEM=1, etc.

enum hwloc_membind_flags_t¶

Memory binding flags.

These flags can be used to refine the binding policy. All flags can be logically OR’ed together with the exception of HWLOC_MEMBIND_PROCESS and HWLOC_MEMBIND_THREAD; these two flags are mutually exclusive.

Not all systems support all kinds of binding. hwloc_topology_get_support() may be used to query about the actual memory binding support in the currently used operating system. See the “Detailed Description” section of Memory binding for a description of errors that can occur.

Values:

enumerator HWLOC_MEMBIND_PROCESS¶: Set policy for all threads of the specified (possibly multithreaded) process. This flag is mutually exclusive with HWLOC_MEMBIND_THREAD.

enumerator HWLOC_MEMBIND_THREAD¶: Set policy for a specific thread of the current process. This flag is mutually exclusive with HWLOC_MEMBIND_PROCESS.

enumerator HWLOC_MEMBIND_STRICT¶

Request strict binding from the OS. The function will fail if the binding can not be guaranteed / completely enforced.

This flag has slightly different meanings depending on which function it is used with.

enumerator HWLOC_MEMBIND_MIGRATE¶: Migrate existing allocated memory. If the memory cannot be migrated and the HWLOC_MEMBIND_STRICT flag is passed, an error will be returned.

enumerator HWLOC_MEMBIND_NOCPUBIND¶

Avoid any effect on CPU binding.

On some operating systems, some underlying memory binding functions also bind the application to the corresponding CPU(s). Using this flag will cause hwloc to avoid using OS functions that could potentially affect CPU bindings. Note, however, that using NOCPUBIND may reduce hwloc’s overall memory binding support. Specifically: some of hwloc’s memory binding functions may fail with errno set to ENOSYS when used with NOCPUBIND.

enumerator HWLOC_MEMBIND_BYNODESET¶

Consider the bitmap argument as a nodeset.

The bitmap argument is considered a nodeset if this flag is given, or a cpuset otherwise by default.

Memory binding by CPU set cannot work for CPU-less NUMA memory nodes. Binding by nodeset should therefore be preferred whenever possible.

Functions

int hwloc_set_membind(hwloc_topology_t topology, hwloc_const_bitmap_t set, hwloc_membind_policy_t policy, int flags)¶

Set the default memory binding policy of the current process or thread to prefer the NUMA node(s) specified by set.

If neither HWLOC_MEMBIND_PROCESS nor HWLOC_MEMBIND_THREAD is specified, the current process is assumed to be single-threaded. This is the most portable form as it permits hwloc to use either process-based OS functions or thread-based OS functions, depending on which are available.

If HWLOC_MEMBIND_BYNODESET is specified, set is considered a nodeset. Otherwise it’s a cpuset.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOSYS if the action is not supported.
Returns:: -1 with errno set to EXDEV if the binding cannot be enforced.

int hwloc_get_membind(hwloc_topology_t topology, hwloc_bitmap_t set, hwloc_membind_policy_t *policy, int flags)¶

Query the default memory binding policy and physical locality of the current process or thread.

The bitmap set (previously allocated by the caller) is filled with the process or thread memory binding.

This function has two output parameters: set and policy. The values returned in these parameters depend on both the flags passed in and the current memory binding policies and nodesets in the queried target.

Passing the HWLOC_MEMBIND_PROCESS flag specifies that the query target is the current policies and nodesets for all the threads in the current process. Passing HWLOC_MEMBIND_THREAD specifies that the query target is the current policy and nodeset for only the thread invoking this function.

If neither of these flags are passed (which is the most portable method), the process is assumed to be single threaded. This allows hwloc to use either process-based OS functions or thread-based OS functions, depending on which are available.

HWLOC_MEMBIND_STRICT is only meaningful when HWLOC_MEMBIND_PROCESS is also specified. In this case, hwloc will check the default memory policies and nodesets for all threads in the process. If they are not identical, -1 is returned and errno is set to EXDEV. If they are identical, the values are returned in set and policy.

Otherwise, if HWLOC_MEMBIND_PROCESS is specified (and HWLOC_MEMBIND_STRICT is not specified), the default set from each thread is logically OR’ed together. If all threads’ default policies are the same, policy is set to that policy. If they are different, policy is set to HWLOC_MEMBIND_MIXED.

In the HWLOC_MEMBIND_THREAD case (or when neither HWLOC_MEMBIND_PROCESS or HWLOC_MEMBIND_THREAD is specified), there is only one set and policy; they are returned in set and policy, respectively.

If HWLOC_MEMBIND_BYNODESET is specified, set is considered a nodeset. Otherwise it’s a cpuset.

If any other flags are specified, -1 is returned and errno is set to EINVAL.

Returns:: 0 on success, -1 on error.

int hwloc_set_proc_membind(hwloc_topology_t topology, hwloc_pid_t pid, hwloc_const_bitmap_t set, hwloc_membind_policy_t policy, int flags)¶

Set the default memory binding policy of the specified process to prefer the NUMA node(s) specified by set.

If HWLOC_MEMBIND_BYNODESET is specified, set is considered a nodeset. Otherwise it’s a cpuset.

Note

hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native Windows platforms.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOSYS if the action is not supported.
Returns:: -1 with errno set to EXDEV if the binding cannot be enforced.

int hwloc_get_proc_membind(hwloc_topology_t topology, hwloc_pid_t pid, hwloc_bitmap_t set, hwloc_membind_policy_t *policy, int flags)¶

Query the default memory binding policy and physical locality of the specified process.

The bitmap set (previously allocated by the caller) is filled with the process memory binding.

This function has two output parameters: set and policy. The values returned in these parameters depend on both the flags passed in and the current memory binding policies and nodesets in the queried target.

Passing the HWLOC_MEMBIND_PROCESS flag specifies that the query target is the current policies and nodesets for all the threads in the specified process. If HWLOC_MEMBIND_PROCESS is not specified (which is the most portable method), the process is assumed to be single threaded. This allows hwloc to use either process-based OS functions or thread-based OS functions, depending on which are available.

Note that it does not make sense to pass HWLOC_MEMBIND_THREAD to this function.

If HWLOC_MEMBIND_STRICT is specified, hwloc will check the default memory policies and nodesets for all threads in the specified process. If they are not identical, -1 is returned and errno is set to EXDEV. If they are identical, the values are returned in set and policy.

Otherwise, set is set to the logical OR of all threads’ default set. If all threads’ default policies are the same, policy is set to that policy. If they are different, policy is set to HWLOC_MEMBIND_MIXED.

If HWLOC_MEMBIND_BYNODESET is specified, set is considered a nodeset. Otherwise it’s a cpuset.

If any other flags are specified, -1 is returned and errno is set to EINVAL.

Note

hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native Windows platforms.

Returns:: 0 on success, -1 on error.

int hwloc_set_area_membind(hwloc_topology_t topology, const void *addr, size_t len, hwloc_const_bitmap_t set, hwloc_membind_policy_t policy, int flags)¶

Bind the already-allocated memory identified by (addr, len) to the NUMA node(s) specified by set.

If HWLOC_MEMBIND_BYNODESET is specified, set is considered a nodeset. Otherwise it’s a cpuset.

Returns:: 0 on success or if len is 0.
Returns:: -1 with errno set to ENOSYS if the action is not supported.
Returns:: -1 with errno set to EXDEV if the binding cannot be enforced.

int hwloc_get_area_membind(hwloc_topology_t topology, const void *addr, size_t len, hwloc_bitmap_t set, hwloc_membind_policy_t *policy, int flags)¶

Query the CPUs near the physical NUMA node(s) and binding policy of the memory identified by (addr, len ).

The bitmap set (previously allocated by the caller) is filled with the memory area binding.

This function has two output parameters: set and policy. The values returned in these parameters depend on both the flags passed in and the memory binding policies and nodesets of the pages in the address range.

If HWLOC_MEMBIND_STRICT is specified, the target pages are first checked to see if they all have the same memory binding policy and nodeset. If they do not, -1 is returned and errno is set to EXDEV. If they are identical across all pages, the set and policy are returned in set and policy, respectively.

If HWLOC_MEMBIND_STRICT is not specified, the union of all NUMA node(s) containing pages in the address range is calculated. If all pages in the target have the same policy, it is returned in policy. Otherwise, policy is set to HWLOC_MEMBIND_MIXED.

If HWLOC_MEMBIND_BYNODESET is specified, set is considered a nodeset. Otherwise it’s a cpuset.

If any other flags are specified, -1 is returned and errno is set to EINVAL.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if len is 0.

int hwloc_get_area_memlocation(hwloc_topology_t topology, const void *addr, size_t len, hwloc_bitmap_t set, int flags)¶

Get the NUMA nodes where memory identified by (addr, len ) is physically allocated.

The bitmap set (previously allocated by the caller) is filled according to the NUMA nodes where the memory area pages are physically allocated. If no page is actually allocated yet, set may be empty.

If pages spread to multiple nodes, it is not specified whether they spread equitably, or whether most of them are on a single node, etc.

The operating system may move memory pages from one processor to another at any time according to their binding, so this function may return something that is already outdated.

If HWLOC_MEMBIND_BYNODESET is specified in flags, set is considered a nodeset. Otherwise it’s a cpuset.

If len is 0, set is emptied.

Returns:: 0 on success, -1 on error.

void *hwloc_alloc(hwloc_topology_t topology, size_t len)¶

Allocate some memory.

This is equivalent to malloc(), except that it tries to allocate page-aligned memory from the OS.

Note

The allocated memory should be freed with hwloc_free().

Returns:: a pointer to the allocated area, or NULL on error.

void *hwloc_alloc_membind(hwloc_topology_t topology, size_t len, hwloc_const_bitmap_t set, hwloc_membind_policy_t policy, int flags)¶

Allocate some memory on NUMA memory nodes specified by set.

If HWLOC_MEMBIND_BYNODESET is specified, set is considered a nodeset. Otherwise it’s a cpuset.

Note

The allocated memory should be freed with hwloc_free().

Returns:: a pointer to the allocated area.
Returns:: NULL with errno set to ENOSYS if the action is not supported and HWLOC_MEMBIND_STRICT is given.
Returns:: NULL with errno set to EXDEV if the binding cannot be enforced and HWLOC_MEMBIND_STRICT is given.
Returns:: NULL with errno set to ENOMEM if the memory allocation failed even before trying to bind.

inline void *hwloc_alloc_membind_policy(hwloc_topology_t topology, size_t len, hwloc_const_bitmap_t set, hwloc_membind_policy_t policy, int flags)¶

Allocate some memory on NUMA memory nodes specified by set.

First, try to allocate properly with hwloc_alloc_membind(). On failure, the current process or thread memory binding policy is changed with hwloc_set_membind() before allocating memory. Thus this function works in more cases, at the expense of changing the current state (possibly affecting future allocations that would not specify any policy).

If HWLOC_MEMBIND_BYNODESET is specified, set is considered a nodeset. Otherwise it’s a cpuset.

Returns:: a pointer to the allocated area, or NULL on error.

int hwloc_free(hwloc_topology_t topology, void *addr, size_t len)¶

Free memory that was previously allocated by hwloc_alloc() or hwloc_alloc_membind().

Returns:: 0 on success, -1 on error.

Changing the Source of Topology Discovery¶

group Changing the Source of Topology Discovery

These functions must be called between hwloc_topology_init() and hwloc_topology_load(). Otherwise, they will return -1 with errno set to EBUSY.

If none of the functions below is called, the default is to detect all the objects of the machine that the caller is allowed to access.

This default behavior may also be modified through environment variables if the application did not modify it already. Setting HWLOC_XMLFILE in the environment enforces the discovery from a XML file as if hwloc_topology_set_xml() had been called. Setting HWLOC_SYNTHETIC enforces a synthetic topology as if hwloc_topology_set_synthetic() had been called.

Finally, HWLOC_THISSYSTEM enforces the return value of hwloc_topology_is_thissystem().

Enums

enum hwloc_topology_components_flag_e¶

Flags to be passed to hwloc_topology_set_components().

Values:

enumerator HWLOC_TOPOLOGY_COMPONENTS_FLAG_BLACKLIST¶: Blacklist the target component from being used.

Functions

int hwloc_topology_set_pid(hwloc_topology_t restrict topology, hwloc_pid_t pid)¶

Change which process the topology is viewed from.

On some systems, processes may have different views of the machine, for instance the set of allowed CPUs. By default, hwloc exposes the view from the current process. Calling hwloc_topology_set_pid() permits to make it expose the topology of the machine from the point of view of another process.

Note

hwloc_pid_t is pid_t on Unix platforms, and HANDLE on native Windows platforms.

Note

-1 is returned and errno is set to ENOSYS on platforms that do not support this feature.

Note

The PID will not actually be used until hwloc_topology_load(). If the corresponding process exits in the meantime, hwloc will ignore the PID. If another process reuses the PID, the view of that process will be used.

Returns:: 0 on success, -1 on error.

int hwloc_topology_set_synthetic(hwloc_topology_t restrict topology, const char *restrict description)¶

Enable synthetic topology.

Gather topology information from the given description, a space-separated string of <type:number> describing the object type and arity at each level. All types may be omitted (space-separated string of numbers) so that hwloc chooses all types according to usual topologies. See also the Synthetic topologies.

Setting the environment variable HWLOC_SYNTHETIC may also result in this behavior.

If description was properly parsed and describes a valid topology configuration, this function returns 0. Otherwise -1 is returned and errno is set to EINVAL.

Note that this function does not actually load topology information; it just tells hwloc where to load it from. You’ll still need to invoke hwloc_topology_load() to actually load the topology information.

Note

For convenience, this backend provides empty binding hooks which just return success.

Note

On success, the synthetic component replaces the previously enabled component (if any), but the topology is not actually modified until hwloc_topology_load().

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if the description was invalid.

int hwloc_topology_set_xml(hwloc_topology_t restrict topology, const char *restrict xmlpath)¶

Enable XML-file based topology.

Gather topology information from the XML file given at xmlpath. Setting the environment variable HWLOC_XMLFILE may also result in this behavior. This file may have been generated earlier with hwloc_topology_export_xml() in hwloc/export.h, or lstopo file.xml.

Note that this function does not actually load topology information; it just tells hwloc where to load it from. You’ll still need to invoke hwloc_topology_load() to actually load the topology information.

Note

See also hwloc_topology_set_userdata_import_callback() for importing application-specific object userdata.

Note

For convenience, this backend provides empty binding hooks which just return success. To have hwloc still actually call OS-specific hooks, the HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM has to be set to assert that the loaded file is really the underlying system.

Note

On success, the XML component replaces the previously enabled component (if any), but the topology is not actually modified until hwloc_topology_load().

Note

If an invalid XML input file is given, the error may be reported either here or later by hwloc_topology_load() depending on the XML library used by hwloc.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL on failure to read the XML file.

int hwloc_topology_set_xmlbuffer(hwloc_topology_t restrict topology, const char *restrict buffer, int size)¶

Enable XML based topology using a memory buffer (instead of a file, as with hwloc_topology_set_xml()).

Gather topology information from the XML memory buffer given at buffer and of length size (including an ending \0). This buffer may have been filled earlier with hwloc_topology_export_xmlbuffer() in hwloc/export.h.

Note that this function does not actually load topology information; it just tells hwloc where to load it from. You’ll still need to invoke hwloc_topology_load() to actually load the topology information.

Note

See also hwloc_topology_set_userdata_import_callback() for importing application-specific object userdata.

Note

For convenience, this backend provides empty binding hooks which just return success. To have hwloc still actually call OS-specific hooks, the HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM has to be set to assert that the loaded file is really the underlying system.

Note

On success, the XML component replaces the previously enabled component (if any), but the topology is not actually modified until hwloc_topology_load().

Note

If an invalid XML input file is given, the error may be reported either here or later by hwloc_topology_load() depending on the XML library used by hwloc.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL on failure to read the XML buffer.

int hwloc_topology_set_components(hwloc_topology_t restrict topology, unsigned long flags, const char *restrict name)¶

Prevent a discovery component from being used for a topology.

name is the name of the discovery component that should not be used when loading topology topology. The name is a string such as “cuda”.

For components with multiple phases, it may also be suffixed with the name of a phase, for instance “linux:io”.

flags should be HWLOC_TOPOLOGY_COMPONENTS_FLAG_BLACKLIST.

This may be used to avoid expensive parts of the discovery process. For instance, CUDA-specific discovery may be expensive and unneeded while generic I/O discovery could still be useful.

Returns:: 0 on success.
Returns:: -1 on error, for instance if flags are invalid.

Topology Detection Configuration and Query¶

group Topology Detection Configuration and Query

Several functions can optionally be called between hwloc_topology_init() and hwloc_topology_load() to configure how the detection should be performed, e.g. to ignore some objects types, define a synthetic topology, etc.

Enums

enum hwloc_topology_flags_e¶

Flags to be set onto a topology context before load.

Flags should be given to hwloc_topology_set_flags(). They may also be returned by hwloc_topology_get_flags().

Values:

enumerator HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED¶

Detect the whole system, ignore reservations, include disallowed objects.

Gather all online resources, even if some were disabled by the administrator. For instance, ignore Linux Cgroup/Cpusets and gather all processors and memory nodes. However offline PUs and NUMA nodes are still ignored.

When this flag is not set, PUs and NUMA nodes that are disallowed are not added to the topology. Parent objects (package, core, cache, etc.) are added only if some of their children are allowed. All existing PUs and NUMA nodes in the topology are allowed. hwloc_topology_get_allowed_cpuset() and hwloc_topology_get_allowed_nodeset() are equal to the root object cpuset and nodeset.

When this flag is set, the actual sets of allowed PUs and NUMA nodes are given by hwloc_topology_get_allowed_cpuset() and hwloc_topology_get_allowed_nodeset(). They may be smaller than the root object cpuset and nodeset.

If the current topology is exported to XML and reimported later, this flag should be set again in the reimported topology so that disallowed resources are reimported as well.

enumerator HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM¶

Assume that the selected backend provides the topology for the system on which we are running.

This forces hwloc_topology_is_thissystem() to return 1, i.e. makes hwloc assume that the selected backend provides the topology for the system on which we are running, even if it is not the OS-specific backend but the XML backend for instance. This means making the binding functions actually call the OS-specific system calls and really do binding, while the XML backend would otherwise provide empty hooks just returning success.

Setting the environment variable HWLOC_THISSYSTEM may also result in the same behavior.

This can be used for efficiency reasons to first detect the topology once, save it to an XML file, and quickly reload it later through the XML backend, but still having binding functions actually do bind.

enumerator HWLOC_TOPOLOGY_FLAG_THISSYSTEM_ALLOWED_RESOURCES¶

Get the set of allowed resources from the local operating system even if the topology was loaded from XML or synthetic description.

If the topology was loaded from XML or from a synthetic string, restrict it by applying the current process restrictions such as Linux Cgroup/Cpuset.

This is useful when the topology is not loaded directly from the local machine (e.g. for performance reason) and it comes with all resources, while the running process is restricted to only parts of the machine.

This flag is ignored unless HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM is also set since the loaded topology must match the underlying machine where restrictions will be gathered from.

Setting the environment variable HWLOC_THISSYSTEM_ALLOWED_RESOURCES would result in the same behavior.

enumerator HWLOC_TOPOLOGY_FLAG_IMPORT_SUPPORT¶

Import support from the imported topology.

When importing a XML topology from a remote machine, binding is disabled by default (see HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM). This disabling is also marked by putting zeroes in the corresponding supported feature bits reported by hwloc_topology_get_support().

The flag HWLOC_TOPOLOGY_FLAG_IMPORT_SUPPORT actually imports support bits from the remote machine. It also sets the flag imported_support in the struct hwloc_topology_misc_support array. If the imported XML did not contain any support information (exporter hwloc is too old), this flag is not set.

Note that these supported features are only relevant for the hwloc installation that actually exported the XML topology (it may vary with the operating system, or with how hwloc was compiled).

Note that setting this flag however does not enable binding for the locally imported hwloc topology, it only reports what the remote hwloc and machine support.

enumerator HWLOC_TOPOLOGY_FLAG_RESTRICT_TO_CPUBINDING¶

Do not consider resources outside of the process CPU binding.

If the binding of the process is limited to a subset of cores, ignore the other cores during discovery.

The resulting topology is identical to what a call to hwloc_topology_restrict() would generate, but this flag also prevents hwloc from ever touching other resources during the discovery.

This flag especially tells the x86 backend to never temporarily rebind a thread on any excluded core. This is useful on Windows because such temporary rebinding can change the process binding. Another use-case is to avoid cores that would not be able to perform the hwloc discovery anytime soon because they are busy executing some high-priority real-time tasks.

If process CPU binding is not supported, the thread CPU binding is considered instead if supported, or the flag is ignored.

This flag requires HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM as well since binding support is required.

enumerator HWLOC_TOPOLOGY_FLAG_RESTRICT_TO_MEMBINDING¶

Do not consider resources outside of the process memory binding.

If the binding of the process is limited to a subset of NUMA nodes, ignore the other NUMA nodes during discovery.

The resulting topology is identical to what a call to hwloc_topology_restrict() would generate, but this flag also prevents hwloc from ever touching other resources during the discovery.

This flag is meant to be used together with HWLOC_TOPOLOGY_FLAG_RESTRICT_TO_CPUBINDING when both cores and NUMA nodes should be ignored outside of the process binding.

If process memory binding is not supported, the thread memory binding is considered instead if supported, or the flag is ignored.

This flag requires HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM as well since binding support is required.

enumerator HWLOC_TOPOLOGY_FLAG_DONT_CHANGE_BINDING¶

Do not ever modify the process or thread binding during discovery.

This flag disables all hwloc discovery steps that require a change of the process or thread binding. This currently only affects the x86 backend which gets entirely disabled.

This is useful when hwloc_topology_load() is called while the application also creates additional threads or modifies the binding.

This flag is also a strict way to make sure the process binding will not change to due thread binding changes on Windows (see HWLOC_TOPOLOGY_FLAG_RESTRICT_TO_CPUBINDING).

enumerator HWLOC_TOPOLOGY_FLAG_NO_DISTANCES¶

Ignore distances.

Ignore distance information from the operating systems (and from XML) and hence do not use distances for grouping.

enumerator HWLOC_TOPOLOGY_FLAG_NO_MEMATTRS¶

Ignore memory attributes and tiers.

Ignore memory attribues from the operating systems (and from XML) Hence also do not try to build memory tiers.

enumerator HWLOC_TOPOLOGY_FLAG_NO_CPUKINDS¶

Ignore CPU Kinds.

Ignore CPU kind information from the operating systems (and from XML).

enum hwloc_type_filter_e¶

Type filtering flags.

By default, most objects are kept (HWLOC_TYPE_FILTER_KEEP_ALL). Instruction caches, memory-side caches, I/O and Misc objects are ignored by default (HWLOC_TYPE_FILTER_KEEP_NONE). Group levels are ignored unless they bring structure (HWLOC_TYPE_FILTER_KEEP_STRUCTURE).

Note that group objects are also ignored individually (without the entire level) when they do not bring structure.

Values:

enumerator HWLOC_TYPE_FILTER_KEEP_ALL¶

Keep all objects of this type.

Cannot be set for HWLOC_OBJ_GROUP (groups are designed only to add more structure to the topology).

enumerator HWLOC_TYPE_FILTER_KEEP_NONE¶

Ignore all objects of this type.

The bottom-level type HWLOC_OBJ_PU, the HWLOC_OBJ_NUMANODE type, and the top-level type HWLOC_OBJ_MACHINE may not be ignored.

enumerator HWLOC_TYPE_FILTER_KEEP_STRUCTURE¶

Only ignore objects if their entire level does not bring any structure.

Keep the entire level of objects if at least one of these objects adds structure to the topology. An object brings structure when it has multiple children and it is not the only child of its parent.

If all objects in the level are the only child of their parent, and if none of them has multiple children, the entire level is removed.

Cannot be set for I/O and Misc objects since the topology structure does not matter there.

enumerator HWLOC_TYPE_FILTER_KEEP_IMPORTANT¶

Only keep likely-important objects of the given type.

It is only useful for I/O object types. For HWLOC_OBJ_PCI_DEVICE and HWLOC_OBJ_OS_DEVICE, it means that only objects of major/common kinds are kept (storage, network, OpenFabrics, CUDA, OpenCL, RSMI, NVML, and displays). Also, only OS devices directly attached on PCI (e.g. no USB) are reported. For HWLOC_OBJ_BRIDGE, it means that bridges are kept only if they have children.

This flag equivalent to HWLOC_TYPE_FILTER_KEEP_ALL for Normal, Memory and Misc types since they are likely important.

Functions

int hwloc_topology_set_flags(hwloc_topology_t topology, unsigned long flags)¶

Set OR’ed flags to non-yet-loaded topology.

Set a OR’ed set of hwloc_topology_flags_e onto a topology that was not yet loaded.

If this function is called multiple times, the last invocation will erase and replace the set of flags that was previously set.

By default, no flags are set (0).

The flags set in a topology may be retrieved with hwloc_topology_get_flags().

Returns:: 0 on success.
Returns:: -1 on error, for instance if flags are invalid.

unsigned long hwloc_topology_get_flags(hwloc_topology_t topology)¶

Get OR’ed flags of a topology.

Get the OR’ed set of hwloc_topology_flags_e of a topology.

If hwloc_topology_set_flags() was not called earlier, no flags are set (0 is returned).

Note

This function may also be called after hwloc_topology_load().

Returns:: the flags previously set with hwloc_topology_set_flags().

int hwloc_topology_is_thissystem(hwloc_topology_t restrict topology)¶

Does the topology context come from this system?

Note

This function may also be called after hwloc_topology_load().

Returns:: 1 if this topology context was built using the system running this program.
Returns:: 0 instead (for instance if using another file-system root, a XML topology file, or a synthetic topology).

const struct hwloc_topology_support *hwloc_topology_get_support(hwloc_topology_t restrict topology)¶

Retrieve the topology support.

Each flag indicates whether a feature is supported. If set to 0, the feature is not supported. If set to 1, the feature is supported, but the corresponding call may still fail in some corner cases.

These features are also listed by hwloc-info –support

The reported features are what the current topology supports on the current machine. If the topology was exported to XML from another machine and later imported here, support still describes what is supported for this imported topology after import. By default, binding will be reported as unsupported in this case (see HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM).

Topology flag HWLOC_TOPOLOGY_FLAG_IMPORT_SUPPORT may be used to report the supported features of the original remote machine instead. If it was successfully imported, imported_support will be set in the struct hwloc_topology_misc_support array.

Note

The function cannot return NULL.

Note

The returned pointer should not be freed, it belongs to the hwloc library.

Note

This function may be called before or after hwloc_topology_load() but the support structure only contains valid information after.

Returns:: A pointer to a support structure.

int hwloc_topology_set_type_filter(hwloc_topology_t topology, hwloc_obj_type_t type, enum hwloc_type_filter_e filter)¶

Set the filtering for the given object type.

Returns:: 0 on success, -1 on error.

int hwloc_topology_get_type_filter(hwloc_topology_t topology, hwloc_obj_type_t type, enum hwloc_type_filter_e *filter)¶

Get the current filtering for the given object type.

Returns:: 0 on success, -1 on error.

int hwloc_topology_set_all_types_filter(hwloc_topology_t topology, enum hwloc_type_filter_e filter)¶

Set the filtering for all object types.

If some types do not support this filtering, they are silently ignored.

Returns:: 0 on success, -1 on error.

int hwloc_topology_set_cache_types_filter(hwloc_topology_t topology, enum hwloc_type_filter_e filter)¶

Set the filtering for all CPU cache object types.

Memory-side caches are not involved since they are not CPU caches.

Returns:: 0 on success, -1 on error.

int hwloc_topology_set_icache_types_filter(hwloc_topology_t topology, enum hwloc_type_filter_e filter)¶

Set the filtering for all CPU instruction cache object types.

Memory-side caches are not involved since they are not CPU caches.

Returns:: 0 on success, -1 on error.

int hwloc_topology_set_io_types_filter(hwloc_topology_t topology, enum hwloc_type_filter_e filter)¶

Set the filtering for all I/O object types.

Returns:: 0 on success, -1 on error.

void hwloc_topology_set_userdata(hwloc_topology_t topology, const void *userdata)¶

Set the topology-specific userdata pointer.

Each topology may store one application-given private data pointer. It is initialized to NULL. hwloc will never modify it.

Use it as you wish, after hwloc_topology_init() and until hwloc_topolog_destroy().

This pointer is not exported to XML.

void *hwloc_topology_get_userdata(hwloc_topology_t topology)¶

Retrieve the topology-specific userdata pointer.

Retrieve the application-given private data pointer that was previously set with hwloc_topology_set_userdata().

Returns:: A pointer to the private-data if any.
Returns:: NULL if no private-data was previoulsy set.

struct hwloc_topology_discovery_support¶: #include <hwloc.h>

Flags describing actual discovery support for this topology.

struct hwloc_topology_cpubind_support¶

#include <hwloc.h>

Flags describing actual PU binding support for this topology.

A flag may be set even if the feature isn’t supported in all cases (e.g. binding to random sets of non-contiguous objects).

struct hwloc_topology_membind_support¶

#include <hwloc.h>

Flags describing actual memory binding support for this topology.

A flag may be set even if the feature isn’t supported in all cases (e.g. binding to random sets of non-contiguous objects).

struct hwloc_topology_misc_support¶: #include <hwloc.h>

Flags describing miscellaneous features.

struct hwloc_topology_support¶

#include <hwloc.h>

Set of flags describing actual support for this topology.

This is retrieved with hwloc_topology_get_support() and will be valid until the topology object is destroyed. Note: the values are correct only after discovery.

Modifying a loaded Topology¶

group Modifying a loaded Topology

Enums

enum hwloc_restrict_flags_e¶

Flags to be given to hwloc_topology_restrict().

Values:

enumerator HWLOC_RESTRICT_FLAG_REMOVE_CPULESS¶: Remove all objects that became CPU-less. By default, only objects that contain no PU and no memory are removed. This flag may not be used with HWLOC_RESTRICT_FLAG_BYNODESET.

enumerator HWLOC_RESTRICT_FLAG_BYNODESET¶: Restrict by nodeset instead of CPU set. Only keep objects whose nodeset is included or partially included in the given set. This flag may not be used with HWLOC_RESTRICT_FLAG_REMOVE_CPULESS.

enumerator HWLOC_RESTRICT_FLAG_REMOVE_MEMLESS¶: Remove all objects that became Memory-less. By default, only objects that contain no PU and no memory are removed. This flag may only be used with HWLOC_RESTRICT_FLAG_BYNODESET.

enumerator HWLOC_RESTRICT_FLAG_ADAPT_MISC¶: Move Misc objects to ancestors if their parents are removed during restriction. If this flag is not set, Misc objects are removed when their parents are removed.

enumerator HWLOC_RESTRICT_FLAG_ADAPT_IO¶: Move I/O objects to ancestors if their parents are removed during restriction. If this flag is not set, I/O devices and bridges are removed when their parents are removed.

enum hwloc_allow_flags_e¶

Flags to be given to hwloc_topology_allow().

Values:

enumerator HWLOC_ALLOW_FLAG_ALL¶

Mark all objects as allowed in the topology.

cpuset and nođeset given to hwloc_topology_allow() must be NULL.

enumerator HWLOC_ALLOW_FLAG_LOCAL_RESTRICTIONS¶

Only allow objects that are available to the current process.

The topology must have HWLOC_TOPOLOGY_FLAG_IS_THISSYSTEM so that the set of available resources can actually be retrieved from the operating system.

cpuset and nođeset given to hwloc_topology_allow() must be NULL.

enumerator HWLOC_ALLOW_FLAG_CUSTOM¶: Allow a custom set of objects, given to hwloc_topology_allow() as cpuset and/or nodeset parameters.

Functions

int hwloc_topology_restrict(hwloc_topology_t restrict topology, hwloc_const_bitmap_t set, unsigned long flags)¶

Restrict the topology to the given CPU set or nodeset.

Topology topology is modified so as to remove all objects that are not included (or partially included) in the CPU set set. All objects CPU and node sets are restricted accordingly.

By default, set is a CPU set. It means that the set of PUs in the topology is restricted. Once some PUs got removed, their parents may also get removed recursively if they became child-less.

If HWLOC_RESTRICT_FLAG_BYNODESET is passed in flags, set is considered a nodeset instead of a CPU set. It means that the set of NUMA nodes in the topology is restricted (instead of PUs). Once some NUMA nodes got removed, their parents may also get removed recursively if they became child-less.

flags is a OR’ed set of hwloc_restrict_flags_e.

Note

Restricting the topology removes some locality information, hence the remaining objects may get reordered (including PUs and NUMA nodes), and their logical indexes may change.

Note

This call may not be reverted by restricting back to a larger set. Once dropped during restriction, objects may not be brought back, except by loading another topology with hwloc_topology_load().

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if the input set is invalid. The topology is not modified in this case.
Returns:: -1 with errno set to ENOMEM on failure to allocate internal data. The topology is reinitialized in this case. It should be either destroyed with hwloc_topology_destroy() or configured and loaded again.

int hwloc_topology_allow(hwloc_topology_t restrict topology, hwloc_const_cpuset_t cpuset, hwloc_const_nodeset_t nodeset, unsigned long flags)¶

Change the sets of allowed PUs and NUMA nodes in the topology.

This function only works if the HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED was set on the topology. It does not modify any object, it only changes the sets returned by hwloc_topology_get_allowed_cpuset() and hwloc_topology_get_allowed_nodeset().

It is notably useful when importing a topology from another process running in a different Linux Cgroup.

flags must be set to one flag among hwloc_allow_flags_e.

Note

This function is allowed on an adopted topology (see Sharing topologies between processes). The flag HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED is required on the original topology (before sharing).

Note

Removing objects from a topology should rather be performed with hwloc_topology_restrict().

Returns:: 0 on success, -1 on error.

hwloc_obj_t hwloc_topology_insert_misc_object(hwloc_topology_t topology, hwloc_obj_t parent, const char *name)¶

Add a MISC object as a leaf of the topology.

A new MISC object will be created and inserted into the topology at the position given by parent. It is appended to the list of existing Misc children, without ever adding any intermediate hierarchy level. This is useful for annotating the topology without actually changing the hierarchy.

name is supposed to be unique across all Misc objects in the topology. It will be duplicated to setup the new object attributes.

The new leaf object will not have any cpuset.

The subtype object attribute may be defined with hwloc_obj_set_subtype() after successful insertion.

Note

If name contains some non-printable characters, they will be dropped when exporting to XML, see hwloc_topology_export_xml() in hwloc/export.h.

Returns:: the newly-created object
Returns:: NULL on error.
Returns:: NULL if Misc objects are filtered-out of the topology (HWLOC_TYPE_FILTER_KEEP_NONE).

hwloc_obj_t hwloc_topology_alloc_group_object(hwloc_topology_t topology)¶

Allocate a Group object to insert later with hwloc_topology_insert_group_object().

This function returns a new Group object.

The caller should (at least) initialize its sets before inserting the object in the topology, see hwloc_topology_insert_group_object(). Or it may decide not to insert and just free the group object by calling hwloc_topology_free_group_object().

Note

If successfully inserted by hwloc_topology_insert_group_object(), the object will be freed when the entire topology is freed. If insertion failed (e.g. NULL or empty CPU and node-sets), it is freed before returning the error.

Returns:: The allocated object on success.
Returns:: NULL on error.

int hwloc_topology_free_group_object(hwloc_topology_t topology, hwloc_obj_t group)¶

Free a group object allocated with hwloc_topology_alloc_group_object().

This function is only useful if the group object was not given to hwloc_topology_insert_group_object() as planned.

Note

topology must be the same as the one previously passed to hwloc_topology_alloc_group_object().

Returns:: 0 on success.
Returns:: -1 on error, for instance if an invalid topology is given.

hwloc_obj_t hwloc_topology_insert_group_object(hwloc_topology_t topology, hwloc_obj_t group)¶

Add more structure to the topology by adding an intermediate Group.

The caller should first allocate a new Group object with hwloc_topology_alloc_group_object(). Then it must setup at least one of its CPU or node sets to specify the final location of the Group in the topology. Then the object can be passed to this function for actual insertion in the topology.

The main use case for this function is to group a subset of siblings among the list of children below a single parent. For instance, if grouping 4 cores out of a 8-core socket, the logical list of cores will be reordered so that the 4 grouped ones are consecutive. Then, if needed, a new depth is added between the parent and those children, and the Group is inserted there. At the end, the 4 grouped cores are now children of the Group, which replaces them as a child of the original parent.

In practice, the grouped objects are specified through cpusets and/or nodesets, for instance using hwloc_obj_add_other_obj_sets() iteratively. Hence it is possible to group objects that are not children of the same parent, for instance some PUs below the 4 cores in example above. However this general case may fail if the expected Group conflicts with the existing hierarchy. For instance if each core has two PUs, it is not possible to insert a Group containing a single PU of each core.

To specify the objects to group, either the cpuset or nodeset field (or both, if compatible) must be set to a non-empty bitmap. The complete_cpuset or complete_nodeset may be set instead if inserting with respect to the complete topology (including disallowed, offline or unknown objects). These sets cannot be larger than the current topology, or they would get restricted silently. The core will setup the other sets after actual insertion.

The subtype object attribute may be defined with hwloc_obj_set_subtype() to display something else than “Group” as the type name for this object in lstopo. Custom name-value info pairs may be added with hwloc_obj_add_info() after insertion.

The group dont_merge attribute may be set to 1 to prevent the hwloc core from ever merging this object with another hierarchically-identical object. This is useful when the Group itself describes an important feature that cannot be exposed anywhere else in the hierarchy.

The group kind attribute may be set to a high value such as 0xffffffff to tell hwloc that this new Group should always be discarded in favor of any existing Group with the same locality.

Note

Inserting a group adds some locality information to the topology, hence the existing objects may get reordered (including PUs and NUMA nodes), and their logical indexes may change.

Note

If the insertion fails, the input group object is freed.

Note

If the group object should be discarded instead of inserted, it may be passed to hwloc_topology_free_group_object() instead.

Note

topology must be the same as the one previously passed to hwloc_topology_alloc_group_object().

Returns:: The inserted object if it was properly inserted.
Returns:: An existing object if the Group was merged or discarded because the topology already contained an object at the same location (the Group did not add any hierarchy information).
Returns:: NULL if the insertion failed because of conflicting sets in topology tree.
Returns:: NULL if Group objects are filtered-out of the topology (HWLOC_TYPE_FILTER_KEEP_NONE).
Returns:: NULL if the object was discarded because no set was initialized in the Group before insert, or all of them were empty.

int hwloc_obj_add_other_obj_sets(hwloc_obj_t dst, hwloc_obj_t src)¶

Setup object cpusets/nodesets by OR’ing another object’s sets.

For each defined cpuset or nodeset in src, allocate the corresponding set in dst and add src to it by OR’ing sets.

This function is convenient between hwloc_topology_alloc_group_object() and hwloc_topology_insert_group_object(). It builds the sets of the new Group that will be inserted as a new intermediate parent of several objects.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOMEM if some internal reallocation failed.

int hwloc_topology_refresh(hwloc_topology_t topology)¶

Refresh internal structures after topology modification.

Modifying the topology (by restricting, adding objects, modifying structures such as distances or memory attributes, etc.) may cause some internal caches to become invalid. These caches are automatically refreshed when accessed but this refreshing is not thread-safe.

This function is not thread-safe either, but it is a good way to end a non-thread-safe phase of topology modification. Once this refresh is done, multiple threads may concurrently consult the topology, objects, distances, attributes, etc.

Kinds of object Type¶

group Kinds of object Type

Each object type is either Normal (i.e. hwloc_obj_type_is_normal() returns 1), or Memory (i.e. hwloc_obj_type_is_memory() returns 1) or I/O (i.e. hwloc_obj_type_is_io() returns 1) or Misc (i.e. equal to HWLOC_OBJ_MISC). It cannot be of more than one of these kinds.

See also Object Kind in Terms and Definitions.

Functions

int hwloc_obj_type_is_normal(hwloc_obj_type_t type)¶

Check whether an object type is Normal.

Normal objects are objects of the main CPU hierarchy (Machine, Package, Core, PU, CPU caches, etc.), but they are not NUMA nodes, I/O devices or Misc objects.

They are attached to parent as Normal children, not as Memory, I/O or Misc children.

Returns:: 1 if an object of type type is a Normal object, 0 otherwise.

int hwloc_obj_type_is_io(hwloc_obj_type_t type)¶

Check whether an object type is I/O.

I/O objects are objects attached to their parents in the I/O children list. This current includes Bridges, PCI and OS devices.

Returns:: 1 if an object of type type is a I/O object, 0 otherwise.

int hwloc_obj_type_is_memory(hwloc_obj_type_t type)¶

Check whether an object type is Memory.

Memory objects are objects attached to their parents in the Memory children list. This current includes NUMA nodes and Memory-side caches.

Returns:: 1 if an object of type type is a Memory object, 0 otherwise.

int hwloc_obj_type_is_cache(hwloc_obj_type_t type)¶

Check whether an object type is a CPU Cache (Data, Unified or Instruction).

Memory-side caches are not CPU caches.

Returns:: 1 if an object of type type is a Cache, 0 otherwise.

int hwloc_obj_type_is_dcache(hwloc_obj_type_t type)¶

Check whether an object type is a CPU Data or Unified Cache.

Memory-side caches are not CPU caches.

Returns:: 1 if an object of type type is a CPU Data or Unified Cache, 0 otherwise.

int hwloc_obj_type_is_icache(hwloc_obj_type_t type)¶

Check whether an object type is a CPU Instruction Cache,.

Memory-side caches are not CPU caches.

Returns:: 1 if an object of type type is a CPU Instruction Cache, 0 otherwise.

Finding Objects inside a CPU set¶

group Finding Objects inside a CPU set

Functions

inline hwloc_obj_t hwloc_get_first_largest_obj_inside_cpuset(hwloc_topology_t topology, hwloc_const_cpuset_t set)¶

Get the first largest object included in the given cpuset set.

This is convenient for iterating over all largest objects within a CPU set by doing a loop getting the first largest object and clearing its CPU set from the remaining CPU set.

Returns:: the first object that is included in set and whose parent is not.
Returns:: NULL if no such object exists.

int hwloc_get_largest_objs_inside_cpuset(hwloc_topology_t topology, hwloc_const_cpuset_t set, hwloc_obj_t *restrict objs, int max)¶

Get the set of largest objects covering exactly a given cpuset set.

Returns:: the number of objects returned in objs.
Returns:: -1 if no set of objects may cover that cpuset.

inline hwloc_obj_t hwloc_get_next_obj_inside_cpuset_by_depth(hwloc_topology_t topology, hwloc_const_cpuset_t set, int depth, hwloc_obj_t prev)¶

Return the next object at depth depth included in CPU set set.

The next invokation should pass the previous return value in prev so as to obtain the next object in set.

Note

Objects with empty CPU sets are ignored (otherwise they would be considered included in any given set).

Note

This function cannot work if objects at the given depth do not have CPU sets (I/O or Misc objects).

Returns:: the first object at depth depth included in set if prev is NULL.
Returns:: the next object at depth depth included in set if prev is not NULL.
Returns:: NULL if there is no next object.

inline hwloc_obj_t hwloc_get_next_obj_inside_cpuset_by_type(hwloc_topology_t topology, hwloc_const_cpuset_t set, hwloc_obj_type_t type, hwloc_obj_t prev)¶

Return the next object of type type included in CPU set set.

The next invokation should pass the previous return value in prev so as to obtain the next object in set.

Note

Objects with empty CPU sets are ignored (otherwise they would be considered included in any given set).

Note

This function cannot work if objects of the given type do not have CPU sets (I/O or Misc objects).

Returns:: the first object of type type included in set if prev is NULL.
Returns:: the next object of type type included in set if prev is not NULL.
Returns:: NULL if there is no next object.
Returns:: NULL if there is no depth for the given type.
Returns:: NULL if there are multiple depths for the given type, the caller should fallback to hwloc_get_next_obj_inside_cpuset_by_depth().

inline hwloc_obj_t hwloc_get_obj_inside_cpuset_by_depth(hwloc_topology_t topology, hwloc_const_cpuset_t set, int depth, unsigned idx)¶

Return the (logically) idx -th object at depth depth included in CPU set set.

Note

Objects with empty CPU sets are ignored (otherwise they would be considered included in any given set).

Note

This function cannot work if objects at the given depth do not have CPU sets (I/O or Misc objects).

Returns:: the object if any, NULL otherwise.

inline hwloc_obj_t hwloc_get_obj_inside_cpuset_by_type(hwloc_topology_t topology, hwloc_const_cpuset_t set, hwloc_obj_type_t type, unsigned idx)¶

Return the idx -th object of type type included in CPU set set.

Note

Objects with empty CPU sets are ignored (otherwise they would be considered included in any given set).

Note

This function cannot work if objects of the given type do not have CPU sets (I/O or Misc objects).

Returns:: the object if any.
Returns:: NULL if there is no such object.
Returns:: NULL if there is no depth for given type.
Returns:: NULL if there are multiple depths for given type, the caller should fallback to hwloc_get_obj_inside_cpuset_by_depth().

inline unsigned hwloc_get_nbobjs_inside_cpuset_by_depth(hwloc_topology_t topology, hwloc_const_cpuset_t set, int depth)¶

Return the number of objects at depth depth included in CPU set set.

Note

Objects with empty CPU sets are ignored (otherwise they would be considered included in any given set).

Note

This function cannot work if objects at the given depth do not have CPU sets (I/O or Misc objects).

Returns:: the number of objects.
Returns:: 0 if the depth is invalid.

inline int hwloc_get_nbobjs_inside_cpuset_by_type(hwloc_topology_t topology, hwloc_const_cpuset_t set, hwloc_obj_type_t type)¶

Return the number of objects of type type included in CPU set set.

Note

Objects with empty CPU sets are ignored (otherwise they would be considered included in any given set).

Note

This function cannot work if objects of the given type do not have CPU sets (I/O objects).

Returns:: the number of objects.
Returns:: 0 if there are no objects of that type in the topology.
Returns:: -1 if there are multiple levels of objects of that type, the caller should fallback to hwloc_get_nbobjs_inside_cpuset_by_depth().

inline int hwloc_get_obj_index_inside_cpuset(hwloc_topology_t topology, hwloc_const_cpuset_t set, hwloc_obj_t obj)¶

Return the logical index among the objects included in CPU set set.

Consult all objects in the same level as obj and inside CPU set set in the logical order, and return the index of obj within them. If set covers the entire topology, this is the logical index of obj. Otherwise, this is similar to a logical index within the part of the topology defined by CPU set set.

Note

Objects with empty CPU sets are ignored (otherwise they would be considered included in any given set).

Note

This function cannot work if obj does not have CPU sets (I/O objects).

Returns:: the logical index among the objects included in the set if any.
Returns:: -1 if the object is not included in the set.

Finding Objects covering at least CPU set¶

group Finding Objects covering at least CPU set

Functions

inline hwloc_obj_t hwloc_get_child_covering_cpuset(hwloc_topology_t topology, hwloc_const_cpuset_t set, hwloc_obj_t parent)¶

Get the child covering at least CPU set set.

Note

This function cannot work if parent does not have a CPU set (I/O or Misc objects).

Returns:: the child that covers the set entirely.
Returns:: NULL if no child matches or if set is empty.

inline hwloc_obj_t hwloc_get_obj_covering_cpuset(hwloc_topology_t topology, hwloc_const_cpuset_t set)¶

Get the lowest object covering at least CPU set set.

Returns:: the lowest object covering the set entirely.
Returns:: NULL if no object matches or if set is empty.

inline hwloc_obj_t hwloc_get_next_obj_covering_cpuset_by_depth(hwloc_topology_t topology, hwloc_const_cpuset_t set, int depth, hwloc_obj_t prev)¶

Iterate through same-depth objects covering at least CPU set set.

The next invokation should pass the previous return value in prev so as to obtain the next object covering at least another part of set.

Note

This function cannot work if objects at the given depth do not have CPU sets (I/O or Misc objects).

Returns:: the first object at depth depth covering at least part of CPU set set if object prev is NULL.
Returns:: the next one if prev is not NULL.
Returns:: NULL if there is no next object.

inline hwloc_obj_t hwloc_get_next_obj_covering_cpuset_by_type(hwloc_topology_t topology, hwloc_const_cpuset_t set, hwloc_obj_type_t type, hwloc_obj_t prev)¶

Iterate through same-type objects covering at least CPU set set.

The next invokation should pass the previous return value in prev so as to obtain the next object of type type covering at least another part of set.

Note

This function cannot work if objects of the given type do not have CPU sets (I/O or Misc objects).

Returns:: the first object of type type covering at least part of CPU set set if object prev is NULL.
Returns:: the next one if prev is not NULL.
Returns:: NULL if there is no next object.
Returns:: NULL if there is no depth for the given type.
Returns:: NULL if there are multiple depths for the given type, the caller should fallback to hwloc_get_next_obj_covering_cpuset_by_depth().

Looking at Ancestor and Child Objects¶

group Looking at Ancestor and Child Objects

Be sure to see the figure in Terms and Definitions that shows a complete topology tree, including depths, child/sibling/cousin relationships, and an example of an asymmetric topology where one package has fewer caches than its peers.

Functions

inline hwloc_obj_t hwloc_get_ancestor_obj_by_depth(hwloc_topology_t topology, int depth, hwloc_obj_t obj)¶

Returns the ancestor object of obj at depth depth.

Note

depth should not be the depth of PU or NUMA objects since they are ancestors of no objects (except Misc or I/O). This function rather expects an intermediate level depth, such as the depth of Packages, Cores, or Caches.

Returns:: the ancestor if any.
Returns:: NULL if no such ancestor exists.

inline hwloc_obj_t hwloc_get_ancestor_obj_by_type(hwloc_topology_t topology, hwloc_obj_type_t type, hwloc_obj_t obj)¶

Returns the ancestor object of obj with type type.

Note

if multiple matching ancestors exist (e.g. multiple levels of HWLOC_OBJ_GROUP) the lowest one is returned.

Note

type should not be HWLOC_OBJ_PU or HWLOC_OBJ_NUMANODE since these objects are ancestors of no objects (except Misc or I/O). This function rather expects an intermediate object type, such as HWLOC_OBJ_PACKAGE, HWLOC_OBJ_CORE, etc.

Returns:: the ancestor if any.
Returns:: NULL if no such ancestor exists.

inline hwloc_obj_t hwloc_get_common_ancestor_obj(hwloc_topology_t topology, hwloc_obj_t obj1, hwloc_obj_t obj2)¶

Returns the common parent object to objects obj1 and obj2.

Note

This function cannot return NULL.

Returns:: the common ancestor.

inline int hwloc_obj_is_in_subtree(hwloc_topology_t topology, hwloc_obj_t obj, hwloc_obj_t subtree_root)¶

Returns true if obj is inside the subtree beginning with ancestor object subtree_root.

Note

This function cannot work if obj and subtree_root objects do not have CPU sets (I/O or Misc objects).

Returns:: 1 is the object is in the subtree, 0 otherwise.

inline hwloc_obj_t hwloc_get_next_child(hwloc_topology_t topology, hwloc_obj_t parent, hwloc_obj_t prev)¶

Return the next child.

Return the next child among the normal children list, then among the memory children list, then among the I/O children list, then among the Misc children list.

Returns:: the first child if prev is NULL.
Returns:: the next child if prev is not NULL.
Returns:: NULL when there is no next child.

Looking at Cache Objects¶

group Looking at Cache Objects

Functions

inline int hwloc_get_cache_type_depth(hwloc_topology_t topology, unsigned cachelevel, hwloc_obj_cache_type_t cachetype)¶

Find the depth of cache objects matching cache level and type.

Return the depth of the topology level that contains cache objects whose attributes match cachelevel and cachetype.

This function is identical to calling hwloc_get_type_depth() with the corresponding type such as HWLOC_OBJ_L1ICACHE, except that it may also return a Unified cache when looking for an instruction cache.

Returns:: the depth of the unique matching unified cache level is returned if cachetype is HWLOC_OBJ_CACHE_UNIFIED.
Returns:: the depth of either a matching cache level or a unified cache level if cachetype is HWLOC_OBJ_CACHE_DATA or HWLOC_OBJ_CACHE_INSTRUCTION.
Returns:: the depth of the matching level if cachetype is -1 but only one level matches.
Returns:: HWLOC_TYPE_DEPTH_MULTIPLE if cachetype is -1 but multiple levels match.
Returns:: HWLOC_TYPE_DEPTH_UNKNOWN if no cache level matches.

inline hwloc_obj_t hwloc_get_cache_covering_cpuset(hwloc_topology_t topology, hwloc_const_cpuset_t set)¶

Get the first data (or unified) cache covering a cpuset set.

Returns:: a covering cache, or NULL if no cache matches.

inline hwloc_obj_t hwloc_get_shared_cache_covering_obj(hwloc_topology_t topology, hwloc_obj_t obj)¶

Get the first data (or unified) cache shared between an object and somebody else.

Returns:: a shared cache.
Returns:: NULL if no cache matches or if an invalid object is given (e.g. I/O object).

Finding objects, miscellaneous helpers¶

group Finding objects, miscellaneous helpers

Be sure to see the figure in Terms and Definitions that shows a complete topology tree, including depths, child/sibling/cousin relationships, and an example of an asymmetric topology where one package has fewer caches than its peers.

Functions

int hwloc_bitmap_singlify_per_core(hwloc_topology_t topology, hwloc_bitmap_t cpuset, unsigned which)¶

Remove simultaneous multithreading PUs from a CPU set.

For each core in topology, if cpuset contains some PUs of that core, modify cpuset to only keep a single PU for that core.

which specifies which PU will be kept. PU are considered in physical index order. If 0, for each core, the function keeps the first PU that was originally set in cpuset.

If which is larger than the number of PUs in a core there were originally set in cpuset, no PU is kept for that core.

Note

PUs that are not below a Core object are ignored (for instance if the topology does not contain any Core object). None of them is removed from cpuset.

Returns:: 0.

inline hwloc_obj_t hwloc_get_pu_obj_by_os_index(hwloc_topology_t topology, unsigned os_index)¶

Returns the object of type HWLOC_OBJ_PU with os_index.

This function is useful for converting a CPU set into the PU objects it contains. When retrieving the current binding (e.g. with hwloc_get_cpubind()), one may iterate over the bits of the resulting CPU set with hwloc_bitmap_foreach_begin(), and find the corresponding PUs with this function.

Returns:: the PU object, or NULL if none matches.

inline hwloc_obj_t hwloc_get_numanode_obj_by_os_index(hwloc_topology_t topology, unsigned os_index)¶

Returns the object of type HWLOC_OBJ_NUMANODE with os_index.

This function is useful for converting a nodeset into the NUMA node objects it contains. When retrieving the current binding (e.g. with hwloc_get_membind() with HWLOC_MEMBIND_BYNODESET), one may iterate over the bits of the resulting nodeset with hwloc_bitmap_foreach_begin(), and find the corresponding NUMA nodes with this function.

Returns:: the NUMA node object, or NULL if none matches.

unsigned hwloc_get_closest_objs(hwloc_topology_t topology, hwloc_obj_t src, hwloc_obj_t *restrict objs, unsigned max)¶

Do a depth-first traversal of the topology to find and sort.

all objects that are at the same depth than src. Report in objs up to max physically closest ones to src.

Note

This function requires the src object to have a CPU set.

Returns:: the number of objects returned in objs.
Returns:: 0 if src is an I/O object.

inline hwloc_obj_t hwloc_get_obj_below_by_type(hwloc_topology_t topology, hwloc_obj_type_t type1, unsigned idx1, hwloc_obj_type_t type2, unsigned idx2)¶

Find an object below another object, both specified by types and indexes.

Start from the top system object and find object of type type1 and logical index idx1. Then look below this object and find another object of type type2 and logical index idx2. Indexes are specified within the parent, not withing the entire system.

For instance, if type1 is PACKAGE, idx1 is 2, type2 is CORE and idx2 is 3, return the fourth core object below the third package.

Note

This function requires these objects to have a CPU set.

Returns:: a matching object if any, NULL otherwise.

inline hwloc_obj_t hwloc_get_obj_below_array_by_type(hwloc_topology_t topology, int nr, hwloc_obj_type_t *typev, unsigned *idxv)¶

Find an object below a chain of objects specified by types and indexes.

This is a generalized version of hwloc_get_obj_below_by_type().

Arrays typev and idxv must contain nr types and indexes.

Start from the top system object and walk the arrays typev and idxv. For each type and logical index couple in the arrays, look under the previously found object to find the index-th object of the given type. Indexes are specified within the parent, not withing the entire system.

For instance, if nr is 3, typev contains NODE, PACKAGE and CORE, and idxv contains 0, 1 and 2, return the third core object below the second package below the first NUMA node.

Note

This function requires all these objects and the root object to have a CPU set.

Returns:: a matching object if any, NULL otherwise.

hwloc_obj_t hwloc_get_obj_with_same_locality(hwloc_topology_t topology, hwloc_obj_t src, hwloc_obj_type_t type, const char *subtype, const char *nameprefix, unsigned long flags)¶

Return an object of a different type with same locality.

If the source object src is a normal or memory type, this function returns an object of type type with same CPU and node sets, either below or above in the hierarchy.

If the source object src is a PCI or an OS device within a PCI device, the function may either return that PCI device, or another OS device in the same PCI parent. This may for instance be useful for converting between OS devices such as “nvml0” or “rsmi1” used in distance structures into the the PCI device, or the CUDA or OpenCL OS device that correspond to the same physical card.

If not NULL, parameter subtype only select objects whose subtype attribute exists and is subtype (case-insensitively), for instance “OpenCL” or “CUDA”.

If not NULL, parameter nameprefix only selects objects whose name attribute exists and starts with nameprefix (case-insensitively), for instance “rsmi” for matching “rsmi0”.

If multiple objects match, the first one is returned.

This function will not walk the hierarchy across bridges since the PCI locality may become different. This function cannot also convert between normal/memory objects and I/O or Misc objects.

flags must be 0 for now.

Returns:: An object with identical locality, matching subtype and nameprefix if any.
Returns:: NULL if no matching object could be found, or if the source object and target type are incompatible, for instance if converting between CPU and I/O objects.

Distributing items over a topology¶

group Distributing items over a topology

Enums

enum hwloc_distrib_flags_e¶

Flags to be given to hwloc_distrib().

Values:

enumerator HWLOC_DISTRIB_FLAG_REVERSE¶: Distrib in reverse order, starting from the last objects.

Functions

inline int hwloc_distrib(hwloc_topology_t topology, hwloc_obj_t *roots, unsigned n_roots, hwloc_cpuset_t *set, unsigned n, int until, unsigned long flags)¶

Distribute n items over the topology under roots.

Array set will be filled with n cpusets recursively distributed linearly over the topology under objects roots, down to depth until (which can be INT_MAX to distribute down to the finest level).

n_roots is usually 1 and roots only contains the topology root object so as to distribute over the entire topology.

This is typically useful when an application wants to distribute n threads over a machine, giving each of them as much private cache as possible and keeping them locally in number order.

The caller may typically want to also call hwloc_bitmap_singlify() before binding a thread so that it does not move at all.

flags should be 0 or a OR’ed set of hwloc_distrib_flags_e.

Note

On hybrid CPUs (or asymmetric platforms), distribution may be suboptimal since the number of cores or PUs inside packages or below caches may vary (the top-down recursive partitioning ignores these numbers until reaching their levels). Hence it is recommended to distribute only inside a single homogeneous domain. For instance on a CPU with energy-efficient E-cores and high-performance P-cores, one should distribute separately N tasks on E-cores and M tasks on P-cores instead of trying to distribute directly M+N tasks on the entire CPUs.

Note

This function requires the roots objects to have a CPU set.

Returns:: 0 on success, -1 on error.

CPU and node sets of entire topologies¶

group CPU and node sets of entire topologies

Functions

hwloc_const_cpuset_t hwloc_topology_get_complete_cpuset(hwloc_topology_t topology)¶

Get complete CPU set.

Note

This function cannot return NULL.

Note

The returned cpuset is not newly allocated and should thus not be changed or freed; hwloc_bitmap_dup() must be used to obtain a local copy.

Note

This is equivalent to retrieving the root object complete CPU-set.

Returns:: the complete CPU set of processors of the system.

hwloc_const_cpuset_t hwloc_topology_get_topology_cpuset(hwloc_topology_t topology)¶

Get topology CPU set.

Note

This function cannot return NULL.

Note

The returned cpuset is not newly allocated and should thus not be changed or freed; hwloc_bitmap_dup() must be used to obtain a local copy.

Note

This is equivalent to retrieving the root object CPU-set.

Returns:: the CPU set of processors of the system for which hwloc provides topology information. This is equivalent to the cpuset of the system object.

hwloc_const_cpuset_t hwloc_topology_get_allowed_cpuset(hwloc_topology_t topology)¶

Get allowed CPU set.

Note

This function cannot return NULL.

Note

If the topology flag HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED was not set, this is identical to hwloc_topology_get_topology_cpuset(), which means all PUs are allowed.

Note

If HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED was set, applying hwloc_bitmap_intersects() on the result of this function and on an object cpuset checks whether there are allowed PUs inside that object. Applying hwloc_bitmap_and() returns the list of these allowed PUs.

Note

The returned cpuset is not newly allocated and should thus not be changed or freed, hwloc_bitmap_dup() must be used to obtain a local copy.

Returns:: the CPU set of allowed processors of the system.

hwloc_const_nodeset_t hwloc_topology_get_complete_nodeset(hwloc_topology_t topology)¶

Get complete node set.

Note

This function cannot return NULL.

Note

The returned nodeset is not newly allocated and should thus not be changed or freed; hwloc_bitmap_dup() must be used to obtain a local copy.

Note

This is equivalent to retrieving the root object complete nodeset.

Returns:: the complete node set of memory of the system.

hwloc_const_nodeset_t hwloc_topology_get_topology_nodeset(hwloc_topology_t topology)¶

Get topology node set.

Note

This function cannot return NULL.

Note

The returned nodeset is not newly allocated and should thus not be changed or freed; hwloc_bitmap_dup() must be used to obtain a local copy.

Note

This is equivalent to retrieving the root object nodeset.

Returns:: the node set of memory of the system for which hwloc provides topology information. This is equivalent to the nodeset of the system object.

hwloc_const_nodeset_t hwloc_topology_get_allowed_nodeset(hwloc_topology_t topology)¶

Get allowed node set.

Note

This function cannot return NULL.

Note

If the topology flag HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED was not set, this is identical to hwloc_topology_get_topology_nodeset(), which means all NUMA nodes are allowed.

Note

If HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED was set, applying hwloc_bitmap_intersects() on the result of this function and on an object nodeset checks whether there are allowed NUMA nodes inside that object. Applying hwloc_bitmap_and() returns the list of these allowed NUMA nodes.

Note

The returned nodeset is not newly allocated and should thus not be changed or freed, hwloc_bitmap_dup() must be used to obtain a local copy.

Returns:: the node set of allowed memory of the system.

Converting between CPU sets and node sets¶

group Converting between CPU sets and node sets

Functions

inline int hwloc_cpuset_to_nodeset(hwloc_topology_t topology, hwloc_const_cpuset_t _cpuset, hwloc_nodeset_t nodeset)¶

Convert a CPU set into a NUMA node set.

For each PU included in the input _cpuset, set the corresponding local NUMA node(s) in the output nodeset.

If some NUMA nodes have no CPUs at all, this function never sets their indexes in the output node set, even if a full CPU set is given in input.

Hence the entire topology CPU set is converted into the set of all nodes that have some local CPUs.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOMEM on internal reallocation failure.

inline int hwloc_cpuset_from_nodeset(hwloc_topology_t topology, hwloc_cpuset_t _cpuset, hwloc_const_nodeset_t nodeset)¶

Convert a NUMA node set into a CPU set.

For each NUMA node included in the input nodeset, set the corresponding local PUs in the output _cpuset.

If some CPUs have no local NUMA nodes, this function never sets their indexes in the output CPU set, even if a full node set is given in input.

Hence the entire topology node set is converted into the set of all CPUs that have some local NUMA nodes.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOMEM on internal reallocation failure.

Finding I/O objects¶

group Finding I/O objects

Functions

inline hwloc_obj_t hwloc_get_non_io_ancestor_obj(hwloc_topology_t topology, hwloc_obj_t ioobj)¶

Get the first non-I/O ancestor object.

Given the I/O object ioobj, find the smallest non-I/O ancestor object. This object (normal or memory) may then be used for binding because it has non-NULL CPU and node sets and because its locality is the same as ioobj.

Note

This function cannot return NULL.

Note

The resulting object is usually a normal object but it could also be a memory object (e.g. NUMA node) in future platforms if I/O objects ever get attached to memory instead of CPUs.

Returns:: a non-I/O object.

inline hwloc_obj_t hwloc_get_next_pcidev(hwloc_topology_t topology, hwloc_obj_t prev)¶

Get the next PCI device in the system.

Returns:: the first PCI device if prev is NULL.
Returns:: the next PCI device if prev is not NULL.
Returns:: NULL if there is no next PCI device.

inline hwloc_obj_t hwloc_get_pcidev_by_busid(hwloc_topology_t topology, unsigned domain, unsigned bus, unsigned dev, unsigned func)¶

Find the PCI device object matching the PCI bus id given domain, bus device and function PCI bus id.

Returns:: a matching PCI device object if any, NULL otherwise.

inline hwloc_obj_t hwloc_get_pcidev_by_busidstring(hwloc_topology_t topology, const char *busid)¶

Find the PCI device object matching the PCI bus id given as a string xxxx:yy:zz.t or yy:zz.t.

Returns:: a matching PCI device object if any, NULL otherwise.

inline hwloc_obj_t hwloc_get_next_osdev(hwloc_topology_t topology, hwloc_obj_t prev)¶

Get the next OS device in the system.

Returns:: the first OS device if prev is NULL.
Returns:: the next OS device if prev is not NULL.
Returns:: NULL if there is no next OS device.

inline hwloc_obj_t hwloc_get_next_bridge(hwloc_topology_t topology, hwloc_obj_t prev)¶

Get the next bridge in the system.

Returns:: the first bridge if prev is NULL.
Returns:: the next bridge if prev is not NULL.
Returns:: NULL if there is no next bridge.

inline int hwloc_bridge_covers_pcibus(hwloc_obj_t bridge, unsigned domain, unsigned bus)¶

The bitmap API¶

group The bitmap API

The hwloc_bitmap_t type represents a set of integers (positive or null). A bitmap may be of infinite size (all bits are set after some point). A bitmap may even be full if all bits are set.

Bitmaps are used by hwloc for sets of OS processors (which may actually be hardware threads) as by hwloc_cpuset_t (a typedef for hwloc_bitmap_t), or sets of NUMA memory nodes as hwloc_nodeset_t (also a typedef for hwloc_bitmap_t). Those are used for cpuset and nodeset fields in the hwloc_obj structure, see Object Sets (hwloc_cpuset_t and hwloc_nodeset_t).

Both CPU and node sets are always indexed by OS physical number. However users should usually not build CPU and node sets manually (e.g. with hwloc_bitmap_set()). One should rather use existing object sets and combine them with hwloc_bitmap_or(), etc. For instance, binding the current thread on a pair of cores may be performed with:

hwloc_obj_t core1 = ... , core2 = ... ;
hwloc_bitmap_t set = hwloc_bitmap_alloc();
hwloc_bitmap_or(set, core1->cpuset, core2->cpuset);
hwloc_set_cpubind(topology, set, HWLOC_CPUBIND_THREAD);
hwloc_bitmap_free(set);

Note

Most functions below return 0 on success and -1 on error. The usual error case would be an internal failure to realloc/extend the storage of the bitmap (errno would be set to ENOMEM). See also Error reporting in the API.

Note

Several examples of using the bitmap API are available under the doc/examples/ directory in the source tree. Regression tests such as tests/hwloc/hwloc_bitmap*.c also make intensive use of this API.

Defines

hwloc_bitmap_foreach_begin(id, bitmap)¶

Loop macro iterating on bitmap bitmap.

The loop must start with hwloc_bitmap_foreach_begin() and end with hwloc_bitmap_foreach_end() followed by a terminating ‘;’.

id is the loop variable; it should be an unsigned int. The first iteration will set id to the lowest index in the bitmap. Successive iterations will iterate through, in order, all remaining indexes set in the bitmap. To be specific: each iteration will return a value for id such that hwloc_bitmap_isset(bitmap, id) is true.

The assert prevents the loop from being infinite if the bitmap is infinitely set.

hwloc_bitmap_foreach_end()¶

End of loop macro iterating on a bitmap.

Needs a terminating ‘;’.

See also

hwloc_bitmap_foreach_begin()

Typedefs

typedef struct hwloc_bitmap_s *hwloc_bitmap_t¶: Set of bits represented as an opaque pointer to an internal bitmap.

typedef const struct hwloc_bitmap_s *hwloc_const_bitmap_t¶: a non-modifiable hwloc_bitmap_t

Functions

hwloc_bitmap_t hwloc_bitmap_alloc(void)¶

Allocate a new empty bitmap.

The bitmap should be freed by a corresponding call to hwloc_bitmap_free().

Returns:: A valid bitmap or NULL.

hwloc_bitmap_t hwloc_bitmap_alloc_full(void)¶

Allocate a new full bitmap.

The bitmap should be freed by a corresponding call to hwloc_bitmap_free().

Returns:: A valid bitmap or NULL.

void hwloc_bitmap_free(hwloc_bitmap_t bitmap)¶

Free bitmap bitmap.

If bitmap is NULL, no operation is performed.

hwloc_bitmap_t hwloc_bitmap_dup(hwloc_const_bitmap_t bitmap)¶

Duplicate bitmap bitmap by allocating a new bitmap and copying bitmap contents.

If bitmap is NULL, NULL is returned.

int hwloc_bitmap_copy(hwloc_bitmap_t dst, hwloc_const_bitmap_t src)¶: Copy the contents of bitmap src into the already allocated bitmap dst.

int hwloc_bitmap_snprintf(char *restrict buf, size_t buflen, hwloc_const_bitmap_t bitmap)¶

Stringify a bitmap in the default hwloc format.

Print the bits set inside a bitmap as a comma-separated list of hexadecimal 32-bit blocks. A bitmap containing bits 1, 33, 34, and all from 64 to 95 is printed as "0xffffffff,0x00000006,0x00000002".

Up to buflen characters may be written in buffer buf.

If buflen is 0, buf may safely be NULL.

Note

If the bitmap is a CPU or nodeset, it contains physical indexes. This should be clearly indicated when displaying such bitmaps to end users. See also How do I convert between logical and OS/physical indexes?

Returns:: the number of characters that were actually written if not truncating, or that would have been written (not including the ending \0).
Returns:: -1 on error.

int hwloc_bitmap_asprintf(char **strp, hwloc_const_bitmap_t bitmap)¶

Stringify a bitmap into a newly allocated string in the default hwloc format.

Print the bits set inside a bitmap as a comma-separated list of hexadecimal 32-bit blocks. A bitmap containing bits 1, 33, 34, and all from 64 to 95 is printed as "0xffffffff,0x00000006,0x00000002".

Note

If the bitmap is a CPU or nodeset, it contains physical indexes. This should be clearly indicated when displaying such bitmaps to end users. See also How do I convert between logical and OS/physical indexes?

Returns:: the number of characters that were written (not including the ending \0).
Returns:: -1 on error, for instance with errno set to ENOMEM on failure to allocate the output string.

int hwloc_bitmap_sscanf(hwloc_bitmap_t bitmap, const char *restrict string)¶

Parse a bitmap string as the default hwloc format and stores it in bitmap bitmap.

The input string should be a comma-separared list of hexadecimal 32-bit blocks. String "0xffffffff,0x6,0x2" is parsed as a bitmap containing all bits between 64 and 95, and bits 33, 34 and 1.

Note

If the bitmap is a CPU or nodeset, the input string must contain physical indexes.

Returns:: 0 on success, -1 on error.

int hwloc_bitmap_list_snprintf(char *restrict buf, size_t buflen, hwloc_const_bitmap_t bitmap)¶

Stringify a bitmap in the list format.

Lists are comma-separated indexes or ranges. Ranges are dash separated indexes. A bitmap containing bits 1, 33, 34, and all from 64 to 95 is printed as "1,33-34,64-95". The last range may not have an ending index if the bitmap is infinitely set.

Up to buflen characters may be written in buffer buf.

If buflen is 0, buf may safely be NULL.

Note

If the bitmap is a CPU or nodeset, it contains physical indexes. This should be clearly indicated when displaying such bitmaps to end users. See also How do I convert between logical and OS/physical indexes?

Returns:: the number of characters that were actually written if not truncating, or that would have been written (not including the ending \0).
Returns:: -1 on error.

int hwloc_bitmap_list_asprintf(char **strp, hwloc_const_bitmap_t bitmap)¶

Stringify a bitmap into a newly allocated list string.

Lists are comma-separated indexes or ranges. Ranges are dash separated indexes. A bitmap containing bits 1, 33, 34, and all from 64 to 95 is printed as "1,33-34,64-95". The last range may not have an ending index if the bitmap is infinitely set.

Note

If the bitmap is a CPU or nodeset, it contains physical indexes. This should be clearly indicated when displaying such bitmaps to end users. See also How do I convert between logical and OS/physical indexes?

Returns:: the number of characters that were written (not including the ending \0).
Returns:: -1 on error, for instance with errno set to ENOMEM on failure to allocate the output string.

int hwloc_bitmap_list_sscanf(hwloc_bitmap_t bitmap, const char *restrict string)¶

Parse a list string and stores it in bitmap bitmap.

Lists are comma-separated indexes or ranges. Ranges are dash separated indexes. String "1,33-34,64-95" is parsed as a bitmap containing bits 1, 33, 34, and all from 64 to 95. The last range may not have an ending index if the bitmap is infinitely set.

Note

If the bitmap is a CPU or nodeset, the input string must contain physical indexes.

Returns:: 0 on success, -1 on error.

int hwloc_bitmap_taskset_snprintf(char *restrict buf, size_t buflen, hwloc_const_bitmap_t bitmap)¶

Stringify a bitmap in the taskset-specific format.

The taskset program manipulates bitmap strings that contain a single (possible very long) hexadecimal number starting with 0x. A bitmap containing bits 1, 33, 34, and all from 64 to 95 is printed as "0xffffffff0000000600000002".

Up to buflen characters may be written in buffer buf.

If buflen is 0, buf may safely be NULL.

Note

If the bitmap is a CPU or nodeset, it contains physical indexes. This should be clearly indicated when displaying such bitmaps to end users. See also How do I convert between logical and OS/physical indexes?

Returns:: the number of characters that were actually written if not truncating, or that would have been written (not including the ending \0).
Returns:: -1 on error.

int hwloc_bitmap_taskset_asprintf(char **strp, hwloc_const_bitmap_t bitmap)¶

Stringify a bitmap into a newly allocated taskset-specific string.

The taskset program manipulates bitmap strings that contain a single (possible very long) hexadecimal number starting with 0x. A bitmap containing bits 1, 33, 34, and all from 64 to 95 is printed as "0xffffffff0000000600000002".

Note

If the bitmap is a CPU or nodeset, it contains physical indexes. This should be clearly indicated when displaying such bitmaps to end users. See also How do I convert between logical and OS/physical indexes?

Returns:: the number of characters that were written (not including the ending \0).
Returns:: -1 on error, for instance with errno set to ENOMEM on failure to allocate the output string.

int hwloc_bitmap_taskset_sscanf(hwloc_bitmap_t bitmap, const char *restrict string)¶

Parse a taskset-specific bitmap string and stores it in bitmap bitmap.

The taskset program manipulates bitmap strings that contain a single (possible very long) hexadecimal number starting with 0x. String "0xffffffff0000000600000002" is parsed as a bitmap containing all bits between 64 and 95, and bits 33, 34 and 1.

Note

If the bitmap is a CPU or nodeset, the input string must contain physical indexes.

Returns:: 0 on success, -1 on error.

void hwloc_bitmap_zero(hwloc_bitmap_t bitmap)¶: Empty the bitmap bitmap.

void hwloc_bitmap_fill(hwloc_bitmap_t bitmap)¶: Fill bitmap bitmap with all possible indexes (even if those objects don’t exist or are otherwise unavailable).

int hwloc_bitmap_only(hwloc_bitmap_t bitmap, unsigned id)¶: Empty the bitmap bitmap and add bit id.

int hwloc_bitmap_allbut(hwloc_bitmap_t bitmap, unsigned id)¶: Fill the bitmap and clear the index id.

int hwloc_bitmap_from_ulong(hwloc_bitmap_t bitmap, unsigned long mask)¶: Setup bitmap bitmap from unsigned long mask.

int hwloc_bitmap_from_ith_ulong(hwloc_bitmap_t bitmap, unsigned i, unsigned long mask)¶: Setup bitmap bitmap from unsigned long mask used as i -th subset.

int hwloc_bitmap_from_ulongs(hwloc_bitmap_t bitmap, unsigned nr, const unsigned long *masks)¶: Setup bitmap bitmap from unsigned longs masks used as first nr subsets.

int hwloc_bitmap_set(hwloc_bitmap_t bitmap, unsigned id)¶: Add index id in bitmap bitmap.

int hwloc_bitmap_set_range(hwloc_bitmap_t bitmap, unsigned begin, int end)¶

Add indexes from begin to end in bitmap bitmap.

If end is -1, the range is infinite.

int hwloc_bitmap_set_ith_ulong(hwloc_bitmap_t bitmap, unsigned i, unsigned long mask)¶: Replace i -th subset of bitmap bitmap with unsigned long mask.

int hwloc_bitmap_clr(hwloc_bitmap_t bitmap, unsigned id)¶: Remove index id from bitmap bitmap.

int hwloc_bitmap_clr_range(hwloc_bitmap_t bitmap, unsigned begin, int end)¶

Remove indexes from begin to end in bitmap bitmap.

If end is -1, the range is infinite.

int hwloc_bitmap_singlify(hwloc_bitmap_t bitmap)¶

Keep a single index among those set in bitmap bitmap.

May be useful before binding so that the process does not have a chance of migrating between multiple processors in the original mask. Instead of running the task on any PU inside the given CPU set, the operating system scheduler will be forced to run it on a single of these PUs. It avoids a migration overhead and cache-line ping-pongs between PUs.

Note

This function is NOT meant to distribute multiple processes within a single CPU set. It always return the same single bit when called multiple times on the same input set. hwloc_distrib() may be used for generating CPU sets to distribute multiple tasks below a single multi-PU object.

Note

This function cannot be applied to an object set directly. It should be applied to a copy (which may be obtained with hwloc_bitmap_dup()).

unsigned long hwloc_bitmap_to_ulong(hwloc_const_bitmap_t bitmap)¶: Convert the beginning part of bitmap bitmap into unsigned long mask.

unsigned long hwloc_bitmap_to_ith_ulong(hwloc_const_bitmap_t bitmap, unsigned i)¶: Convert the i -th subset of bitmap bitmap into unsigned long mask.

int hwloc_bitmap_to_ulongs(hwloc_const_bitmap_t bitmap, unsigned nr, unsigned long *masks)¶

Convert the first nr subsets of bitmap bitmap into the array of nr unsigned long masks.

nr may be determined earlier with hwloc_bitmap_nr_ulongs().

Returns:: 0

int hwloc_bitmap_nr_ulongs(hwloc_const_bitmap_t bitmap)¶

Return the number of unsigned longs required for storing bitmap bitmap entirely.

This is the number of contiguous unsigned longs from the very first bit of the bitmap (even if unset) up to the last set bit. This is useful for knowing the nr parameter to pass to hwloc_bitmap_to_ulongs() (or which calls to hwloc_bitmap_to_ith_ulong() are needed) to entirely convert a bitmap into multiple unsigned longs.

When called on the output of hwloc_topology_get_topology_cpuset(), the returned number is large enough for all cpusets of the topology.

Returns:: the number of unsigned longs required.
Returns:: -1 if bitmap is infinite.

int hwloc_bitmap_isset(hwloc_const_bitmap_t bitmap, unsigned id)¶

Test whether index id is part of bitmap bitmap.

Returns:: 1 if the bit at index id is set in bitmap bitmap, 0 otherwise.

int hwloc_bitmap_iszero(hwloc_const_bitmap_t bitmap)¶

Test whether bitmap bitmap is empty.

Returns:: 1 if bitmap is empty, 0 otherwise.

int hwloc_bitmap_isfull(hwloc_const_bitmap_t bitmap)¶

Test whether bitmap bitmap is completely full.

Note

A full bitmap is always infinitely set.

Returns:: 1 if bitmap is full, 0 otherwise.

int hwloc_bitmap_first(hwloc_const_bitmap_t bitmap)¶

Compute the first index (least significant bit) in bitmap bitmap.

Returns:: the first index set in bitmap.
Returns:: -1 if bitmap is empty.

int hwloc_bitmap_next(hwloc_const_bitmap_t bitmap, int prev)¶

Compute the next index in bitmap bitmap which is after index prev.

Returns:: the first index set in bitmap if prev is -1.
Returns:: the next index set in bitmap if prev is not -1.
Returns:: -1 if no index with higher index is set in bitmap.

int hwloc_bitmap_last(hwloc_const_bitmap_t bitmap)¶

Compute the last index (most significant bit) in bitmap bitmap.

Returns:: the last index set in bitmap.
Returns:: -1 if bitmap is empty, or if bitmap is infinitely set.

int hwloc_bitmap_weight(hwloc_const_bitmap_t bitmap)¶

Compute the “weight” of bitmap bitmap (i.e., number of indexes that are in the bitmap).

Returns:: the number of indexes that are in the bitmap.
Returns:: -1 if bitmap is infinitely set.

int hwloc_bitmap_first_unset(hwloc_const_bitmap_t bitmap)¶

Compute the first unset index (least significant bit) in bitmap bitmap.

Returns:: the first unset index in bitmap.
Returns:: -1 if bitmap is full.

int hwloc_bitmap_next_unset(hwloc_const_bitmap_t bitmap, int prev)¶

Compute the next unset index in bitmap bitmap which is after index prev.

Returns:: the first index unset in bitmap if prev is -1.
Returns:: the next index unset in bitmap if prev is not -1.
Returns:: -1 if no index with higher index is unset in bitmap.

int hwloc_bitmap_last_unset(hwloc_const_bitmap_t bitmap)¶

Compute the last unset index (most significant bit) in bitmap bitmap.

Returns:: the last index unset in bitmap.
Returns:: -1 if bitmap is full, or if bitmap is not infinitely set.

int hwloc_bitmap_or(hwloc_bitmap_t res, hwloc_const_bitmap_t bitmap1, hwloc_const_bitmap_t bitmap2)¶

Or bitmaps bitmap1 and bitmap2 and store the result in bitmap res.

res can be the same as bitmap1 or bitmap2

int hwloc_bitmap_and(hwloc_bitmap_t res, hwloc_const_bitmap_t bitmap1, hwloc_const_bitmap_t bitmap2)¶

And bitmaps bitmap1 and bitmap2 and store the result in bitmap res.

res can be the same as bitmap1 or bitmap2

int hwloc_bitmap_andnot(hwloc_bitmap_t res, hwloc_const_bitmap_t bitmap1, hwloc_const_bitmap_t bitmap2)¶

And bitmap bitmap1 and the negation of bitmap2 and store the result in bitmap res.

res can be the same as bitmap1 or bitmap2

int hwloc_bitmap_xor(hwloc_bitmap_t res, hwloc_const_bitmap_t bitmap1, hwloc_const_bitmap_t bitmap2)¶

Xor bitmaps bitmap1 and bitmap2 and store the result in bitmap res.

res can be the same as bitmap1 or bitmap2

int hwloc_bitmap_not(hwloc_bitmap_t res, hwloc_const_bitmap_t bitmap)¶

Negate bitmap bitmap and store the result in bitmap res.

res can be the same as bitmap

int hwloc_bitmap_intersects(hwloc_const_bitmap_t bitmap1, hwloc_const_bitmap_t bitmap2)¶

Test whether bitmaps bitmap1 and bitmap2 intersects.

Note

The empty bitmap does not intersect any other bitmap.

Returns:: 1 if bitmaps intersect, 0 otherwise.

int hwloc_bitmap_isincluded(hwloc_const_bitmap_t sub_bitmap, hwloc_const_bitmap_t super_bitmap)¶

Test whether bitmap sub_bitmap is part of bitmap super_bitmap.

Note

The empty bitmap is considered included in any other bitmap.

Returns:: 1 if sub_bitmap is included in super_bitmap, 0 otherwise.

int hwloc_bitmap_isequal(hwloc_const_bitmap_t bitmap1, hwloc_const_bitmap_t bitmap2)¶

Test whether bitmap bitmap1 is equal to bitmap bitmap2.

Returns:: 1 if bitmaps are equal, 0 otherwise.

int hwloc_bitmap_compare_first(hwloc_const_bitmap_t bitmap1, hwloc_const_bitmap_t bitmap2)¶

Compare bitmaps bitmap1 and bitmap2 using their lowest index.

A bitmap is considered smaller if its least significant bit is smaller. The empty bitmap is considered higher than anything (because its least significant bit does not exist).

For instance comparing binary bitmaps 0011 and 0110 returns -1 (hence 0011 is considered smaller than 0110) because least significant bit of 0011 (0001) is smaller than least significant bit of 0110 (0010). Comparing 01001 and 00110 would also return -1 for the same reason.

Returns:: -1 if bitmap1 is considered smaller than bitmap2.
Returns:: 1 if bitmap1 is considered larger than bitmap2.
Returns:: 0 if bitmaps are considered equal, even if they are not strictly equal. They just need to have the same least significant bit. For instance, comparing binary bitmaps 0010 and 0110 returns 0 because they have the same least significant bit.

int hwloc_bitmap_compare(hwloc_const_bitmap_t bitmap1, hwloc_const_bitmap_t bitmap2)¶

Compare bitmaps bitmap1 and bitmap2 in lexicographic order.

Lexicographic comparison of bitmaps, starting for their highest indexes. Compare last indexes first, then second, etc. The empty bitmap is considered lower than anything.

For instance comparing binary bitmaps 0011 and 0110 returns -1 (hence 0011 is considered smaller than 0110). Comparing 00101 and 01010 returns -1 too.

Note

This is different from the non-existing hwloc_bitmap_compare_last() which would only compare the highest index of each bitmap.

Returns:: -1 if bitmap1 is considered smaller than bitmap2.
Returns:: 1 if bitmap1 is considered larger than bitmap2.
Returns:: 0 if bitmaps are equal (contrary to hwloc_bitmap_compare_first()).

Exporting Topologies to XML¶

group Exporting Topologies to XML

Enums

enum hwloc_topology_export_xml_flags_e¶

Flags for exporting XML topologies.

Flags to be given as a OR’ed set to hwloc_topology_export_xml().

Values:

enumerator HWLOC_TOPOLOGY_EXPORT_XML_FLAG_V2¶: Export XML that is loadable by hwloc v2.x. However, the export may miss some details about the topology.

Functions

int hwloc_topology_export_xml(hwloc_topology_t topology, const char *xmlpath, unsigned long flags)¶

Export the topology into an XML file.

This file may be loaded later through hwloc_topology_set_xml().

By default, the latest export format is used, which means older hwloc releases (e.g. v2.x) may not be able to import it. Exporting to v2.x specific XML format is possible using flag HWLOC_TOPOLOGY_EXPORT_XML_FLAG_V2 but it may miss some details about the topology. If there is any chance that the exported file may ever be imported back by a process using hwloc 2.x, one should consider detecting it at runtime and using the corresponding export format.

flags is a OR’ed set of hwloc_topology_export_xml_flags_e.

Note

See also hwloc_topology_set_userdata_export_callback() for exporting application-specific object userdata.

Note

The topology-specific userdata pointer is ignored when exporting to XML.

Note

Only printable characters may be exported to XML string attributes. Any other character, especially any non-ASCII character, will be silently dropped.

Note

If name is “-”, the XML output is sent to the standard output.

Returns:: 0 on success, or -1 on error.

int hwloc_topology_export_xmlbuffer(hwloc_topology_t topology, char **xmlbuffer, int *buflen, unsigned long flags)¶

Export the topology into a newly-allocated XML memory buffer.

xmlbuffer is allocated by the callee and should be freed with hwloc_free_xmlbuffer() later in the caller.

This memory buffer may be loaded later through hwloc_topology_set_xmlbuffer().

By default, the latest export format is used, which means older hwloc releases (e.g. v2.x) may not be able to import it. Exporting to v2.x specific XML format is possible using flag HWLOC_TOPOLOGY_EXPORT_XML_FLAG_V2 but it may miss some details about the topology. If there is any chance that the exported buffer may ever be imported back by a process using hwloc 2.x, one should consider detecting it at runtime and using the corresponding export format.

The returned buffer ends with a \0 that is included in the returned length.

flags is a OR’ed set of hwloc_topology_export_xml_flags_e.

Note

See also hwloc_topology_set_userdata_export_callback() for exporting application-specific object userdata.

Note

The topology-specific userdata pointer is ignored when exporting to XML.

Note

Only printable characters may be exported to XML string attributes. Any other character, especially any non-ASCII character, will be silently dropped.

Returns:: 0 on success, or -1 on error.

void hwloc_free_xmlbuffer(hwloc_topology_t topology, char *xmlbuffer)¶: Free a buffer allocated by hwloc_topology_export_xmlbuffer().

void hwloc_topology_set_userdata_export_callback(hwloc_topology_t topology, void (*export_cb)(void *reserved, hwloc_topology_t topology, hwloc_obj_t obj))¶

Set the application-specific callback for exporting object userdata.

The object userdata pointer is not exported to XML by default because hwloc does not know what it contains.

This function lets applications set export_cb to a callback function that converts this opaque userdata into an exportable string.

export_cb is invoked during XML export for each object whose userdata pointer is not NULL. The callback should use hwloc_export_obj_userdata() or hwloc_export_obj_userdata_base64() to actually export something to XML (possibly multiple times per object).

export_cb may be set to NULL if userdata should not be exported to XML.

Note

The topology-specific userdata pointer is ignored when exporting to XML.

int hwloc_export_obj_userdata(void *reserved, hwloc_topology_t topology, hwloc_obj_t obj, const char *name, const void *buffer, size_t length)¶

Export some object userdata to XML.

This function may only be called from within the export() callback passed to hwloc_topology_set_userdata_export_callback(). It may be invoked one of multiple times to export some userdata to XML. The buffer content of length length is stored with optional name name.

When importing this XML file, the import() callback (if set) will be called exactly as many times as hwloc_export_obj_userdata() was called during export(). It will receive the corresponding name, buffer and length arguments.

reserved, topology and obj must be the first three parameters that were given to the export callback.

Only printable characters may be exported to XML string attributes.

If exporting binary data, the application should first encode into printable characters only (or use hwloc_export_obj_userdata_base64()). It should also take care of portability issues if the export may be reimported on a different architecture.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if a non-printable character is passed in name or buffer.

int hwloc_export_obj_userdata_base64(void *reserved, hwloc_topology_t topology, hwloc_obj_t obj, const char *name, const void *buffer, size_t length)¶

Encode and export some object userdata to XML.

This function is similar to hwloc_export_obj_userdata() but it encodes the input buffer into printable characters before exporting. On import, decoding is automatically performed before the data is given to the import() callback if any.

This function may only be called from within the export() callback passed to hwloc_topology_set_userdata_export_callback().

The name must be made of printable characters for export to XML string attributes.

The function does not take care of portability issues if the export may be reimported on a different architecture.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if a non-printable character is passed in name.

void hwloc_topology_set_userdata_import_callback(hwloc_topology_t topology, void (*import_cb)(hwloc_topology_t topology, hwloc_obj_t obj, const char *name, const void *buffer, size_t length))¶

Set the application-specific callback for importing userdata.

On XML import, userdata is ignored by default because hwloc does not know how to store it in memory.

This function lets applications set import_cb to a callback function that will get the XML-stored userdata and store it in the object as expected by the application.

import_cb is called during hwloc_topology_load() as many times as hwloc_export_obj_userdata() was called during export. The topology is not entirely setup yet. Object attributes are ready to consult, but links between objects are not.

import_cb may be NULL if userdata should be ignored during import.

Note

buffer contains length characters followed by a null byte (‘\0’).

Note

This function should be called before hwloc_topology_load().

Note

The topology-specific userdata pointer is ignored when importing from XML.

Exporting Topologies to Synthetic¶

group Exporting Topologies to Synthetic

Enums

enum hwloc_topology_export_synthetic_flags_e¶

Flags for exporting synthetic topologies.

Flags to be given as a OR’ed set to hwloc_topology_export_synthetic().

Values:

enumerator HWLOC_TOPOLOGY_EXPORT_SYNTHETIC_FLAG_NO_EXTENDED_TYPES¶

Export extended types such as L2dcache as basic types such as Cache.

This is required if loading the synthetic description with hwloc < 1.9.

enumerator HWLOC_TOPOLOGY_EXPORT_SYNTHETIC_FLAG_NO_ATTRS¶

Do not export level attributes.

Ignore level attributes such as memory/cache sizes or PU indexes. This is required if loading the synthetic description with hwloc < 1.10.

enumerator HWLOC_TOPOLOGY_EXPORT_SYNTHETIC_FLAG_V1¶

Export the memory hierarchy as expected in hwloc 1.x.

Instead of attaching memory children to levels, export single NUMA node child as normal intermediate levels, when possible. This is required if loading the synthetic description with hwloc 1.x. However this may fail if some objects have multiple local NUMA nodes.

enumerator HWLOC_TOPOLOGY_EXPORT_SYNTHETIC_FLAG_IGNORE_MEMORY¶

Do not export memory information.

Only export the actual hierarchy of normal CPU-side objects and ignore where memory is attached. This is useful for when the hierarchy of CPUs is what really matters, but it behaves as if there was a single machine-wide NUMA node.

Functions

int hwloc_topology_export_synthetic(hwloc_topology_t topology, char *buffer, size_t buflen, unsigned long flags)¶

Export the topology as a synthetic string.

At most buflen characters will be written in buffer, including the terminating \0.

This exported string may be given back to hwloc_topology_set_synthetic().

flags is a OR’ed set of hwloc_topology_export_synthetic_flags_e.

Note

I/O and Misc children are ignored, the synthetic string only describes normal children.

Note

A 1024-byte buffer should be large enough for exporting topologies in the vast majority of cases.

Returns:: The number of characters that were written, not including the terminating \0.
Returns:: -1 if the topology could not be exported, for instance if it is not symmetric.

Retrieve distances between objects¶

group Retrieve distances between objects

Enums

enum hwloc_distances_kind_e¶

Kinds of distance matrices.

The kind attribute of struct hwloc_distances_s is a OR’ed set of kinds.

Each distance matrix may have only one kind among HWLOC_DISTANCES_KIND_FROM_* specifying where distance information comes from, and one kind among HWLOC_DISTANCES_KIND_VALUE_* specifying and exactly one kind HWLOC_DISTANCES_KIND_VALUE_* specifying whether values are latencies or bandwidths, etc.

Values:

enumerator HWLOC_DISTANCES_KIND_FROM_OS¶: These distances were obtained from the operating system or hardware.

enumerator HWLOC_DISTANCES_KIND_FROM_USER¶: These distances were provided by the user.

enumerator HWLOC_DISTANCES_KIND_VALUE_LATENCY¶: Distance values are similar to latencies between objects. Values are smaller for closer objects, hence minimal on the diagonal of the matrix (distance between an object and itself).

enumerator HWLOC_DISTANCES_KIND_VALUE_BANDWIDTH¶: Distance values are similar to bandwidths between objects. Values are higher for closer objects, hence maximal on the diagonal of the matrix (distance between an object and itself). Such values are currently ignored for distance-based grouping.

enumerator HWLOC_DISTANCES_KIND_VALUE_HOPS¶: Distance values are numbers of hops between objects. Values are smaller for closer objects, zero on the diagonal of the matrix (no hop between an object and itself).

enumerator HWLOC_DISTANCES_KIND_HETEROGENEOUS_TYPES¶: This distances structure covers objects of different types. This may apply to the “NVLinkBandwidth” structure in presence of a NVSwitch or POWER processor NVLink port.

enum hwloc_distances_transform_e¶

Transformations of distances structures.

Values:

enumerator HWLOC_DISTANCES_TRANSFORM_REMOVE_NULL¶

Remove NULL objects from the distances structure.

Every object that was replaced with NULL in the objs array is removed and the values array is updated accordingly.

At least 2 objects must remain, otherwise hwloc_distances_transform() will return -1 with errno set to EINVAL.

kind will be updated with or without HWLOC_DISTANCES_KIND_HETEROGENEOUS_TYPES according to the remaining objects.

enumerator HWLOC_DISTANCES_TRANSFORM_LINKS¶

Replace bandwidth values with a number of links.

Usually all values will be either 0 (no link) or 1 (one link). However some matrices could get larger values if some pairs of peers are connected by different numbers of links.

Values on the diagonal are set to 0.

This transformation only applies to bandwidth matrices.

enumerator HWLOC_DISTANCES_TRANSFORM_MERGE_SWITCH_PORTS¶

Merge switches with multiple ports into a single object.

This currently only applies to NVSwitches where GPUs seem connected to different switch ports. Switch ports must be objects with subtype “NVSwitch” as in the NVLinkBandwidth matrix.

This transformation will replace all ports with only the first one, now connected to all GPUs. Other ports are removed by applying HWLOC_DISTANCES_TRANSFORM_REMOVE_NULL internally.

enumerator HWLOC_DISTANCES_TRANSFORM_TRANSITIVE_CLOSURE¶

Apply a transitive closure to the matrix to connect objects across switches.

All pairs of GPUs will be reported as directly connected instead GPUs being only connected to switches.

Switch ports must be objects with subtype “NVSwitch” as in the NVLinkBandwidth matrix.

Functions

int hwloc_distances_get(hwloc_topology_t topology, unsigned *nr, struct hwloc_distances_s **distances, unsigned long kind, unsigned long flags)¶

Retrieve distance matrices.

Retrieve distance matrices from the topology into the distances array.

flags is currently unused, should be 0.

kind serves as a filter. If 0, all distance matrices are returned. If it contains some HWLOC_DISTANCES_KIND_FROM_*, only distance matrices whose kind matches one of these are returned. If it contains some HWLOC_DISTANCES_KIND_VALUE_*, only distance matrices whose kind matches one of these are returned.

On input, nr points to the number of distance matrices that may be stored in distances. On output, nr points to the number of distance matrices that were actually found, even if some of them couldn’t be stored in distances. Distance matrices that couldn’t be stored are ignored, but the function still returns success (0). The caller may find out by comparing the value pointed by nr before and after the function call.

Each distance matrix returned in the distances array should be released by the caller using hwloc_distances_release().

Returns:: 0 on success, -1 on error.

int hwloc_distances_get_by_depth(hwloc_topology_t topology, int depth, unsigned *nr, struct hwloc_distances_s **distances, unsigned long kind, unsigned long flags)¶

Retrieve distance matrices for object at a specific depth in the topology.

Identical to hwloc_distances_get() with the additional depth filter.

Returns:: 0 on success, -1 on error.

int hwloc_distances_get_by_type(hwloc_topology_t topology, hwloc_obj_type_t type, unsigned *nr, struct hwloc_distances_s **distances, unsigned long kind, unsigned long flags)¶

Retrieve distance matrices for object of a specific type.

Identical to hwloc_distances_get() with the additional type filter.

Returns:: 0 on success, -1 on error.

int hwloc_distances_get_by_name(hwloc_topology_t topology, const char *name, unsigned *nr, struct hwloc_distances_s **distances, unsigned long flags)¶

Retrieve a distance matrix with the given name.

Usually only one distances structure may match a given name.

The name of the most common structure is “NUMALatency”. Others include “XGMIBandwidth”, “XGMIHops”, “XeLinkBandwidth”, and “NVLinkBandwidth”.

Returns:: 0 on success, -1 on error.

const char *hwloc_distances_get_name(hwloc_topology_t topology, struct hwloc_distances_s *distances)¶

Get a description of what a distances structure contains.

For instance “NUMALatency” for hardware-provided NUMA distances (ACPI SLIT), or NULL if unknown.

Note

The returned name should not be freed by the caller, it belongs to the hwloc library.

Returns:: the constant string with the name of the distance structure.

void hwloc_distances_release(hwloc_topology_t topology, struct hwloc_distances_s *distances)¶: Release a distance matrix structure previously returned by hwloc_distances_get().

Note

This function is not required if the structure is removed with hwloc_distances_release_remove().

int hwloc_distances_transform(hwloc_topology_t topology, struct hwloc_distances_s *distances, enum hwloc_distances_transform_e transform, void *transform_attr, unsigned long flags)¶

Apply a transformation to a distances structure.

Modify a distances structure that was previously obtained with hwloc_distances_get() or one of its variants.

This modifies the local copy of the distances structures but does not modify the distances information stored inside the topology (retrieved by another call to hwloc_distances_get() or exported to XML). To do so, one should add a new distances structure with same name, kind, objects and values (see Add distances between objects) and then remove this old one with hwloc_distances_release_remove().

transform must be one of the transformations listed in hwloc_distances_transform_e.

These transformations may modify the contents of the objs or values arrays.

transform_attr must be NULL for now.

flags must be 0 for now.

Note

Objects in distances array objs may be directly modified in place without using hwloc_distances_transform(). One may use hwloc_get_obj_with_same_locality() to easily convert between similar objects of different types.

Returns:: 0 on success, -1 on error for instance if flags are invalid.

struct hwloc_distances_s¶

#include <distances.h>

Matrix of distances between a set of objects.

The most common matrix contains latencies between NUMA nodes (as reported in the System Locality Distance Information Table (SLIT) in the ACPI specification), which may or may not be physically accurate. It corresponds to the latency for accessing the memory of one node from a core in another node. The corresponding kind is HWLOC_DISTANCES_KIND_VALUE_LATENCY | HWLOC_DISTANCES_KIND_FROM_USER. The name of this distances structure is “NUMALatency”.

The matrix may also contain bandwidths or number of hops between random sets of objects, possibly provided by the user, as specified in the kind attribute. Others common distance structures include and “XGMIBandwidth”, “XGMIHops”, “XeLinkBandwidth” and “NVLinkBandwidth”.

Pointers objs and values should not be replaced, reallocated, freed, etc. However callers are allowed to modify kind as well as the contents of objs and values arrays. For instance, if there is a single NUMA node per Package, hwloc_get_obj_with_same_locality() may be used to convert between them and replace NUMA nodes in the objs array with the corresponding Packages. See also hwloc_distances_transform() for applying some transformations to the structure.

Helpers for consulting distance matrices¶

group Helpers for consulting distance matrices

Functions

inline int hwloc_distances_obj_index(struct hwloc_distances_s *distances, hwloc_obj_t obj)¶

Find the index of an object in a distances structure.

Returns:: the index of the object in the distances structure if any.
Returns:: -1 if object obj is not involved in structure distances.

inline int hwloc_distances_obj_pair_values(struct hwloc_distances_s *distances, hwloc_obj_t obj1, hwloc_obj_t obj2, hwloc_uint64_t *value1to2, hwloc_uint64_t *value2to1)¶

Find the values between two objects in a distance matrices.

The distance from obj1 to obj2 is stored in the value pointed by value1to2 and reciprocally.

Returns:: 0 on success.
Returns:: -1 if object obj1 or obj2 is not involved in structure distances.

Add distances between objects¶

group Add distances between objects

The usual way to add distances is:

hwloc_distances_add_handle_t handle;
int err = -1;
handle = hwloc_distances_add_create(topology, "name", kind, 0);
if (handle) {
  err = hwloc_distances_add_values(topology, handle, nbobjs, objs, values, 0);
  if (!err)
    err = hwloc_distances_add_commit(topology, handle, flags);
}

If err is 0 at the end, then addition was successful.

Typedefs

typedef void *hwloc_distances_add_handle_t¶: Handle to a new distances structure during its addition to the topology.

Enums

enum hwloc_distances_add_flag_e¶

Flags for adding a new distances to a topology.

Values:

enumerator HWLOC_DISTANCES_ADD_FLAG_GROUP¶: Try to group objects based on the newly provided distance information. Grouping is only performed when the distances structure contains latencies or hops, and when all objects are of the same type.

enumerator HWLOC_DISTANCES_ADD_FLAG_GROUP_INACCURATE¶: If grouping, consider the distance values as inaccurate and relax the comparisons during the grouping algorithms. The actual accuracy may be modified through the HWLOC_GROUPING_ACCURACY environment variable (see Environment Variables).

Functions

hwloc_distances_add_handle_t hwloc_distances_add_create(hwloc_topology_t topology, const char *name, unsigned long kind, unsigned long flags)¶

Create a new empty distances structure.

Create an empty distances structure to be filled with hwloc_distances_add_values() and then committed with hwloc_distances_add_commit().

Parameter name is optional, it may be NULL. Otherwise, it will be copied internally and may later be freed by the caller.

kind specifies the kind of distance as a OR’ed set of hwloc_distances_kind_e. Only one kind of meaning and one kind of provenance may be given if appropriate (e.g. HWLOC_DISTANCES_KIND_VALUE_BANDWIDTH and HWLOC_DISTANCES_KIND_FROM_USER). Kind HWLOC_DISTANCES_KIND_HETEROGENEOUS_TYPES will be automatically set according to objects having different types in hwloc_distances_add_values().

flags must be 0 for now.

Returns:: A hwloc_distances_add_handle_t that should then be passed to hwloc_distances_add_values() and hwloc_distances_add_commit().
Returns:: NULL on error.

int hwloc_distances_add_values(hwloc_topology_t topology, hwloc_distances_add_handle_t handle, unsigned nbobjs, hwloc_obj_t *objs, hwloc_uint64_t *values, unsigned long flags)¶

Specify the objects and values in a new empty distances structure.

Specify the objects and values for a new distances structure that was returned as a handle by hwloc_distances_add_create(). The structure must then be committed with hwloc_distances_add_commit().

The number of objects is nbobjs and the array of objects is objs. Distance values are stored as a one-dimension array in values. The distance from object i to object j is in slot i*nbobjs+j.

nbobjs must be at least 2.

Arrays objs and values will be copied internally, they may later be freed by the caller.

On error, the temporary distances structure and its content are destroyed.

flags must be 0 for now.

Returns:: 0 on success.
Returns:: -1 on error.

int hwloc_distances_add_commit(hwloc_topology_t topology, hwloc_distances_add_handle_t handle, unsigned long flags)¶

Commit a new distances structure.

This function finalizes the distances structure and inserts in it the topology.

Parameter handle was previously returned by hwloc_distances_add_create(). Then objects and values were specified with hwloc_distances_add_values().

flags configures the behavior of the function using an optional OR’ed set of hwloc_distances_add_flag_e. It may be used to request the grouping of existing objects based on distances.

On error, the temporary distances structure and its content are destroyed.

Returns:: 0 on success.
Returns:: -1 on error.

Remove distances between objects¶

group Remove distances between objects

Functions

int hwloc_distances_remove(hwloc_topology_t topology)¶

Remove all distance matrices from a topology.

Remove all distance matrices, either provided by the user or gathered through the OS.

If these distances were used to group objects, these additional Group objects are not removed from the topology.

Returns:: 0 on success, -1 on error.

int hwloc_distances_remove_by_depth(hwloc_topology_t topology, int depth)¶

Remove distance matrices for objects at a specific depth in the topology.

Identical to hwloc_distances_remove() but only applies to one level of the topology.

Returns:: 0 on success, -1 on error.

inline int hwloc_distances_remove_by_type(hwloc_topology_t topology, hwloc_obj_type_t type)¶

Remove distance matrices for objects of a specific type in the topology.

Identical to hwloc_distances_remove() but only applies to one level of the topology.

Returns:: 0 on success, -1 on error.

int hwloc_distances_release_remove(hwloc_topology_t topology, struct hwloc_distances_s *distances)¶

Release and remove the given distance matrice from the topology.

This function includes a call to hwloc_distances_release().

Returns:: 0 on success, -1 on error.

Comparing memory node attributes for finding where to allocate on¶

group Comparing memory node attributes for finding where to allocate on

Platforms with heterogeneous memory require ways to decide whether a buffer should be allocated on “fast” memory (such as HBM), “normal” memory (DDR) or even “slow” but large-capacity memory (non-volatile memory). These memory nodes are called “Targets” while the CPU accessing them is called the “Initiator”. Access performance depends on their locality (NUMA platforms) as well as the intrinsic performance of the targets (heterogeneous platforms).

The following attributes describe the performance of memory accesses from an Initiator to a memory Target, for instance their latency or bandwidth. Initiators performing these memory accesses are usually some PUs or Cores (described as a CPU set). Hence a Core may choose where to allocate a memory buffer by comparing the attributes of different target memory nodes nearby.

There are also some attributes that are system-wide. Their value does not depend on a specific initiator performing an access. The memory node Capacity is an example of such attribute without initiator.

One way to use this API is to start with a cpuset describing the Cores where a program is bound. The best target NUMA node for allocating memory in this program on these Cores may be obtained by passing this cpuset as an initiator to hwloc_memattr_get_best_target() with the relevant memory attribute. For instance, if the code is latency limited, use the Latency attribute.

A more flexible approach consists in getting the list of local NUMA nodes by passing this cpuset to hwloc_get_local_numanode_objs(). Attribute values for these nodes, if any, may then be obtained with hwloc_memattr_get_value() and manually compared with the desired criteria.

Memory attributes are also used internally to build Memory Tiers which provide an easy way to distinguish NUMA nodes of different kinds, as explained in Heterogeneous Memory.

Beside tiers, hwloc defines a set of “default” nodes where normal memory allocations should be made from (see hwloc_topology_get_default_nodeset()). This is also useful for dividing the machine into a set of non-overlapping NUMA domains, for instance for binding tasks per domain.

See also

An example is available in doc/examples/memory-attributes.c in the source tree.

Note

The API also supports specific objects as initiator, but it is currently not used internally by hwloc. Users may for instance use it to provide custom performance values for host memory accesses performed by GPUs.

Note

The interface actually also accepts targets that are not NUMA nodes.

Typedefs

typedef unsigned hwloc_memattr_id_t¶

A memory attribute identifier.

hwloc predefines some commonly-used attributes in hwloc_memattr_id_e. One may then dynamically register custom ones with hwloc_memattr_register(), they will be assigned IDs immediately after the predefined ones. See Managing memory attributes for more information about existing attribute IDs.

Enums

enum hwloc_memattr_id_e¶

Predefined memory attribute IDs. See hwloc_memattr_id_t for the generic definition of IDs for predefined or custom attributes.

Values:

enumerator HWLOC_MEMATTR_ID_CAPACITY¶

The “Capacity” is returned in bytes (local_memory attribute in objects).

Best capacity nodes are nodes with higher capacity.

No initiator is involved when looking at this attribute. The corresponding attribute flags are HWLOC_MEMATTR_FLAG_HIGHER_FIRST.

Capacity values may not be modified using hwloc_memattr_set_value().

enumerator HWLOC_MEMATTR_ID_LOCALITY¶

The “Locality” is returned as the number of PUs in that locality (e.g. the weight of its cpuset).

Best locality nodes are nodes with smaller locality (nodes that are local to very few PUs). Poor locality nodes are nodes with larger locality (nodes that are local to the entire machine).

No initiator is involved when looking at this attribute. The corresponding attribute flags are HWLOC_MEMATTR_FLAG_HIGHER_FIRST.

Locality values may not be modified using hwloc_memattr_set_value().

enumerator HWLOC_MEMATTR_ID_BANDWIDTH¶

The “Bandwidth” is returned in MiB/s, as seen from the given initiator location.

Best bandwidth nodes are nodes with higher bandwidth.

The corresponding attribute flags are HWLOC_MEMATTR_FLAG_HIGHER_FIRST and HWLOC_MEMATTR_FLAG_NEED_INITIATOR.

This is the average bandwidth for read and write accesses. If the platform provides individual read and write bandwidths but no explicit average value, hwloc computes and returns the average.

enumerator HWLOC_MEMATTR_ID_READ_BANDWIDTH¶

The “ReadBandwidth” is returned in MiB/s, as seen from the given initiator location.

Best bandwidth nodes are nodes with higher bandwidth.

The corresponding attribute flags are HWLOC_MEMATTR_FLAG_HIGHER_FIRST and HWLOC_MEMATTR_FLAG_NEED_INITIATOR.

enumerator HWLOC_MEMATTR_ID_WRITE_BANDWIDTH¶

The “WriteBandwidth” is returned in MiB/s, as seen from the given initiator location.

Best bandwidth nodes are nodes with higher bandwidth.

The corresponding attribute flags are HWLOC_MEMATTR_FLAG_HIGHER_FIRST and HWLOC_MEMATTR_FLAG_NEED_INITIATOR.

enumerator HWLOC_MEMATTR_ID_LATENCY¶

The “Latency” is returned as nanoseconds, as seen from the given initiator location.

Best latency nodes are nodes with smaller latency.

The corresponding attribute flags are HWLOC_MEMATTR_FLAG_LOWER_FIRST and HWLOC_MEMATTR_FLAG_NEED_INITIATOR.

This is the average latency for read and write accesses. If the platform provides individual read and write latencies but no explicit average value, hwloc computes and returns the average.

enumerator HWLOC_MEMATTR_ID_READ_LATENCY¶

The “ReadLatency” is returned as nanoseconds, as seen from the given initiator location.

Best latency nodes are nodes with smaller latency.

The corresponding attribute flags are HWLOC_MEMATTR_FLAG_LOWER_FIRST and HWLOC_MEMATTR_FLAG_NEED_INITIATOR.

enumerator HWLOC_MEMATTR_ID_WRITE_LATENCY¶

The “WriteLatency” is returned as nanoseconds, as seen from the given initiator location.

Best latency nodes are nodes with smaller latency.

The corresponding attribute flags are HWLOC_MEMATTR_FLAG_LOWER_FIRST and HWLOC_MEMATTR_FLAG_NEED_INITIATOR.

enumerator HWLOC_MEMATTR_ID_MAX¶: Sentinel value for predefined attributes. Dynamically registered custom attributes start here.

enum hwloc_location_type_e¶

Type of location.

Values:

enumerator HWLOC_LOCATION_TYPE_CPUSET¶: Location is given as a cpuset, in the location cpuset union field.

enumerator HWLOC_LOCATION_TYPE_OBJECT¶: Location is given as an object, in the location object union field.

enum hwloc_local_numanode_flag_e¶

Flags for selecting target NUMA nodes.

Values:

enumerator HWLOC_LOCAL_NUMANODE_FLAG_LARGER_LOCALITY¶: Select NUMA nodes whose locality is larger than the given cpuset. For instance, if a single PU (or its cpuset) is given in initiator, select all nodes close to the package that contains this PU.

enumerator HWLOC_LOCAL_NUMANODE_FLAG_SMALLER_LOCALITY¶: Select NUMA nodes whose locality is smaller than the given cpuset. For instance, if a package (or its cpuset) is given in initiator, also select nodes that are attached to only a half of that package.

enumerator HWLOC_LOCAL_NUMANODE_FLAG_INTERSECT_LOCALITY¶: Select NUMA nodes whose locality intersects the given cpuset. This includes larger and smaller localities as well as localities that are partially included. For instance, if the locality is one core of both packages, a NUMA node local to one package is neither larger nor smaller than this locality, but it intersects it.

enumerator HWLOC_LOCAL_NUMANODE_FLAG_ALL¶: Select all NUMA nodes in the topology. The initiator initiator is ignored.

Functions

int hwloc_memattr_get_by_name(hwloc_topology_t topology, const char *name, hwloc_memattr_id_t *id)¶

Return the identifier of the memory attribute with the given name.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if no such attribute exists.

int hwloc_get_local_numanode_objs(hwloc_topology_t topology, struct hwloc_location *location, unsigned *nr, hwloc_obj_t *nodes, unsigned long flags)¶

Return an array of local NUMA nodes.

By default only select the NUMA nodes whose locality is exactly the given location. More nodes may be selected if additional flags are given as a OR’ed set of hwloc_local_numanode_flag_e.

If location is given as an explicit object, its CPU set is used to find NUMA nodes with the corresponding locality. If the object does not have a CPU set (e.g. I/O object), the CPU parent (where the I/O object is attached) is used.

On input, nr points to the number of nodes that may be stored in the nodes array. On output, nr will be changed to the number of stored nodes, or the number of nodes that would have been stored if there were enough room.

Note

Some of these NUMA nodes may not have any memory attribute values and hence not be reported as actual targets in other functions.

Note

The number of NUMA nodes in the topology (obtained by hwloc_bitmap_weight() on the root object nodeset) may be used to allocate the nodes array.

Note

When an object CPU set is given as locality, for instance a Package, and when flags contain both HWLOC_LOCAL_NUMANODE_FLAG_LARGER_LOCALITY and HWLOC_LOCAL_NUMANODE_FLAG_SMALLER_LOCALITY, the returned array corresponds to the nodeset of that object.

Returns:: 0 on success or -1 on error.

int hwloc_topology_get_default_nodeset(hwloc_topology_t topology, hwloc_nodeset_t nodeset, unsigned long flags)¶

Return the set of default NUMA nodes.

In machines with heterogeneous memory, some NUMA nodes are considered the default ones, i.e. where basic allocations should be made from. These are usually DRAM nodes.

Other nodes may be reserved for specific use (I/O device memory, e.g. GPU memory), small but high performance (HBM), large but slow memory (NVM), etc. Buffers should usually not be allocated from there unless explicitly required.

This function fills nodeset with the bits of NUMA nodes considered default.

It is guaranteed that these nodes have non-intersecting CPU sets, i.e. cores may not have multiple local NUMA nodes anymore. Hence this may be used to iterate over the platform divided into separate NUMA localities, for instance for binding one task per NUMA domain.

Any core that had some local NUMA node(s) in the initial topology should still have one in the default nodeset. Corner cases where this would be wrong consist in asymmetric platforms with missing DRAM nodes, or topologies that were already restricted to less NUMA nodes.

The returned nodeset may be passed to hwloc_topology_restrict() with HWLOC_RESTRICT_FLAG_BYNODESET to remove all non-default nodes from the topology. The resulting topology will be easier to use when iterating over (now homogeneous) NUMA nodes.

The heuristics for finding default nodes relies on memory tiers and subtypes (see Heterogeneous Memory) as well as the assumption that hardware vendors list default nodes first in hardware tables.

flags must be 0 for now.

Note

The returned nodeset usually contains all nodes from a single memory tier, likely the DRAM one.

Note

The returned nodeset is included in the list of available nodes returned by hwloc_topology_get_topology_nodeset(). It is strictly smaller if the machine has heterogeneous memory.

Note

The heuristics may return a suboptimal set of nodes if hwloc could not guess memory types and/or if some default nodes were removed earlier from the topology (e.g. with hwloc_topology_restrict()).

Returns:: 0 on success.
Returns:: -1 on error.

int hwloc_memattr_get_value(hwloc_topology_t topology, hwloc_memattr_id_t attribute, hwloc_obj_t target_node, struct hwloc_location *initiator, unsigned long flags, hwloc_uint64_t *value)¶

Return an attribute value for a specific target NUMA node.

If the attribute does not relate to a specific initiator (it does not have the flag HWLOC_MEMATTR_FLAG_NEED_INITIATOR), location initiator is ignored and may be NULL.

target_node cannot be NULL. If attribute is HWLOC_MEMATTR_ID_CAPACITY, target_node must be a NUMA node. If it is HWLOC_MEMATTR_ID_LOCALITY, target_node must have a CPU set.

flags must be 0 for now.

Note

The initiator initiator should be of type HWLOC_LOCATION_TYPE_CPUSET when refering to accesses performed by CPU cores. HWLOC_LOCATION_TYPE_OBJECT is currently unused internally by hwloc, but users may for instance use it to provide custom information about host memory accesses performed by GPUs.

Returns:: 0 on success.
Returns:: -1 on error, for instance with errno set to EINVAL if flags are invalid or no such attribute exists.

int hwloc_memattr_get_best_target(hwloc_topology_t topology, hwloc_memattr_id_t attribute, struct hwloc_location *initiator, unsigned long flags, hwloc_obj_t *best_target, hwloc_uint64_t *value)¶

Return the best target NUMA node for the given attribute and initiator.

If the attribute does not relate to a specific initiator (it does not have the flag HWLOC_MEMATTR_FLAG_NEED_INITIATOR), location initiator is ignored and may be NULL.

If value is non NULL, the corresponding value is returned there.

If multiple targets have the same attribute values, only one is returned (and there is no way to clarify how that one is chosen). Applications that want to detect targets with identical/similar values, or that want to look at values for multiple attributes, should rather get all values using hwloc_memattr_get_value() and manually select the target they consider the best.

flags must be 0 for now.

Note

The initiator initiator should be of type HWLOC_LOCATION_TYPE_CPUSET when refering to accesses performed by CPU cores. HWLOC_LOCATION_TYPE_OBJECT is currently unused internally by hwloc, but users may for instance use it to provide custom information about host memory accesses performed by GPUs.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOENT if there are no matching targets.
Returns:: -1 with errno set to EINVAL if flags are invalid, or no such attribute exists.

int hwloc_memattr_get_best_initiator(hwloc_topology_t topology, hwloc_memattr_id_t attribute, hwloc_obj_t target_node, unsigned long flags, struct hwloc_location *best_initiator, hwloc_uint64_t *value)¶

Return the best initiator for the given attribute and target NUMA node.

If value is non NULL, the corresponding value is returned there.

If multiple initiators have the same attribute values, only one is returned (and there is no way to clarify how that one is chosen). Applications that want to detect initiators with identical/similar values, or that want to look at values for multiple attributes, should rather get all values using hwloc_memattr_get_value() and manually select the initiator they consider the best.

The returned initiator should not be modified or freed, it belongs to the topology.

target_node cannot be NULL.

flags must be 0 for now.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOENT if there are no matching initiators.
Returns:: -1 with errno set to EINVAL if the attribute does not relate to a specific initiator (it does not have the flag HWLOC_MEMATTR_FLAG_NEED_INITIATOR).

int hwloc_memattr_get_targets(hwloc_topology_t topology, hwloc_memattr_id_t attribute, struct hwloc_location *initiator, unsigned long flags, unsigned *nr, hwloc_obj_t *targets, hwloc_uint64_t *values)¶

Return the target NUMA nodes that have some values for a given attribute.

Return targets for the given attribute in the targets array (for the given initiator if any). If values is not NULL, the corresponding attribute values are stored in the array it points to.

On input, nr points to the number of targets that may be stored in the array targets (and values). On output, nr points to the number of targets (and values) that were actually found, even if some of them couldn’t be stored in the array. Targets that couldn’t be stored are ignored, but the function still returns success (0). The caller may find out by comparing the value pointed by nr before and after the function call.

The returned targets should not be modified or freed, they belong to the topology.

Argument initiator is ignored if the attribute does not relate to a specific initiator (it does not have the flag HWLOC_MEMATTR_FLAG_NEED_INITIATOR). Otherwise initiator may be non NULL to report only targets that have a value for that initiator.

flags must be 0 for now.

Note

This function is meant for tools and debugging (listing internal information) rather than for application queries. Applications should rather select useful NUMA nodes with hwloc_get_local_numanode_objs() and then look at their attribute values.

Note

The initiator initiator should be of type HWLOC_LOCATION_TYPE_CPUSET when referring to accesses performed by CPU cores. HWLOC_LOCATION_TYPE_OBJECT is currently unused internally by hwloc, but users may for instance use it to provide custom information about host memory accesses performed by GPUs.

Returns:: 0 on success or -1 on error.

int hwloc_memattr_get_initiators(hwloc_topology_t topology, hwloc_memattr_id_t attribute, hwloc_obj_t target_node, unsigned long flags, unsigned *nr, struct hwloc_location *initiators, hwloc_uint64_t *values)¶

Return the initiators that have values for a given attribute for a specific target NUMA node.

Return initiators for the given attribute and target node in the initiators array. If values is not NULL, the corresponding attribute values are stored in the array it points to.

On input, nr points to the number of initiators that may be stored in the array initiators (and values). On output, nr points to the number of initiators (and values) that were actually found, even if some of them couldn’t be stored in the array. Initiators that couldn’t be stored are ignored, but the function still returns success (0). The caller may find out by comparing the value pointed by nr before and after the function call.

The returned initiators should not be modified or freed, they belong to the topology.

target_node cannot be NULL.

flags must be 0 for now.

If the attribute does not relate to a specific initiator (it does not have the flag HWLOC_MEMATTR_FLAG_NEED_INITIATOR), no initiator is returned.

Note

This function is meant for tools and debugging (listing internal information) rather than for application queries. Applications should rather select useful NUMA nodes with hwloc_get_local_numanode_objs() and then look at their attribute values for some relevant initiators.

Returns:: 0 on success or -1 on error.

struct hwloc_location¶: #include <memattrs.h>

Where to measure attributes from.

Managing memory attributes¶

group Managing memory attributes

Memory attribues are identified by an ID (hwloc_memattr_id_t) and a name. hwloc_memattr_get_name() and hwloc_memattr_get_by_name() convert between them (or return error if the attribute does not exist).

The set of valid hwloc_memattr_id_t is a contigous set starting at 0. It first contains predefined attributes, as listed in hwloc_memattr_id_e (from 0 to HWLOC_MEMATTR_ID_MAX-1). Then custom attributes may be dynamically registered with hwloc_memattr_register(). They will get the following IDs (HWLOC_MEMATTR_ID_MAX for the first one, etc.).

To iterate over all valid attributes (either predefined or dynamically registered custom ones), one may iterate over IDs starting from 0 until hwloc_memattr_get_name() or hwloc_memattr_get_flags() returns an error.

The values for an existing attribute or for custom dynamically registered ones may be set or modified with hwloc_memattr_set_value().

Enums

enum hwloc_memattr_flag_e¶

Memory attribute flags. Given to hwloc_memattr_register() and returned by hwloc_memattr_get_flags().

Values:

enumerator HWLOC_MEMATTR_FLAG_HIGHER_FIRST¶: The best nodes for this memory attribute are those with the higher values. For instance Bandwidth.

enumerator HWLOC_MEMATTR_FLAG_LOWER_FIRST¶: The best nodes for this memory attribute are those with the lower values. For instance Latency.

enumerator HWLOC_MEMATTR_FLAG_NEED_INITIATOR¶: The value returned for this memory attribute depends on the given initiator. For instance Bandwidth and Latency, but not Capacity.

Functions

int hwloc_memattr_get_name(hwloc_topology_t topology, hwloc_memattr_id_t attribute, const char **name)¶

Return the name of a memory attribute.

The output pointer name cannot be NULL.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if the attribute does not exist.

int hwloc_memattr_get_flags(hwloc_topology_t topology, hwloc_memattr_id_t attribute, unsigned long *flags)¶

Return the flags of the given attribute.

Flags are a OR’ed set of hwloc_memattr_flag_e.

The output pointer flags cannot be NULL.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if the attribute does not exist.

int hwloc_memattr_register(hwloc_topology_t topology, const char *name, unsigned long flags, hwloc_memattr_id_t *id)¶

Register a new memory attribute.

Add a new custom memory attribute. Flags are a OR’ed set of hwloc_memattr_flag_e. It must contain one of HWLOC_MEMATTR_FLAG_HIGHER_FIRST or HWLOC_MEMATTR_FLAG_LOWER_FIRST but not both.

The new attribute id is immediately after the last existing attribute ID (which is either the ID of the last registered attribute if any, or the ID of the last predefined attribute in hwloc_memattr_id_e).

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if an invalid set of flags is given.
Returns:: -1 with errno set to EBUSY if another attribute already uses this name.

int hwloc_memattr_set_value(hwloc_topology_t topology, hwloc_memattr_id_t attribute, hwloc_obj_t target_node, struct hwloc_location *initiator, unsigned long flags, hwloc_uint64_t value)¶

Set an attribute value for a specific target NUMA node.

If the attribute does not relate to a specific initiator (it does not have the flag HWLOC_MEMATTR_FLAG_NEED_INITIATOR), location initiator is ignored and may be NULL.

The initiator will be copied into the topology, the caller should free anything allocated to store the initiator, for instance the cpuset.

target_node cannot be NULL.

attribute cannot be HWLOC_MEMATTR_ID_CAPACITY or HWLOC_MEMATTR_ID_LOCALITY.

flags must be 0 for now.

Note

The initiator initiator should be of type HWLOC_LOCATION_TYPE_CPUSET when referring to accesses performed by CPU cores. HWLOC_LOCATION_TYPE_OBJECT is currently unused internally by hwloc, but users may for instance use it to provide custom information about host memory accesses performed by GPUs.

Returns:: 0 on success or -1 on error.

Kinds of CPU cores¶

group Kinds of CPU cores

Platforms with heterogeneous CPUs may have some cores with different features or frequencies. This API exposes identical PUs in sets called CPU kinds. Each PU of the topology may only be in a single kind.

The number of kinds may be obtained with hwloc_cpukinds_get_nr(). If the platform is homogeneous, there may be a single kind with all PUs. If the platform or operating system does not expose any information about CPU cores, there may be no kind at all.

The index of the kind that describes a given CPU set (if any, and not partially) may be obtained with hwloc_cpukinds_get_by_cpuset().

From the index of a kind, it is possible to retrieve information with hwloc_cpukinds_get_info(): an abstracted efficiency value, and an array of info attributes (for instance the “CoreType” and “FrequencyMaxMHz”, see CPU Kinds).

A higher efficiency value means greater intrinsic performance (and possibly less performance/power efficiency). Kinds with lower efficiency values are ranked first: Passing 0 as kind_index to hwloc_cpukinds_get_info() will return information about the CPU kind with lower performance but higher energy-efficiency. Higher kind_index values would rather return information about power-hungry high-performance cores.

When available, efficiency values are gathered from the operating system. If so, cpukind_efficiency is set in the struct hwloc_topology_discovery_support array. This is currently available on Windows 10, Mac OS X (Darwin), and on some Linux platforms where core “capacity” is exposed in sysfs.

If the operating system does not expose core efficiencies natively, hwloc tries to compute efficiencies by comparing CPU kinds using frequencies (on ARM), or core types and frequencies (on other architectures). The environment variable HWLOC_CPUKINDS_RANKING may be used to change this heuristics, see Environment Variables.

If hwloc fails to rank any kind, for instance because the operating system does not expose efficiencies and core frequencies, all kinds will have an unknown efficiency (-1), and they are not indexed/ordered in any specific way.

Functions

int hwloc_cpukinds_get_nr(hwloc_topology_t topology, unsigned long flags)¶

Get the number of different kinds of CPU cores in the topology.

flags must be 0 for now.

Returns:: The number of CPU kinds (positive integer) on success.
Returns:: 0 if no information about kinds was found.
Returns:: -1 with errno set to EINVAL if flags is invalid.

int hwloc_cpukinds_get_by_cpuset(hwloc_topology_t topology, hwloc_const_bitmap_t cpuset, unsigned long flags)¶

Get the index of the CPU kind that contains CPUs listed in cpuset.

flags must be 0 for now.

Returns:: The index of the CPU kind (positive integer or 0) on success.
Returns:: -1 with errno set to EXDEV if cpuset is only partially included in the some kind.
Returns:: -1 with errno set to ENOENT if cpuset is not included in any kind, even partially.
Returns:: -1 with errno set to EINVAL if parameters are invalid.

int hwloc_cpukinds_get_info(hwloc_topology_t topology, unsigned kind_index, hwloc_bitmap_t cpuset, int *efficiency, struct hwloc_infos_s **infosp, unsigned long flags)¶

Get the CPU set and infos about a CPU kind in the topology.

kind_index identifies one kind of CPU between 0 and the number of kinds returned by hwloc_cpukinds_get_nr() minus 1.

If not NULL, the bitmap cpuset will be filled with the set of PUs of this kind.

The integer pointed by efficiency, if not NULL will, be filled with the ranking of this kind of CPU in term of efficiency (see above). It ranges from 0 to the number of kinds (as reported by hwloc_cpukinds_get_nr()) minus 1.

Kinds with lower efficiency are reported first.

If there is a single kind in the topology, its efficiency 0. If the efficiency of some kinds of cores is unknown, the efficiency of all kinds is set to -1, and kinds are reported in no specific order.

If infosp is not NULL, it will receive a pointer to a structure containing the array of info attributes (for instance “CoreType”, “FrequencyMaxMHz” or “FrequencyBaseMHz”, see CPU Kinds) and its length. This structure belongs to the hwloc library, it should not be freed or modified.

flags must be 0 for now.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOENT if kind_index does not match any CPU kind.
Returns:: -1 with errno set to EINVAL if parameters are invalid.

int hwloc_cpukinds_register(hwloc_topology_t topology, hwloc_bitmap_t cpuset, int forced_efficiency, struct hwloc_infos_s *infos, unsigned long flags)¶

Register a kind of CPU in the topology.

Mark the PUs listed in cpuset as being of the same kind with respect to the given attributes.

forced_efficiency should be -1 if unknown. Otherwise it is an abstracted efficiency value to enforce the ranking of all kinds if all of them have valid (and different) efficiencies.

The array infos may be used to provide info names and values describing this kind of PUs, or it may be NULL.

flags must be 0 for now.

Parameters cpuset and infos will be duplicated internally, the caller is responsible for freeing them.

If cpuset overlaps with some existing kinds, those might get modified or split. For instance if existing kind A contains PUs 0 and 1, and one registers another kind for PU 1 and 2, there will be 3 resulting kinds: existing kind A is restricted to only PU 0; new kind B contains only PU 1 and combines information from A and from the newly-registered kind; new kind C contains only PU 2 and only gets information from the newly-registered kind.

Note

The efficiency forced_efficiency provided to this function may be different from the one reported later by hwloc_cpukinds_get_info() because hwloc will scale efficiency values down to between 0 and the number of kinds minus 1.

Returns:: 0 on success.
Returns:: -1 with errno set to EINVAL if some parameters are invalid, for instance if cpuset is NULL or empty.

Linux-specific helpers¶

group Linux-specific helpers

This includes helpers for manipulating Linux kernel cpumap files, and hwloc equivalents of the Linux sched_setaffinity and sched_getaffinity system calls.

Functions

int hwloc_linux_set_tid_cpubind(hwloc_topology_t topology, pid_t tid, hwloc_const_cpuset_t set)¶

Bind a thread tid on cpus given in cpuset set.

The behavior is exactly the same as the Linux sched_setaffinity system call, but uses a hwloc cpuset.

Note

This is equivalent to calling hwloc_set_proc_cpubind() with HWLOC_CPUBIND_THREAD as flags.

Returns:: 0 on success, -1 on error.

int hwloc_linux_get_tid_cpubind(hwloc_topology_t topology, pid_t tid, hwloc_cpuset_t set)¶

Get the current binding of thread tid.

The CPU-set set (previously allocated by the caller) is filled with the list of PUs which the thread was last bound to.

The behavior is exactly the same as the Linux sched_getaffinity system call, but uses a hwloc cpuset.

Note

This is equivalent to calling hwloc_get_proc_cpubind() with HWLOC_CPUBIND_THREAD as flags.

Returns:: 0 on success, -1 on error.

int hwloc_linux_get_tid_last_cpu_location(hwloc_topology_t topology, pid_t tid, hwloc_bitmap_t set)¶

Get the last physical CPU where thread tid ran.

The CPU-set set (previously allocated by the caller) is filled with the PU which the thread last ran on.

Note

This is equivalent to calling hwloc_get_proc_last_cpu_location() with HWLOC_CPUBIND_THREAD as flags.

Returns:: 0 on success, -1 on error.

int hwloc_linux_read_path_as_cpumask(const char *path, hwloc_bitmap_t set)¶

Convert a linux kernel cpumask file path into a hwloc bitmap set.

Might be used when reading CPU set from sysfs attributes such as topology and caches for processors, or local_cpus for devices.

Note

This function ignores the HWLOC_FSROOT environment variable.

Returns:: 0 on success, -1 on error.

Interoperability with Linux libnuma unsigned long masks¶

group Interoperability with Linux libnuma unsigned long masks

This interface helps converting between Linux libnuma unsigned long masks and hwloc cpusets and nodesets.

Note

Topology topology must match the current machine.

Note

The behavior of libnuma is undefined if the kernel is not NUMA-aware. (when CONFIG_NUMA is not set in the kernel configuration). This helper and libnuma may thus not be strictly compatible in this case, which may be detected by checking whether numa_available() returns -1.

Functions

inline int hwloc_cpuset_to_linux_libnuma_ulongs(hwloc_topology_t topology, hwloc_const_cpuset_t cpuset, unsigned long *mask, unsigned long *maxnode)¶

Convert hwloc CPU set cpuset into the array of unsigned long mask.

mask is the array of unsigned long that will be filled. maxnode contains the maximal node number that may be stored in mask. maxnode will be set to the maximal node number that was found, plus one.

This function may be used before calling set_mempolicy, mbind, migrate_pages or any other function that takes an array of unsigned long and a maximal node number as input parameter.

Returns:: 0.

inline int hwloc_nodeset_to_linux_libnuma_ulongs(hwloc_topology_t topology, hwloc_const_nodeset_t nodeset, unsigned long *mask, unsigned long *maxnode)¶

Convert hwloc NUMA node set nodeset into the array of unsigned long mask.

mask is the array of unsigned long that will be filled. maxnode contains the maximal node number that may be stored in mask. maxnode will be set to the maximal node number that was found, plus one.

This function may be used before calling set_mempolicy, mbind, migrate_pages or any other function that takes an array of unsigned long and a maximal node number as input parameter.

Returns:: 0.

inline int hwloc_cpuset_from_linux_libnuma_ulongs(hwloc_topology_t topology, hwloc_cpuset_t cpuset, const unsigned long *mask, unsigned long maxnode)¶

Convert the array of unsigned long mask into hwloc CPU set.

mask is a array of unsigned long that will be read. maxnode contains the maximal node number that may be read in mask.

This function may be used after calling get_mempolicy or any other function that takes an array of unsigned long as output parameter (and possibly a maximal node number as input parameter).

Returns:: 0 on success.
Returns:: -1 on error, for instance if failing an internal reallocation.

inline int hwloc_nodeset_from_linux_libnuma_ulongs(hwloc_topology_t topology, hwloc_nodeset_t nodeset, const unsigned long *mask, unsigned long maxnode)¶

Convert the array of unsigned long mask into hwloc NUMA node set.

mask is a array of unsigned long that will be read. maxnode contains the maximal node number that may be read in mask.

This function may be used after calling get_mempolicy or any other function that takes an array of unsigned long as output parameter (and possibly a maximal node number as input parameter).

Returns:: 0 on success.
Returns:: -1 with errno set to ENOMEM if some internal reallocation failed.

Interoperability with Linux libnuma bitmask¶

group Interoperability with Linux libnuma bitmask

This interface helps converting between Linux libnuma bitmasks and hwloc cpusets and nodesets.

Note

Topology topology must match the current machine.

Note

The behavior of libnuma is undefined if the kernel is not NUMA-aware. (when CONFIG_NUMA is not set in the kernel configuration). This helper and libnuma may thus not be strictly compatible in this case, which may be detected by checking whether numa_available() returns -1.

Functions

inline struct bitmask *hwloc_cpuset_to_linux_libnuma_bitmask(hwloc_topology_t topology, hwloc_const_cpuset_t cpuset)¶

Convert hwloc CPU set cpuset into the returned libnuma bitmask.

The returned bitmask should later be freed with numa_bitmask_free.

This function may be used before calling many numa_ functions that use a struct bitmask as an input parameter.

Returns:: newly allocated struct bitmask, or NULL on error.

inline struct bitmask *hwloc_nodeset_to_linux_libnuma_bitmask(hwloc_topology_t topology, hwloc_const_nodeset_t nodeset)¶

Convert hwloc NUMA node set nodeset into the returned libnuma bitmask.

The returned bitmask should later be freed with numa_bitmask_free.

This function may be used before calling many numa_ functions that use a struct bitmask as an input parameter.

Returns:: newly allocated struct bitmask, or NULL on error.

inline int hwloc_cpuset_from_linux_libnuma_bitmask(hwloc_topology_t topology, hwloc_cpuset_t cpuset, const struct bitmask *bitmask)¶

Convert libnuma bitmask bitmask into hwloc CPU set cpuset.

This function may be used after calling many numa_ functions that use a struct bitmask as an output parameter.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOMEM if some internal reallocation failed.

inline int hwloc_nodeset_from_linux_libnuma_bitmask(hwloc_topology_t topology, hwloc_nodeset_t nodeset, const struct bitmask *bitmask)¶

Convert libnuma bitmask bitmask into hwloc NUMA node set nodeset.

This function may be used after calling many numa_ functions that use a struct bitmask as an output parameter.

Returns:: 0 on success.
Returns:: -1 with errno set to ENOMEM if some internal reallocation failed.

Windows-specific helpers¶

group Windows-specific helpers

These functions query Windows processor groups. These groups partition the operating system into virtual sets of up to 64 neighbor PUs. Threads and processes may only be bound inside a single group. Although Windows processor groups may be exposed in the hwloc hierarchy as hwloc Groups, they are also often merged into existing hwloc objects such as NUMA nodes or Packages. This API provides explicit information about Windows processor groups so that applications know whether binding to a large set of PUs may fail because it spans over multiple Windows processor groups.

Functions

int hwloc_windows_get_nr_processor_groups(hwloc_topology_t topology, unsigned long flags)¶

Get the number of Windows processor groups.

flags must be 0 for now.

Returns:: at least 1 on success.
Returns:: -1 on error, for instance if the topology does not match the current system (e.g. loaded from another machine through XML).

int hwloc_windows_get_processor_group_cpuset(hwloc_topology_t topology, unsigned pg_index, hwloc_cpuset_t cpuset, unsigned long flags)¶

Get the CPU-set of a Windows processor group.

Get the set of PU included in the processor group specified by pg_index. pg_index must be between 0 and the value returned by hwloc_windows_get_nr_processor_groups() minus 1.

flags must be 0 for now.

Returns:: 0 on success.
Returns:: -1 on error, for instance if pg_index is invalid, or if the topology does not match the current system (e.g. loaded from another machine through XML).

Interoperability with glibc sched affinity¶

group Interoperability with glibc sched affinity

This interface offers ways to convert between hwloc cpusets and glibc cpusets such as those manipulated by sched_getaffinity() or pthread_attr_setaffinity_np().

Note

Topology topology must match the current machine.

Functions

inline int hwloc_cpuset_to_glibc_sched_affinity(hwloc_topology_t topology, hwloc_const_cpuset_t hwlocset, cpu_set_t *schedset, size_t schedsetsize)¶

Convert hwloc CPU set toposet into glibc sched affinity CPU set schedset.

This function may be used before calling sched_setaffinity or any other function that takes a cpu_set_t as input parameter.

schedsetsize should be sizeof(cpu_set_t) unless schedset was dynamically allocated with CPU_ALLOC

Returns:: 0.

inline int hwloc_cpuset_from_glibc_sched_affinity(hwloc_topology_t topology, hwloc_cpuset_t hwlocset, const cpu_set_t *schedset, size_t schedsetsize)¶

Convert glibc sched affinity CPU set schedset into hwloc CPU set.

This function may be used before calling sched_setaffinity or any other function that takes a cpu_set_t as input parameter.

schedsetsize should be sizeof(cpu_set_t) unless schedset was dynamically allocated with CPU_ALLOC

Returns:: 0 on success.
Returns:: -1 with errno set to ENOMEM if some internal reallocation failed.

Interoperability with OpenCL¶

group Interoperability with OpenCL

This interface offers ways to retrieve topology information about OpenCL devices.

Only AMD and NVIDIA OpenCL implementations currently offer useful locality information about their devices.

Functions

inline int hwloc_opencl_get_device_pci_busid(cl_device_id device, unsigned *domain, unsigned *bus, unsigned *dev, unsigned *func)¶

Return the domain, bus and device IDs of the OpenCL device device.

Device device must match the local machine.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline int hwloc_opencl_get_device_cpuset(hwloc_topology_t topology, cl_device_id device, hwloc_cpuset_t set)¶

Get the CPU set of processors that are physically close to OpenCL device device.

Store in set the CPU-set describing the locality of the OpenCL device device.

Topology topology and device device must match the local machine. I/O devices detection and the OpenCL component are not needed in the topology.

The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_opencl_get_device_osdev() and hwloc_opencl_get_device_osdev_by_index().

This function is currently only implemented in a meaningful way for Linux with the AMD or NVIDIA OpenCL implementation; other systems will simply get a full cpuset.

Returns:: 0 on success.
Returns:: -1 on error, for instance if the device could not be found.

inline hwloc_obj_t hwloc_opencl_get_device_osdev_by_index(hwloc_topology_t topology, unsigned platform_index, unsigned device_index)¶

Get the hwloc OS device object corresponding to the OpenCL device for the given indexes.

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the OpenCL component must be enabled in the topology.

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object (unless PCI devices are filtered out).

Returns:: The hwloc OS device object describing the OpenCL device whose platform index is platform_index, and whose device index within this platform if device_index.
Returns:: NULL if there is none.

inline hwloc_obj_t hwloc_opencl_get_device_osdev(hwloc_topology_t topology, cl_device_id device)¶

Get the hwloc OS device object corresponding to OpenCL device deviceX.

This function currently only works on AMD and NVIDIA OpenCL devices that support relevant OpenCL extensions. hwloc_opencl_get_device_osdev_by_index() should be preferred whenever possible, i.e. when platform and device index are known.

Topology topology and device device must match the local machine. I/O devices detection and the OpenCL component must be enabled in the topology. If not, the locality of the object may still be found using hwloc_opencl_get_device_cpuset().

Note

This function cannot work if PCI devices are filtered out.

Note

The corresponding hwloc PCI device may be found by looking at the result parent pointer (unless PCI devices are filtered out).

Returns:: The hwloc OS device object corresponding to the given OpenCL device device.
Returns:: NULL if none could be found, for instance if required OpenCL attributes are not available.

Interoperability with the CUDA Driver API¶

group Interoperability with the CUDA Driver API

This interface offers ways to retrieve topology information about CUDA devices when using the CUDA Driver API.

Functions

inline int hwloc_cuda_get_device_pci_ids(hwloc_topology_t topology, CUdevice cudevice, int *domain, int *bus, int *dev)¶

Return the domain, bus and device IDs of the CUDA device cudevice.

Device cudevice must match the local machine.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline int hwloc_cuda_get_device_cpuset(hwloc_topology_t topology, CUdevice cudevice, hwloc_cpuset_t set)¶

Get the CPU set of processors that are physically close to device cudevice.

Store in set the CPU-set describing the locality of the CUDA device cudevice.

Topology topology and device cudevice must match the local machine. I/O devices detection and the CUDA component are not needed in the topology.

The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_cuda_get_device_osdev() and hwloc_cuda_get_device_osdev_by_index().

This function is currently only implemented in a meaningful way for Linux; other systems will simply get a full cpuset.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline hwloc_obj_t hwloc_cuda_get_device_pcidev(hwloc_topology_t topology, CUdevice cudevice)¶

Get the hwloc PCI device object corresponding to the CUDA device cudevice.

Topology topology and device cudevice must match the local machine. I/O devices detection must be enabled in topology topology. The CUDA component is not needed in the topology.

Returns:: The hwloc PCI device object describing the CUDA device cudevice.
Returns:: NULL if none could be found.

inline hwloc_obj_t hwloc_cuda_get_device_osdev(hwloc_topology_t topology, CUdevice cudevice)¶

Get the hwloc OS device object corresponding to CUDA device cudevice.

Topology topology and device cudevice must match the local machine. I/O devices detection and the CUDA component must be enabled in the topology. If not, the locality of the object may still be found using hwloc_cuda_get_device_cpuset().

Note

This function cannot work if PCI devices are filtered out.

Note

The corresponding hwloc PCI device may be found by looking at the result parent pointer (unless PCI devices are filtered out).

Returns:: The hwloc OS device object that describes the given CUDA device cudevice.
Returns:: NULL if none could be found.

inline hwloc_obj_t hwloc_cuda_get_device_osdev_by_index(hwloc_topology_t topology, unsigned idx)¶

Get the hwloc OS device object corresponding to the CUDA device whose index is idx.

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the CUDA component must be enabled in the topology.

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object (unless PCI devices are filtered out).

Note

This function is identical to hwloc_cudart_get_device_osdev_by_index().

Returns:: The hwloc OS device object describing the CUDA device whose index is idx.
Returns:: NULL if none could be found.

Interoperability with the CUDA Runtime API¶

group Interoperability with the CUDA Runtime API

This interface offers ways to retrieve topology information about CUDA devices when using the CUDA Runtime API.

Functions

inline int hwloc_cudart_get_device_pci_ids(hwloc_topology_t topology, int idx, int *domain, int *bus, int *dev)¶

Return the domain, bus and device IDs of the CUDA device whose index is idx.

Device index idx must match the local machine.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline int hwloc_cudart_get_device_cpuset(hwloc_topology_t topology, int idx, hwloc_cpuset_t set)¶

Get the CPU set of processors that are physically close to device idx.

Store in set the CPU-set describing the locality of the CUDA device whose index is idx.

Topology topology and device idx must match the local machine. I/O devices detection and the CUDA component are not needed in the topology.

The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_cudart_get_device_osdev_by_index().

This function is currently only implemented in a meaningful way for Linux; other systems will simply get a full cpuset.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline hwloc_obj_t hwloc_cudart_get_device_pcidev(hwloc_topology_t topology, int idx)¶

Get the hwloc PCI device object corresponding to the CUDA device whose index is idx.

Topology topology and device idx must match the local machine. I/O devices detection must be enabled in topology topology. The CUDA component is not needed in the topology.

Returns:: The hwloc PCI device object describing the CUDA device whose index is idx.
Returns:: NULL if none could be found.

inline hwloc_obj_t hwloc_cudart_get_device_osdev_by_index(hwloc_topology_t topology, unsigned idx)¶

Get the hwloc OS device object corresponding to the CUDA device whose index is idx.

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the CUDA component must be enabled in the topology. If not, the locality of the object may still be found using hwloc_cudart_get_device_cpuset().

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object (unless PCI devices are filtered out).

Note

This function is identical to hwloc_cuda_get_device_osdev_by_index().

Returns:: The hwloc OS device object describing the CUDA device whose index is idx.
Returns:: NULL if none could be found.

Interoperability with the NVIDIA Management Library¶

group Interoperability with the NVIDIA Management Library

This interface offers ways to retrieve topology information about devices managed by the NVIDIA Management Library (NVML).

Functions

inline int hwloc_nvml_get_device_cpuset(hwloc_topology_t topology, nvmlDevice_t device, hwloc_cpuset_t set)¶

Get the CPU set of processors that are physically close to NVML device device.

Store in set the CPU-set describing the locality of the NVML device device.

Topology topology and device device must match the local machine. I/O devices detection and the NVML component are not needed in the topology.

The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_nvml_get_device_osdev() and hwloc_nvml_get_device_osdev_by_index().

This function is currently only implemented in a meaningful way for Linux; other systems will simply get a full cpuset.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline hwloc_obj_t hwloc_nvml_get_device_osdev_by_index(hwloc_topology_t topology, unsigned idx)¶

Get the hwloc OS device object corresponding to the NVML device whose index is idx.

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the NVML component must be enabled in the topology.

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object (unless PCI devices are filtered out).

Returns:: The hwloc OS device object describing the NVML device whose index is idx.
Returns:: NULL if none could be found.

inline hwloc_obj_t hwloc_nvml_get_device_osdev(hwloc_topology_t topology, nvmlDevice_t device)¶

Get the hwloc OS device object corresponding to NVML device device.

Topology topology and device device must match the local machine. I/O devices detection and the NVML component must be enabled in the topology. If not, the locality of the object may still be found using hwloc_nvml_get_device_cpuset().

Note

The corresponding hwloc PCI device may be found by looking at the result parent pointer (unless PCI devices are filtered out).

Returns:: The hwloc OS device object that describes the given NVML device device.
Returns:: NULL if none could be found.

Interoperability with the ROCm SMI Management Library¶

group Interoperability with the ROCm SMI Management Library

This interface offers ways to retrieve topology information about devices managed by the ROCm SMI Management Library.

Functions

inline int hwloc_rsmi_get_device_cpuset(hwloc_topology_t topology, uint32_t dv_ind, hwloc_cpuset_t set)¶

Get the CPU set of logical processors that are physically close to AMD GPU device whose index is dv_ind.

Store in set the CPU-set describing the locality of the AMD GPU device whose index is dv_ind.

Topology topology and device dv_ind must match the local machine. I/O devices detection and the ROCm SMI component are not needed in the topology.

The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_rsmi_get_device_osdev() and hwloc_rsmi_get_device_osdev_by_index().

This function is currently only implemented in a meaningful way for Linux; other systems will simply get a full cpuset.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline hwloc_obj_t hwloc_rsmi_get_device_osdev_by_index(hwloc_topology_t topology, uint32_t dv_ind)¶

Get the hwloc OS device object corresponding to the AMD GPU device whose index is dv_ind.

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the ROCm SMI component must be enabled in the topology.

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object (unless PCI devices are filtered out).

Returns:: The hwloc OS device object describing the AMD GPU device whose index is dv_ind.
Returns:: NULL if none could be found.

inline hwloc_obj_t hwloc_rsmi_get_device_osdev(hwloc_topology_t topology, uint32_t dv_ind)¶

Get the hwloc OS device object corresponding to AMD GPU device, whose index is dv_ind.

Topology topology and device dv_ind must match the local machine. I/O devices detection and the ROCm SMI component must be enabled in the topology. If not, the locality of the object may still be found using hwloc_rsmi_get_device_cpuset().

Note

The corresponding hwloc PCI device may be found by looking at the result parent pointer (unless PCI devices are filtered out).

Returns:: The hwloc OS device object that describes the given AMD GPU, whose index is dv_ind.
Returns:: NULL if none could be found.

Interoperability with the oneAPI Level Zero interface¶

group Interoperability with the oneAPI Level Zero interface.

This interface offers ways to retrieve topology information about devices managed by the Level Zero API, both for main Core devices (ZE API) and the Sysman devices (ZES API).

Functions

inline int hwloc_levelzero_get_device_cpuset(hwloc_topology_t topology, ze_device_handle_t device, hwloc_cpuset_t set)¶

Get the CPU set of logical processors that are physically close to the Level Zero device device.

Store in set the CPU-set describing the locality of the Level Zero device device.

Topology topology and device device must match the local machine. The Level Zero library must have been initialized with zeInit(). I/O devices detection and the Level Zero component are not needed in the topology.

The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_levelzero_get_device_osdev().

This function is currently only implemented in a meaningful way for Linux; other systems will simply get a full cpuset.

Note

zeDevicePciGetPropertiesExt() must be supported, or the entire machine locality will be returned.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline int hwloc_levelzero_get_sysman_device_cpuset(hwloc_topology_t topology, zes_device_handle_t device, hwloc_cpuset_t set)¶

Get the CPU set of logical processors that are physically close to the Level Zero Sysman device device.

Store in set the CPU-set describing the locality of the Level Zero device device.

Topology topology and device device must match the local machine. The Level Zero library must have been initialized with Sysman enabled with zesInit(). I/O devices detection and the Level Zero component are not needed in the topology.

The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_levelzero_get_device_osdev().

This function is currently only implemented in a meaningful way for Linux; other systems will simply get a full cpuset.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline hwloc_obj_t hwloc_levelzero_get_device_osdev(hwloc_topology_t topology, ze_device_handle_t device)¶

Get the hwloc OS device object corresponding to Level Zero device device.

Topology topology and device dv_ind must match the local machine. The Level Zero library must have been initialized with zeInit(). I/O devices detection and the Level Zero component must be enabled in the topology. If not, the locality of the object may still be found using hwloc_levelzero_get_device_cpuset().

Note

If the input ZE device is actually a subdevice, then its parent (root device) is actually translated, i.e. the main hwloc OS device is returned instead of one of its children.

Note

The corresponding hwloc PCI device may be found by looking at the result parent pointer (unless PCI devices are filtered out).

Note

zeDevicePciGetPropertiesExt() must be supported.

Returns:: The hwloc OS device object that describes the given Level Zero device device.
Returns:: NULL if none could be found.

inline hwloc_obj_t hwloc_levelzero_get_sysman_device_osdev(hwloc_topology_t topology, zes_device_handle_t device)¶

Get the hwloc OS device object corresponding to Level Zero Sysman device device.

Topology topology and device dv_ind must match the local machine. The Level Zero library must have been initialized with Sysman enabled with zesInit(). I/O devices detection and the Level Zero component must be enabled in the topology. If not, the locality of the object may still be found using hwloc_levelzero_get_device_cpuset().

Note

If the input ZES device is actually a subdevice, then its parent (root device) is actually translated, i.e. the main hwloc OS device is returned instead of one of its children.

Note

The corresponding hwloc PCI device may be found by looking at the result parent pointer (unless PCI devices are filtered out).

Returns:: The hwloc OS device object that describes the given Level Zero device device.
Returns:: NULL if none could be found.

Interoperability with OpenGL displays¶

group Interoperability with OpenGL displays

This interface offers ways to retrieve topology information about OpenGL displays.

Only the NVIDIA display locality information is currently available, using the NV-CONTROL X11 extension and the NVCtrl library.

Functions

inline hwloc_obj_t hwloc_gl_get_display_osdev_by_port_device(hwloc_topology_t topology, unsigned port, unsigned device)¶

Get the hwloc OS device object corresponding to the OpenGL display given by port and device index.

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the GL component must be enabled in the topology.

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object (unless PCI devices are filtered out).

Returns:: The hwloc OS device object describing the OpenGL display whose port (server) is port and device (screen) is device.
Returns:: NULL if none could be found.

inline hwloc_obj_t hwloc_gl_get_display_osdev_by_name(hwloc_topology_t topology, const char *name)¶

Get the hwloc OS device object corresponding to the OpenGL display given by name.

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the GL component must be enabled in the topology.

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object (unless PCI devices are filtered out).

Returns:: The hwloc OS device object describing the OpenGL display whose name is name, built as “:port.device” such as “:0.0” .
Returns:: NULL if none could be found.

inline int hwloc_gl_get_display_by_osdev(hwloc_topology_t topology, hwloc_obj_t osdev, unsigned *port, unsigned *device)¶

Get the OpenGL display port and device corresponding to the given hwloc OS object.

Retrieves the OpenGL display port (server) in port and device (screen) in screen that correspond to the given hwloc OS device object.

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection and the GL component must be enabled in the topology.

Returns:: 0 on success.
Returns:: -1 if none could be found.

Interoperability with OpenFabrics¶

group Interoperability with OpenFabrics

This interface offers ways to retrieve topology information about OpenFabrics devices (InfiniBand, Omni-Path, usNIC, etc).

Functions

inline int hwloc_ibv_get_device_cpuset(hwloc_topology_t topology, struct ibv_device *ibdev, hwloc_cpuset_t set)¶

Get the CPU set of processors that are physically close to device ibdev.

Store in set the CPU-set describing the locality of the OpenFabrics device ibdev (InfiniBand, etc).

Topology topology and device ibdev must match the local machine. I/O devices detection is not needed in the topology.

The function only returns the locality of the device. If more information about the device is needed, OS objects should be used instead, see hwloc_ibv_get_device_osdev() and hwloc_ibv_get_device_osdev_by_name().

This function is currently only implemented in a meaningful way for Linux; other systems will simply get a full cpuset.

Returns:: 0 on success.
Returns:: -1 on error, for instance if device information could not be found.

inline hwloc_obj_t hwloc_ibv_get_device_osdev_by_name(hwloc_topology_t topology, const char *ibname)¶

Get the hwloc OS device object corresponding to the OpenFabrics device named ibname.

The name ibname is usually obtained from ibv_get_device_name().

The topology topology does not necessarily have to match the current machine. For instance the topology may be an XML import of a remote host. I/O devices detection must be enabled in the topology.

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object.

Returns:: The hwloc OS device object describing the OpenFabrics device (InfiniBand, Omni-Path, usNIC, etc) whose name is ibname (mlx5_0, hfi1_0, usnic_0, qib0, etc).
Returns:: NULL if none could be found.

inline hwloc_obj_t hwloc_ibv_get_device_osdev(hwloc_topology_t topology, struct ibv_device *ibdev)¶

Get the hwloc OS device object corresponding to the OpenFabrics device ibdev.

Topology topology and device ibdev must match the local machine. I/O devices detection must be enabled in the topology. If not, the locality of the object may still be found using hwloc_ibv_get_device_cpuset().

Note

The corresponding PCI device object can be obtained by looking at the OS device parent object.

Returns:: The hwloc OS device object describing the OpenFabrics device ibdev (InfiniBand, etc).
Returns:: NULL if none could be found.

Topology differences¶

group Topology differences

Applications that manipulate many similar topologies, for instance one for each node of a homogeneous cluster, may want to compress topologies to reduce the memory footprint.

This file offers a way to manipulate the difference between topologies and export/import it to/from XML. Compression may therefore be achieved by storing one topology entirely while the others are only described by their differences with the former. The actual topology can be reconstructed when actually needed by applying the precomputed difference to the reference topology.

This interface targets very similar nodes. Only very simple differences between topologies are actually supported, for instance a change in the memory size, the name of the object, or some info attribute. More complex differences such as adding or removing objects cannot be represented in the difference structures and therefore return errors. Differences between object sets or topology-wide allowed sets, cannot be represented either.

It means that there is no need to apply the difference when looking at the tree organization (how many levels, how many objects per level, what kind of objects, CPU and node sets, etc) and when binding to objects. However the difference must be applied when looking at object attributes such as the name, the memory size or info attributes.

Typedefs

typedef enum hwloc_topology_diff_obj_attr_type_e hwloc_topology_diff_obj_attr_type_t¶: Type of one object attribute difference.

typedef enum hwloc_topology_diff_type_e hwloc_topology_diff_type_t¶: Type of one element of a difference list.

typedef union hwloc_topology_diff_u *hwloc_topology_diff_t¶: One element of a difference list between two topologies.

Enums

enum hwloc_topology_diff_obj_attr_type_e¶

Type of one object attribute difference.

Values:

enumerator HWLOC_TOPOLOGY_DIFF_OBJ_ATTR_SIZE¶: The object local memory is modified. The union is a hwloc_topology_diff_obj_attr_u::hwloc_topology_diff_obj_attr_uint64_s (and the index field is ignored).

enumerator HWLOC_TOPOLOGY_DIFF_OBJ_ATTR_NAME¶: The object name is modified. The union is a hwloc_topology_diff_obj_attr_u::hwloc_topology_diff_obj_attr_string_s (and the name field is ignored).

enumerator HWLOC_TOPOLOGY_DIFF_OBJ_ATTR_INFO¶: the value of an info attribute is modified. The union is a hwloc_topology_diff_obj_attr_u::hwloc_topology_diff_obj_attr_string_s.

enum hwloc_topology_diff_type_e¶

Type of one element of a difference list.

Values:

enumerator HWLOC_TOPOLOGY_DIFF_OBJ_ATTR¶: An object attribute was changed. The union is a hwloc_topology_diff_u::hwloc_topology_diff_obj_attr_s.

enumerator HWLOC_TOPOLOGY_DIFF_TOO_COMPLEX¶

The difference is too complex, it cannot be represented. The difference below this object has not been checked. hwloc_topology_diff_build() will return 1.

The union is a hwloc_topology_diff_u::hwloc_topology_diff_too_complex_s.

enum hwloc_topology_diff_apply_flags_e¶

Flags to be given to hwloc_topology_diff_apply().

Values:

enumerator HWLOC_TOPOLOGY_DIFF_APPLY_REVERSE¶: Apply topology diff in reverse direction.

Functions

int hwloc_topology_diff_build(hwloc_topology_t topology, hwloc_topology_t newtopology, unsigned long flags, hwloc_topology_diff_t *diff)¶

Compute the difference between 2 topologies.

The difference is stored as a list of hwloc_topology_diff_t entries starting at diff. It is computed by doing a depth-first traversal of both topology trees simultaneously.

If the difference between 2 objects is too complex to be represented (for instance if some objects have different types, or different numbers of children), a special diff entry of type HWLOC_TOPOLOGY_DIFF_TOO_COMPLEX is queued. The computation of the diff does not continue below these objects. So each such diff entry means that the difference between two subtrees could not be computed.

Note

flags is currently not used. It should be 0.

Note

The output diff has to be freed with hwloc_topology_diff_destroy().

Note

The output diff can only be exported to XML or passed to hwloc_topology_diff_apply() if 0 was returned, i.e. if no entry of type HWLOC_TOPOLOGY_DIFF_TOO_COMPLEX is listed.

Note

The output diff may be modified by removing some entries from the list. The removed entries should be freed by passing them to to hwloc_topology_diff_destroy() (possible as another list).

Returns:: 0 if the difference can be represented properly.
Returns:: 0 with diff pointing to NULL if there is no difference between the topologies.
Returns:: 1 if the difference is too complex (see above). Some entries in the list will be of type HWLOC_TOPOLOGY_DIFF_TOO_COMPLEX.
Returns:: -1 on any other error.

int hwloc_topology_diff_apply(hwloc_topology_t topology, hwloc_topology_diff_t diff, unsigned long flags)¶

Apply a topology diff to an existing topology.

flags is an OR’ed set of hwloc_topology_diff_apply_flags_e.

The new topology is modified in place. hwloc_topology_dup() may be used to duplicate it before patching.

If the difference cannot be applied entirely, all previous applied elements are unapplied before returning.

Returns:: 0 on success.
Returns:: -N if applying the difference failed while trying to apply the N-th part of the difference. For instance -1 is returned if the very first difference element could not be applied.

int hwloc_topology_diff_destroy(hwloc_topology_diff_t diff)¶

Destroy a list of topology differences.

Returns:: 0.

int hwloc_topology_diff_load_xml(const char *xmlpath, hwloc_topology_diff_t *diff, char **refname)¶

Load a list of topology differences from a XML file.

If not NULL, refname will be filled with the identifier string of the reference topology for the difference file, if any was specified in the XML file. This identifier is usually the name of the other XML file that contains the reference topology.

Note

the pointer returned in refname should later be freed by the caller.

Returns:: 0 on success, -1 on error.

int hwloc_topology_diff_export_xml(hwloc_topology_diff_t diff, const char *refname, const char *xmlpath)¶

Export a list of topology differences to a XML file.

If not NULL, refname defines an identifier string for the reference topology which was used as a base when computing this difference. This identifier is usually the name of the other XML file that contains the reference topology. This attribute is given back when reading the diff from XML.

Returns:: 0 on success, -1 on error.

int hwloc_topology_diff_load_xmlbuffer(const char *xmlbuffer, int buflen, hwloc_topology_diff_t *diff, char **refname)¶

Load a list of topology differences from a XML buffer.

Build a list of differences from the XML memory buffer given at xmlbuffer and of length buflen (including an ending \0). This buffer may have been filled earlier with hwloc_topology_diff_export_xmlbuffer().

If not NULL, refname will be filled with the identifier string of the reference topology for the difference file, if any was specified in the XML file. This identifier is usually the name of the other XML file that contains the reference topology.

Note

the pointer returned in refname should later be freed by the caller.

Returns:: 0 on success, -1 on error.

int hwloc_topology_diff_export_xmlbuffer(hwloc_topology_diff_t diff, const char *refname, char **xmlbuffer, int *buflen)¶

Export a list of topology differences to a XML buffer.

If not NULL, refname defines an identifier string for the reference topology which was used as a base when computing this difference. This identifier is usually the name of the other XML file that contains the reference topology. This attribute is given back when reading the diff from XML.

The returned buffer ends with a \0 that is included in the returned length.

Note

The XML buffer should later be freed with hwloc_free_xmlbuffer().

Returns:: 0 on success, -1 on error.

union hwloc_topology_diff_obj_attr_u¶

#include <diff.h>

One object attribute difference.

Public Members

struct hwloc_topology_diff_obj_attr_u.hwloc_topology_diff_obj_attr_generic_s generic¶

struct hwloc_topology_diff_obj_attr_u.hwloc_topology_diff_obj_attr_uint64_s uint64¶

struct hwloc_topology_diff_obj_attr_u.hwloc_topology_diff_obj_attr_string_s string¶

struct hwloc_topology_diff_obj_attr_generic_s¶: #include <diff.h>

struct hwloc_topology_diff_obj_attr_uint64_s¶: #include <diff.h>

Integer attribute modification with an optional index.

struct hwloc_topology_diff_obj_attr_string_s¶: #include <diff.h>

String attribute modification with an optional name.

union hwloc_topology_diff_u¶

#include <diff.h>

One element of a difference list between two topologies.

Public Members

struct hwloc_topology_diff_u.hwloc_topology_diff_generic_s generic¶

struct hwloc_topology_diff_u.hwloc_topology_diff_obj_attr_s obj_attr¶

struct hwloc_topology_diff_u.hwloc_topology_diff_too_complex_s too_complex¶

struct hwloc_topology_diff_generic_s¶: #include <diff.h>

struct hwloc_topology_diff_obj_attr_s¶: #include <diff.h>

struct hwloc_topology_diff_too_complex_s¶: #include <diff.h>

Sharing topologies between processes¶

group Sharing topologies between processes

These functions are used to share a topology between processes by duplicating it into a file-backed shared-memory buffer.

The master process must first get the required shared-memory size for storing this topology with hwloc_shmem_topology_get_length().

Then it must find a virtual memory area of that size that is available in all processes (identical virtual addresses in all processes). On Linux, this can be done by comparing holes found in /proc/<pid>/maps for each process.

Once found, it must open a destination file for storing the buffer, and pass it to hwloc_shmem_topology_write() together with virtual memory address and length obtained above.

Other processes may then adopt this shared topology by opening the same file and passing it to hwloc_shmem_topology_adopt() with the exact same virtual memory address and length.

Functions

int hwloc_shmem_topology_get_length(hwloc_topology_t topology, size_t *lengthp, unsigned long flags)¶

Get the required shared memory length for storing a topology.

This length (in bytes) must be used in hwloc_shmem_topology_write() and hwloc_shmem_topology_adopt() later.

Note

Flags flags are currently unused, must be 0.

Returns:: the length, or -1 on error, for instance if flags are invalid.

int hwloc_shmem_topology_write(hwloc_topology_t topology, int fd, hwloc_uint64_t fileoffset, void *mmap_address, size_t length, unsigned long flags)¶

Duplicate a topology to a shared memory file.

Temporarily map a file in virtual memory and duplicate the topology topology by allocating duplicates in there.

The segment of the file pointed by descriptor fd, starting at offset fileoffset, and of length length (in bytes), will be temporarily mapped at virtual address mmap_address during the duplication.

The mapping length length must have been previously obtained with hwloc_shmem_topology_get_length() and the topology must not have been modified in the meantime.

Note

Flags flags are currently unused, must be 0.

Note

The object userdata pointer is duplicated but the pointed buffer is not. However the caller may also allocate it manually in shared memory to share it as well.

Returns:: 0 on success.
Returns:: -1 with errno set to EBUSY if the virtual memory mapping defined by mmap_address and length isn’t available in the process.
Returns:: -1 with errno set to EINVAL if fileoffset, mmap_address or length aren’t page-aligned.

int hwloc_shmem_topology_adopt(hwloc_topology_t *topologyp, int fd, hwloc_uint64_t fileoffset, void *mmap_address, size_t length, unsigned long flags)¶

Adopt a shared memory topology stored in a file.

Map a file in virtual memory and adopt the topology that was previously stored there with hwloc_shmem_topology_write().

The returned adopted topology in topologyp can be used just like any topology. And it must be destroyed with hwloc_topology_destroy() as usual.

However the topology is read-only. For instance, it cannot be modified with hwloc_topology_restrict() and object userdata pointers cannot be changed. The only exception is hwloc_topology_allow() which may be used to expose a different set of allowed CPUs and memory. This requires that the original topology (before sharing) was loaded with flag HWLOC_TOPOLOGY_FLAG_INCLUDE_DISALLOWED.

The segment of the file pointed by descriptor fd, starting at offset fileoffset, and of length length (in bytes), will be mapped at virtual address mmap_address.

The file pointed by descriptor fd, the offset fileoffset, the requested mapping virtual address mmap_address and the length length must be identical to what was given to hwloc_shmem_topology_write() earlier.

Note

Flags flags are currently unused, must be 0.

Note

The object userdata pointer should not be used unless the process that created the shared topology also placed userdata-pointed buffers in shared memory.

Note

This function takes care of calling hwloc_topology_abi_check().

Returns:: 0 on success.
Returns:: -1 with errno set to EBUSY if the virtual memory mapping defined by mmap_address and length isn’t available in the process.
Returns:: -1 with errno set to EINVAL if fileoffset, mmap_address or length aren’t page-aligned, or do not match what was given to hwloc_shmem_topology_write() earlier.
Returns:: -1 with errno set to EINVAL if the layout of the topology structure is different between the writer process and the adopter process.

Components and Plugins: Discovery components and backends¶

group Components and Plugins: Discovery components and backends

Note

These structures and functions may change when HWLOC_COMPONENT_ABI is modified.

Defines

HWLOC_BACKEND_PRIVATE_DATA(_backend)¶: Return the private data pointer at the end of a backend structure.

Typedefs

typedef enum hwloc_disc_phase_e hwloc_disc_phase_t¶: Discovery phase.

Enums

enum hwloc_disc_phase_e¶

Discovery phase.

Values:

enumerator HWLOC_DISC_PHASE_GLOBAL¶: xml or synthetic, platform-specific components. Discovers everything including CPU, memory, I/O and everything else. A component with a Global phase usually excludes all other phases.

enumerator HWLOC_DISC_PHASE_CPU¶: CPU discovery.

enumerator HWLOC_DISC_PHASE_MEMORY¶: Attach memory to existing CPU objects.

enumerator HWLOC_DISC_PHASE_PCI¶: Attach PCI devices and bridges to existing CPU objects.

enumerator HWLOC_DISC_PHASE_IO¶: I/O discovery that requires PCI devices (OS devices such as OpenCL, CUDA, etc.).

enumerator HWLOC_DISC_PHASE_MISC¶: Misc objects that gets added below anything else.

enumerator HWLOC_DISC_PHASE_ANNOTATE¶: Annotating existing objects, adding distances, etc.

enumerator HWLOC_DISC_PHASE_TWEAK¶: Final tweaks to a ready-to-use topology. This phase runs once the topology is loaded, before it is returned to the topology. Hence it may only use the main hwloc API for modifying the topology, for instance by restricting it, adding info attributes, etc.

enum hwloc_disc_status_flag_e¶

Discovery status flags.

Values:

enumerator HWLOC_DISC_STATUS_FLAG_GOT_ALLOWED_RESOURCES¶: The sets of allowed resources were already retrieved.

Functions

struct hwloc_backend *hwloc_backend_alloc(struct hwloc_topology *topology, struct hwloc_disc_component *component, unsigned long private_data_size)¶: Allocate a backend structure, set good default values, initialize backend->component and topology, etc. The caller will then modify whatever needed, and call hwloc_backend_enable(). If private_data_size is positive, additional space is allocated at the end of the structure for storing backend private data. That space may be obtained with HWLOC_BACKEND_PRIVATE_DATA() and may be freely modified by the backend.

int hwloc_backend_enable(struct hwloc_backend *backend)¶: Enable a previously allocated and setup backend.

struct hwloc_disc_component¶

#include <plugins.h>

Discovery component structure.

This is the major kind of components, taking care of the discovery. They are registered by generic components, either statically-built or as plugins.

struct hwloc_disc_status¶

#include <plugins.h>

Discovery status structure.

Used by the core and backends to inform about what has been/is being done during the discovery process.

struct hwloc_backend¶

#include <plugins.h>

Discovery backend structure.

A backend is the instantiation of a discovery component. When a component gets enabled for a topology, its instantiate() callback creates a backend.

hwloc_backend_alloc() initializes all fields to default values that the component may change (except “component” and “next”) before enabling the backend with hwloc_backend_enable().

Most backends assume that the topology is_thissystem flag is set because they talk to the underlying operating system. However they may still be used in topologies without the is_thissystem flag for debugging reasons. In practice, they are usually auto-disabled in such cases (excluded by xml or synthetic backends, or by environment variables when changing the Linux fsroot or the x86 cpuid path).

Components and Plugins: Generic components¶

group Components and Plugins: Generic components

Note

These structures and functions may change when HWLOC_COMPONENT_ABI is modified.

Typedefs

typedef enum hwloc_component_type_e hwloc_component_type_t¶: Generic component type.

Enums

enum hwloc_component_type_e¶

Generic component type.

Values:

enumerator HWLOC_COMPONENT_TYPE_DISC¶: The data field must point to a struct hwloc_disc_component.

enumerator HWLOC_COMPONENT_TYPE_XML¶: The data field must point to a struct hwloc_xml_component.

Functions

inline int hwloc_plugin_check_namespace(const char *pluginname, const char *symbol)¶

Make sure that plugins can lookup core symbols.

This is a sanity check to avoid lazy-lookup failures when libhwloc is loaded within a plugin, and later tries to load its own plugins. This may fail (and abort the program) if libhwloc symbols are in a private namespace.

Plugins should call this function in their init() callback to avoid later crashes if lazy symbol resolution is used by the upper layer that loaded hwloc (e.g. OpenCL implementations using dlopen with RTLD_LAZY).

Note

The build system must define HWLOC_INSIDE_PLUGIN if and only if building the caller as a plugin.

Note

This function should remain inline so plugins can call it even when they cannot find libhwloc symbols.

Returns:: 0 on success.
Returns:: -1 if the plugin cannot be successfully loaded. The caller plugin init() callback should return a negative error code as well.

struct hwloc_component¶

#include <plugins.h>

Generic component structure.

Generic components structure, either statically listed by configure in static-components.h or dynamically loaded as a plugin.

Components and Plugins: Core functions to be used by components¶

group Components and Plugins: Core functions to be used by components

Note

These structures and functions may change when HWLOC_COMPONENT_ABI is modified.

Defines

HWLOC_SHOW_CRITICAL_ERRORS()¶

HWLOC_SHOW_ALL_ERRORS()¶

Functions

int hwloc_hide_errors(void)¶

Check whether error messages are hidden.

Callers should print critical error messages (e.g. invalid hw topo info, invalid config) only if this function returns strictly less than 2.

Callers should print non-critical error messages (e.g. failure to initialize CUDA) if this function returns 0.

This function return 1 by default (show critical only), 0 in lstopo (show all), or anything set in HWLOC_HIDE_ERRORS in the environment.

Use macros HWLOC_SHOW_CRITICAL_ERRORS() and HWLOC_SHOW_ALL_ERRORS() for clarity.

hwloc_obj_t hwloc__insert_object_by_cpuset(struct hwloc_topology *topology, hwloc_obj_t root, hwloc_obj_t obj, const char *reason)¶

Add an object to the topology.

Insert new object obj in the topology starting under existing object root (if NULL, the topology root object is used).

It is sorted along the tree of other objects according to the inclusion of cpusets, to eventually be added as a child of the smallest object including this object.

If the cpuset is empty, the type of the object (and maybe some attributes) must be enough to find where to insert the object. This is especially true for NUMA nodes with memory and no CPUs.

The given object should not have children.

This shall only be called before levels are built.

The caller should check whether the object type is filtered-out before calling this function.

The topology cpuset/nodesets will be enlarged to include the object sets.

reason is a unique string identifying where and why this insertion call was performed (it will be displayed in case of internal insertion error).

Returns the object on success. Returns NULL and frees obj on error. Returns another object and frees obj if it was merged with an identical pre-existing object.

void hwloc_insert_object_by_parent(struct hwloc_topology *topology, hwloc_obj_t parent, hwloc_obj_t obj)¶

Insert an object somewhere in the topology.

It is added as the last child of the given parent. The cpuset is completely ignored, so strange objects such as I/O devices should preferably be inserted with this.

When used for “normal” children with cpusets (when importing from XML when duplicating a topology), the caller should make sure that:

children are inserted in order,
children cpusets do not intersect.

The given object may have normal, I/O or Misc children, as long as they are in order as well. These children must have valid parent and next_sibling pointers.

The caller should check whether the object type is filtered-out before calling this function.

hwloc_obj_t hwloc_alloc_setup_object(hwloc_topology_t topology, hwloc_obj_type_t type, unsigned os_index)¶

Allocate and initialize an object of the given type and physical index.

If os_index is unknown or irrelevant, use HWLOC_UNKNOWN_INDEX.

int hwloc_obj_add_children_sets(hwloc_obj_t obj)¶

Setup object cpusets/nodesets by OR’ing its children.

Used when adding an object late in the topology. Will update the new object by OR’ing all its new children sets.

Used when PCI backend adds a hostbridge parent, when distances add a new Group, etc.

int hwloc_topology_reconnect(hwloc_topology_t topology, unsigned long flags)¶

Request a reconnection of children and levels in the topology.

May be used by backends during discovery if they need arrays or lists of object within levels or children to be fully connected.

flags is currently unused, must 0.

Components and Plugins: Filtering objects¶

group Components and Plugins: Filtering objects

Note

These structures and functions may change when HWLOC_COMPONENT_ABI is modified.

Functions

inline int hwloc_filter_check_pcidev_subtype_important(unsigned classid)¶

Check whether the given PCI device classid is important.

Returns:: 1 if important, 0 otherwise.

inline int hwloc_filter_check_osdev_subtype_important(hwloc_obj_osdev_types_t subtype)¶

Check whether the given OS device subtype is important.

Returns:: 1 if important, 0 otherwise.

inline int hwloc_filter_check_keep_object_type(hwloc_topology_t topology, hwloc_obj_type_t type)¶

Check whether a non-I/O object type should be filtered-out.

Cannot be used for I/O objects.

Returns:: 1 if the object type should be kept, 0 otherwise.

inline int hwloc_filter_check_keep_object(hwloc_topology_t topology, hwloc_obj_t obj)¶

Check whether the given object should be filtered-out.

Returns:: 1 if the object type should be kept, 0 otherwise.

Components and Plugins: helpers for PCI discovery¶

group Components and Plugins: helpers for PCI discovery

Note

These structures and functions may change when HWLOC_COMPONENT_ABI is modified.

Functions

unsigned hwloc_pcidisc_find_cap(const unsigned char *config, unsigned cap)¶

Return the offset of the given capability in the PCI config space buffer.

This function requires a 256-bytes config space. Unknown/unavailable bytes should be set to 0xff.

int hwloc_pcidisc_find_linkspeed(const unsigned char *config, unsigned offset, float *linkspeed)¶

Fill linkspeed by reading the PCI config space where PCI_CAP_ID_EXP is at position offset.

Needs 20 bytes of EXP capability block starting at offset in the config space for registers up to link status.

hwloc_obj_type_t hwloc_pcidisc_check_bridge_type(unsigned device_class, const unsigned char *config)¶

Return the hwloc object type (PCI device or Bridge) for the given class and configuration space.

This function requires 16 bytes of common configuration header at the beginning of config.

int hwloc_pcidisc_find_bridge_buses(unsigned domain, unsigned bus, unsigned dev, unsigned func, unsigned *secondary_busp, unsigned *subordinate_busp, const unsigned char *config)¶

Fills the attributes of the given PCI bridge using the given PCI config space.

This function requires 32 bytes of common configuration header at the beginning of config.

Returns -1 and destroys /p obj if bridge fields are invalid.

void hwloc_pcidisc_tree_insert_by_busid(struct hwloc_obj **treep, struct hwloc_obj *obj)¶

Insert a PCI object in the given PCI tree by looking at PCI bus IDs.

If treep points to NULL, the new object is inserted there.

int hwloc_pcidisc_tree_attach(struct hwloc_topology *topology, struct hwloc_obj *tree)¶

Add some hostbridges on top of the given tree of PCI objects and attach them to the topology.

Other backends may lookup PCI objects or localities (for instance to attach OS devices) by using hwloc_pcidisc_find_by_busid() or hwloc_pcidisc_find_busid_parent().

Components and Plugins: finding PCI objects during other discoveries¶

group Components and Plugins: finding PCI objects during other discoveries

Note

These structures and functions may change when HWLOC_COMPONENT_ABI is modified.

Functions

struct hwloc_obj *hwloc_pci_find_parent_by_busid(struct hwloc_topology *topology, unsigned domain, unsigned bus, unsigned dev, unsigned func)¶

Find the object or a parent of a PCI bus ID.

When attaching a new object (typically an OS device) whose locality is specified by PCI bus ID, this function returns the PCI object to use as a parent for attaching.

If the exact PCI device with this bus ID exists, it is returned. Otherwise (for instance if it was filtered out), the function returns another object with similar locality (for instance a parent bridge, or the local CPU Package).

struct hwloc_obj *hwloc_pci_find_by_busid(struct hwloc_topology *topology, unsigned domain, unsigned bus, unsigned dev, unsigned func)¶

Find the PCI device or bridge matching a PCI bus ID exactly.

This is useful for adding specific information about some objects based on their PCI id. When it comes to attaching objects based on PCI locality, hwloc_pci_find_parent_by_busid() should be preferred.

Components and Plugins: distances¶

group Components and Plugins: distances

Note

These structures and functions may change when HWLOC_COMPONENT_ABI is modified.

Typedefs

typedef void *hwloc_backend_distances_add_handle_t¶: Handle to a new distances structure during its addition to the topology.

Functions

hwloc_backend_distances_add_handle_t hwloc_backend_distances_add_create(hwloc_topology_t topology, const char *name, unsigned long kind, unsigned long flags)¶

Create a new empty distances structure.

This is identical to hwloc_distances_add_create() but this variant is designed for backend inserting distances during topology discovery.

int hwloc_backend_distances_add_values(hwloc_topology_t topology, hwloc_backend_distances_add_handle_t handle, unsigned nbobjs, hwloc_obj_t *objs, hwloc_uint64_t *values, unsigned long flags)¶

Specify the objects and values in a new empty distances structure.

This is similar to hwloc_distances_add_values() but this variant is designed for backend inserting distances during topology discovery.

The only semantical difference is that objs and values are not duplicated, but directly attached to the topology. On success, these arrays are given to the core and should not ever be freed by the caller anymore.

int hwloc_backend_distances_add_commit(hwloc_topology_t topology, hwloc_backend_distances_add_handle_t handle, unsigned long flags)¶

Commit a new distances structure.

This is similar to hwloc_distances_add_commit() but this variant is designed for backend inserting distances during topology discovery.