dedalus.core.distributor

Classes for available data layouts and the paths between them.

Module Contents

logger

GROUP_TRANSFORMS

TRANSPOSE_LIBRARY

GROUP_TRANSPOSES

SYNC_TRANSPOSES

ALLTOALLV

class Distributor(domain, comm=None, mesh=None)

Directs parallelized distribution and transformation of fields over a domain.

Variables:

• comm (MPI communicator) – Global MPI communicator
• rank (int) – Internal MPI process number
• size (int) – Number of MPI processes
• mesh (tuple of ints, optional) – Process mesh for parallelization (default: 1-D mesh of available processes)
• comm_cart (MPI communicator) – Cartesian MPI communicator over mesh
• coords (array of ints) – Coordinates in cartesian communicator (None if outside mesh)
• layouts (list of layout objects) – Available layouts for domain
• paths (list of path objects) – Transforms and transposes between layouts

Notes

Computations are parallelized by splitting D-dimensional data fields over an R-dimensional mesh of MPI processes, where R < D. In coefficient space, we take the first R dimensions of the data to be distributed over the mesh, leaving the last (D-R) dimensions local. To transform such a data cube to grid space, we loop backwards over the D dimensions, performing each transform if the corresponding dimension is local, and performing an MPI transpose with the next dimension otherwise. This effectively bubbles the first local dimension up from the (D-R)-th to the first dimension, transforming to grid space along the way. In grid space, then, the first dimensional is local, followed by R dimensions distributed over the mesh, and the last (D-R-1) dimensions local.

The distributor object for a given domain constructs layout objects describing each of the (D+R+1) layouts (sets of transform/distribution states) and the paths between them (D transforms and R transposes).

get_layout_object(self, input)

Dereference layout identifiers.

buffer_size(self, scales)

Compute necessary buffer size (bytes) for all layouts.

class Layout(domain, mesh, coords, local, grid_space, dtype)

Object describing the data distribution for a given transform and distribution state.

Variables:

• local (array of bools) – Axis locality flags (True/False for local/distributed)
• grid_space (array of bools) – Axis grid-space flags (True/False for grid/coeff space)
• dtype (numeric type) – Data type
• methods require a tuple of the current transform scales. (All) –

global_shape(self, scales)

Compute global data shape.

blocks(self, scales)

Compute block sizes for data distribution.

start(self, scales)

Compute starting coordinates for local data.

local_shape(self, scales)

Compute local data shape.

slices(self, scales)

Compute slices for selecting local portion of global data.

buffer_size(self, scales)

Compute necessary buffer size (bytes).

class Transform(layout0, layout1, axis, basis)

Directs transforms between two layouts.

group_data(self, nfields, scales)

increment_group(self, fields)

decrement_group(self, fields)

increment_single(self, field)

Backward transform.

decrement_single(self, field)

Forward transform.

increment(self, fields)

Backward transform.

decrement(self, fields)

Forward transform.

class Transpose(layout0, layout1, axis, comm_cart)

Directs transposes between two layouts.

increment(self, fields)

Transpose from layout0 to layout1.

decrement(self, fields)

Transpose from layout1 to layout0.

increment_single(self, field)

Transpose field from layout0 to layout1.

decrement_single(self, field)

Transpose field from layout1 to layout0.

increment_group(self, *fields)

Transpose group from layout0 to layout1.

decrement_group(self, *fields)

Transpose group from layout1 to layout0.