dedalus.core.distributor
¶
Classes for available data layouts and the paths between them.
Module Contents¶
-
logger
¶
-
GROUP_TRANSFORMS
¶
-
TRANSPOSE_LIBRARY
¶
-
GROUP_TRANSPOSES
¶
-
SYNC_TRANSPOSES
¶
-
ALLTOALLV
¶
-
class
Distributor
(domain, comm=None, mesh=None)¶ Directs parallelized distribution and transformation of fields over a domain.
Variables: - comm (MPI communicator) – Global MPI communicator
- rank (int) – Internal MPI process number
- size (int) – Number of MPI processes
- mesh (tuple of ints, optional) – Process mesh for parallelization (default: 1-D mesh of available processes)
- comm_cart (MPI communicator) – Cartesian MPI communicator over mesh
- coords (array of ints) – Coordinates in cartesian communicator (None if outside mesh)
- layouts (list of layout objects) – Available layouts for domain
- paths (list of path objects) – Transforms and transposes between layouts
Notes
Computations are parallelized by splitting D-dimensional data fields over an R-dimensional mesh of MPI processes, where R < D. In coefficient space, we take the first R dimensions of the data to be distributed over the mesh, leaving the last (D-R) dimensions local. To transform such a data cube to grid space, we loop backwards over the D dimensions, performing each transform if the corresponding dimension is local, and performing an MPI transpose with the next dimension otherwise. This effectively bubbles the first local dimension up from the (D-R)-th to the first dimension, transforming to grid space along the way. In grid space, then, the first dimensional is local, followed by R dimensions distributed over the mesh, and the last (D-R-1) dimensions local.
The distributor object for a given domain constructs layout objects describing each of the (D+R+1) layouts (sets of transform/distribution states) and the paths between them (D transforms and R transposes).
-
get_layout_object
(self, input)¶ Dereference layout identifiers.
-
buffer_size
(self, scales)¶ Compute necessary buffer size (bytes) for all layouts.
-
class
Layout
(domain, mesh, coords, local, grid_space, dtype)¶ Object describing the data distribution for a given transform and distribution state.
Variables: - local (array of bools) – Axis locality flags (True/False for local/distributed)
- grid_space (array of bools) – Axis grid-space flags (True/False for grid/coeff space)
- dtype (numeric type) – Data type
- methods require a tuple of the current transform scales. (All) –
-
global_shape
(self, scales)¶ Compute global data shape.
-
blocks
(self, scales)¶ Compute block sizes for data distribution.
-
start
(self, scales)¶ Compute starting coordinates for local data.
-
local_shape
(self, scales)¶ Compute local data shape.
-
slices
(self, scales)¶ Compute slices for selecting local portion of global data.
-
buffer_size
(self, scales)¶ Compute necessary buffer size (bytes).
-
class
Transform
(layout0, layout1, axis, basis)¶ Directs transforms between two layouts.
-
group_data
(self, nfields, scales)¶
-
increment_group
(self, fields)¶
-
decrement_group
(self, fields)¶
-
increment_single
(self, field)¶ Backward transform.
-
decrement_single
(self, field)¶ Forward transform.
-
increment
(self, fields)¶ Backward transform.
-
decrement
(self, fields)¶ Forward transform.
-
-
class
Transpose
(layout0, layout1, axis, comm_cart)¶ Directs transposes between two layouts.
-
increment
(self, fields)¶ Transpose from layout0 to layout1.
-
decrement
(self, fields)¶ Transpose from layout1 to layout0.
-
increment_single
(self, field)¶ Transpose field from layout0 to layout1.
-
decrement_single
(self, field)¶ Transpose field from layout1 to layout0.
-
increment_group
(self, *fields)¶ Transpose group from layout0 to layout1.
-
decrement_group
(self, *fields)¶ Transpose group from layout1 to layout0.
-