armi.mpiActions module

This module provides an abstract class to be used to implement “MPI actions.”.

MPI actions are tasks, activities, or work that can be executed on the worker nodes. The standard workflow is essentially that the primary node creates an MpiAction, sends it to the workers, and then both the primary and the workers invoke() together. For example:

Sample MPI Action Workflow

Step	Code	Notes
1	primary: distributeState = DistributeStateAction() worker: action = context.MPI_COMM.bcast(None, root=0)	primary: Initializing a distribute state action. worker: Waiting for something to do, as determined by the primary, this happens within the worker’s workerOperate().
2	primary: context.MPI_COMM.bcast(distributeState, root=0) worker: action = context.MPI_COMM.bcast(None, root=0)	primary: Broadcasts a distribute state action to all the worker nodes worker: Receives the action from the primary, which is a DistributeStateAction.
3	primary: distributeState.invoke(self.o, self.r, self.cs) worker: action.invoke(self.o, self.r, self.cs)	Both invoke the action, and are in sync. Any broadcast or receive within the action should also be synced up.

In order to create a new, custom MPI Action, inherit from MpiAction, and override the invokeHook() method.

class armi.mpiActions.MpiAction

Bases: object

Base of all MPI actions.

MPI Actions are tasks that can be executed without needing lots of other information. When a worker node sits in its main loop, and receives an MPI Action, it will simply call invoke().

property parallel

classmethod invokeAsMaster(o, r, cs)

Simplified method to call from the primary process.

This can be used in place of:

someInstance = MpiAction() someInstance = COMM_WORLD.bcast(someInstance, root=0) someInstance.invoke(o, r, cs)

Interestingly, the code above can be used in two ways:

1. Both the primary and worker can call the above code at the same time, or

2. the primary can run the above code, which will be handled by the worker’s main loop.

Option number 2 is the most common usage.

Warning

This method will not work if the constructor (i.e. __init__) requires additional arguments. Since the method body is so simple, it is strong discouraged to add a *args or **kwargs arguments to this method.

Parameters:

• o (armi.operators.Operator) – If an operator is not necessary, supply None.

• r (armi.operators.Reactor) – If a reactor is not necessary, supply None.

broadcast(obj=None)

A wrapper around bcast, on the primary node can be run with an equals sign, so that it can be consistent within both primary and worker nodes.

Parameters:

obj – This is any object that can be broadcast, if it is None, then it will broadcast itself, which triggers it to run on the workers (assuming the workers are in the worker main loop.

See also

armi.operators.operator.OperatorMPI.workerOperate

receives this on the workers and calls invoke

Notes

The standard bcast method creates a new instance even for the root process. Consequently, when passing an object, references can be broken to the original object. Therefore, this method, returns the original object when called by the primary node, or the broadcasted object when called on the worker nodes.

gather(obj=None)

A wrapper around MPI_COMM.gather.

Parameters:

obj – This is any object that can be gathered, if it is None, then it will gather itself.

Notes

The returned list will contain a reference to the original gathered object, without making a copy of it.

invoke(o, r, cs)

This method is called by worker nodes, and passed the worker node’s operator, reactor and settings file.

Parameters:

• o (armi.operators.operator.Operator) – the operator for this process

• r (armi.reactor.reactors.Reactor) – the reactor represented in this process

• cs (armi.settings.caseSettings.Settings) – the case settings

Returns:

result – result from invokeHook

Return type:

object

static mpiFlatten(allCPUResults)

Flatten results to the same order they were in before making a list of mpiIter results.

See also

mpiIter

used for distributing objects/tasks

static mpiIter(objectsForAllCoresToIter)

Generate the subset of objects one node is responsible for in MPI.

Notes

Each CPU will get similar number of objects. E.G. if there are 12 objects and 5 CPUs, the first 2 CPUs will get 3 objects and the last 3 CPUS will get 2.

Parameters:

objectsForAllCoresToIter (list) – List of all objects that need to have an MPI calculation performed on. Note, that since len() is needed this method cannot accept a generator.

See also

mpiFlatten

used for collecting results

invokeHook()

This method must be overridden in sub-clases.

This method is called by worker nodes, and has access to the worker node’s operator, reactor, and settings (through self.o, self.r, and self.cs). It must return a boolean value of True or False, otherwise the worker node will raise an exception and terminate execution.

Returns:

result – Dependent on implementation

Return type:

object

armi.mpiActions.runActions(o, r, cs, actions, numPerNode=None, serial=False)

Run a series of MpiActions in parallel, or in series if serial=True.

Notes

The number of actions DOES NOT need to match context.MPI_SIZE.

Calling this method may invoke MPI Split which will change the MPI_SIZE during the action. This allows someone to call MPI operations without being blocked by tasks which are not doing the same thing.

armi.mpiActions.runActionsInSerial(o, r, cs, actions)

Run a series of MpiActions in serial.

Notes

This will set the MpiAction.serial attribute to True, and the MpiAction.broadcast and MpiAction.gather methods will basically just return the value being supplied.

class armi.mpiActions.DistributionAction(actions)

Bases: MpiAction

This MpiAction scatters the workload of multiple actions to available resources.

Notes

This currently only works from the root (of COMM_WORLD). Eventually, it would be nice to make it possible for sub-tasks to manage their own communicators and spawn their own work within some sub-communicator.

This performs an MPI Split operation and takes over the context.MPI_COMM and associated varaibles. For this reason, it is possible that when someone thinks they have distributed information to all nodes, it may only be a subset that was necessary to perform the number of actions needed by this DsitributionAction.

invokeHook()

Overrides invokeHook to distribute work amongst available resources as requested.

Notes

Two things about this method make it non-recursive

exception armi.mpiActions.MpiActionError

Bases: Exception

Exception class raised when error conditions occur during an MpiAction.

class armi.mpiActions.DistributeStateAction(skipInterfaces=False)

Bases: MpiAction

invokeHook()

Sync up all nodes with the reactor, the cs, and the interfaces.

Notes

This is run by all workers and the primary any time the code needs to sync all processors.