armi.bookkeeping.memoryProfiler module

Interface to help diagnose memory issues during debugging/development.

There are many approaches to memory profiling.

1. You can ask psutil for the memory used by the process from an OS perspective. This is great for top-down analysis. This module provides printouts that show info from every process running. This is very fast.

2. You can use gc.get_objects() to list all objects that the garbage collector is tracking. If you want, you can filter it down and get the counts and sizes of objects of interest (e.g. all armi objects).

This module has tools to do all of this. It should help you out.

NOTE: Psutil and sys.getsizeof will certainly report slightly different results.

NOTE: In Windows, it seems that even if your garbage is collected, Windows does not de-allocate all the memory. So if you are a worker and you just got a 2GB reactor but then deleted it, Windows will keep you at 2GB for a while.

See also

https

//pythonhosted.org/psutil/

https

//docs.python.org/3/library/gc.html#gc.garbage

armi.bookkeeping.memoryProfiler.describeInterfaces(cs)

Function for exposing interface(s) to other code.

armi.bookkeeping.memoryProfiler.getTotalJobMemory(nTasks, cpusPerTask)

Function to calculate the total memory of a job. This is a constant during a simulation.

armi.bookkeeping.memoryProfiler.getCurrentMemoryUsage()

This scavenges the memory profiler in ARMI to get the current memory usage.

class armi.bookkeeping.memoryProfiler.MemoryProfiler(r, cs)

Bases: Interface

name: Optional[str] = 'memoryProfiler'

The name of the interface. This is undefined for the base class, and must be overridden by any concrete class that extends this one.

interactBOL()

interactEveryNode(cycle, node)

interactEOL()

End of life hook. Good place to wrap up or print out summary outputs.

printCurrentMemoryState()

Print the current memory footprint and available memory.

displayMemoryUsage(timeDescription)

Print out some information to stdout about the memory usage of ARMI.

Useful when the debugMem setting is set to True.

Turn these on as appropriate to find all your problems.

checkForDuplicateObjectsOnArmiModel(attrName, refObject)

Scans thorugh ARMI model for duplicate objects.

static getReferrers(obj)

Print referrers in a useful way (as opposed to gigabytes of text.

class armi.bookkeeping.memoryProfiler.KlassCounter(reportSize)

Bases: object

Helper class, to allow us to count instances of various classes in the Python standard library garbage collector (gc).

Counting can be done simply, or by memory footprint.

countObjects(ao)

Recursively find objects inside arbitrarily-deeply-nested containers.

This is designed to work with the garbage collector, so it focuses on objects potentially being held in dict, tuple, list, or sets.

class armi.bookkeeping.memoryProfiler.InstanceCounter(classType, reportSize)

Bases: object

add(item)

class armi.bookkeeping.memoryProfiler.ProfileMemoryUsageAction(timeDescription)

Bases: MpiAction

invokeHook()

class armi.bookkeeping.memoryProfiler.SystemAndProcessMemoryUsage

Bases: object

class armi.bookkeeping.memoryProfiler.PrintSystemMemoryUsageAction

Bases: MpiAction

property minProcessMemoryInMB

property maxProcessMemoryInMB

invokeHook()

printUsage(description=None)

This method prints the usage of all MPI nodes.

The printout looks something like:

SYS_MEM HOSTNAME 14.4% RAM. Proc mem (MB): 491 472 471 471 471 470 SYS_MEM HOSTNAME 13.9% RAM. Proc mem (MB): 474 473 472 471 460 461 SYS_MEM HOSTNAME … SYS_MEM HOSTNAME …