armi.bookkeeping.db package

The db package is responsible for reading and writing the state of the reactor to/from disk.

As an ARMI run progresses, this is periodically updated as the primary output file. It can also be an input file for follow-on analysis or restart runs.

The database can be visualized through various tools such as XTVIEW.

This module contains factories for selecting and building DB-related objects.

Some notes on versions

Persistent storage of ARMI models has seen many changes throughout the years. Calculation results were originally stored on a SQL database (version 1), which has been fully deprecated at this point.

Version 2 was the first to use HDF5 as the primary storage format. This was beneficial, as it did not rely on any external infrastructure to operate, and benefited from the suite of tools that help interact with HDF5 files. It was eventually replaced because it did not store a complete model of the reactor, but rather a ghost of assembly, block, and reactor parameters that could be applied to an existing reactor model (so long as the dimensions were consistent!). This led to loading reactors being inconvenient and error-prone, and also posed a limitation for representing more complex systems that have non-core components.

Version 3 was created to make the schema more flexible and to permit storing the entire reactor model within the HDF5 file. All objects in the ARMI Composite Model are written to the database, and the model can be recovered in its entirety just from the HDF5 file. Since it’s inception, it has seen a number of tweaks to improve its functionality and fix bugs.

Being a serialization format, the code associated with reading and writing database files may not benefit from Don’t Repeat Yourself (DRY) practices in the same way as other code. Therefore, we do not share much, if any, code between different major versions of the databases. As such, new major-versioned database implementations should exist in their own modules. Minor revisions (e.g. M.(N+1)) to the database structure should be simple enough that specialized logic can be used to support all minor versions without posing a maintenance burden. A detailed change log should be maintained of each minor revision.

class armi.bookkeeping.db.Database3(fileName: os.PathLike, permission: str)

Bases: armi.bookkeeping.db.database.Database

Version 3 of the ARMI Database, handling serialization and loading of Reactor states.

This implementation of the database pushes all objects in the Composite Reactor Model into the database. This process is aided by the Layout class, which handles the packing and unpacking of the structure of the objects, their relationships, and their non-parameter attributes.

See also

doc/user/outputs/database for more details.

Create a new Database3 object.

Parameters

• fileName – name of the file

• permission – file permissions, write (“w”) or read (“r”)

timeNodeGroupPattern = re.compile('^c(\\d\\d)n(\\d\\d)$')

property version: str

property versionMajor

property versionMinor

open()

static grabLocalCommitHash()

Try to determine the local Git commit.

We have to be sure to handle the errors where the code is run on a system that doesn’t have Git installed. Or if the code is simply not run from inside a repo.

Returns

The commit hash if it exists, otherwise “unknown”.

Return type

str

close(completedSuccessfully=False)

Close the DB and perform cleanups and auto-conversions.

splitDatabase(keepTimeSteps: Sequence[Tuple[int, int]], label: str) → str

Discard all data except for specific time steps, retaining old data in a separate file.

This is useful when performing more exotic analyses, where each “time step” may not represent a specific point in time, but something more nuanced. For example, equilibrium cases store a new “cycle” for each iteration as it attempts to converge the equilibrium cycle. At the end of the run, the last “cycle” is the converged equilibrium cycle, whereas the previous cycles constitute the path to convergence, which we typically wish to discard before further analysis.

Parameters

• keepTimeSteps – A collection of the time steps to retain

• label – An informative label for the backed-up database. Usually something like “-all-iterations”. Will be interposed between the source name and the “.h5” extension.

Returns

The name of the new, backed-up database file.

Return type

str

property fileName

loadCS()

Attempt to load settings from the database file

Notes

There are no guarantees here. If the database was written from a different version of ARMI than you are using, these results may not be usable. For instance, the database could have been written from a vastly old or future version of ARMI from the code you are using.

loadBlueprints()

Attempt to load reactor blueprints from the database file

Notes

There are no guarantees here. If the database was written from a different version of ARMI than you are using, these results may not be usable. For instance, the database could have been written from a vastly old or future version of ARMI from the code you are using.

loadGeometry()

This is primarily just used for migrations. The “geometry files” were replaced by systems: and grids: sections of Blueprints.

writeInputsToDB(cs, csString=None, geomString=None, bpString=None)

Write inputs into the database based the CaseSettings.

This is not DRY on purpose. The goal is that any particular Database implementation should be very stable, so we dont want it to be easy to change one Database implementation’s behavior when trying to change another’s.

Notes

This is hard-coded to read the entire file contents into memory and write that directly into the database. We could have the cs/blueprints/geom write to a string, however the ARMI log file contains a hash of each files’ contents. In the future, we should be able to reproduce a calculation with confidence that the inputs are identical.

readInputsFromDB()

mergeHistory(inputDB, startCycle, startNode)

Copy time step data up to, but not including the passed cycle and node.

Notes

This is used for restart runs with the standard operator for example. The current time step (being loaded from) should not be copied, as that time steps data will be written at the end of the time step.

genTimeStepGroups(timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Generator[h5py._hl.group.Group, None, None]

Returns a generator of HDF5 Groups for all time nodes, or for the passed selection.

getLayout(cycle, node)

Return a Layout object representing the requested cycle and time node.

genTimeSteps() → Generator[Tuple[int, int], None, None]

Returns a generator of (cycle, node) tuples that are present in the DB.

genAuxiliaryData(ts: Tuple[int, int]) → Generator[str, None, None]

Returns a generator of names of auxiliary data on the requested time point.

getAuxiliaryDataPath(ts: Tuple[int, int], name: str) → str

Get a string describing a path to an auxiliary data location.

Parameters

• ts – The time step that the auxiliary data belongs to

• name – The name of the auxiliary data

Returns

An absolute location for storing auxiliary data with the given name for the given time step

Return type

str

keys()

getH5Group(r, statePointName=None)

Get the H5Group for the current ARMI timestep.

This method can be used to allow other interfaces to place data into the database at the correct timestep.

hasTimeStep(cycle, timeNode, statePointName='')

Returns True if (cycle, timeNode, statePointName) is contained in the database.

writeToDB(reactor, statePointName=None)

Write reactor data to the DB

syncToSharedFolder()

Copy DB to run working directory.

Needed when multiple MPI processes need to read the same db, for example when a history is needed from independent runs (e.g. for fuel performance on a variety of assemblies).

Notes

At some future point, we may implement a client-server like DB system which would render this kind of operation unnecessary.

load(cycle, node, cs=None, bp=None, statePointName=None, allowMissing=False)

Load a new reactor from (cycle, node).

Case settings, blueprints, and geom can be provided by the client, or read from the database itself. Providing these from the client could be useful when performing snapshot runs or the like, where it is expected to use results from a run using different settings, then continue with new settings. Even in this case, the blueprints and geom should probably be the same as the original run.

Parameters

• cycle (int) – cycle number

• node (int) – time node

• cs (armi.settings.Settings (optional)) – if not provided one is read from the database

• bp (armi.reactor.Blueprints (Optional)) – if not provided one is read from the database

• statePointName (str) – Optional arbitrary statepoint name (e.g., “special” for “c00n00-special/”)

• allowMissing (bool) – Whether to emit a warning, rather than crash if reading a database with undefined parameters. Default False.

Returns

root – The top-level object stored in the database; usually a Reactor.

Return type

ArmiObject

getHistoryByLocation(comp: armi.reactor.composites.ArmiObject, params: Optional[List[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[str, Dict[Tuple[int, int], Any]]

Get the parameter histories at a specific location.

getHistoriesByLocation(comps: Sequence[armi.reactor.composites.ArmiObject], params: Optional[List[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[armi.reactor.composites.ArmiObject, Dict[str, Dict[Tuple[int, int], Any]]]

Get the parameter histories at specific locations.

This has a number of limitations, which should in practice not be too limiting:

• The passed objects must have IndexLocations. This type of operation doesn’t make much sense otherwise.

• The passed objects must exist in a hierarchy that leads to a Core object, which serves as an anchor that can fully define all index locations. This could possibly be made more general by extending grids, but that gets a little more complicated.

• All requested objects must exist under the same anchor object, and at the same depth below it.

• All requested objects must have the same type.

Parameters

• comps (list of ArmiObject) – The components/composites that currently occupy the location that you want histories at. ArmiObjects are passed, rather than locations, because this makes it easier to figure out things related to layout.

• params (List of str, optional) – The parameter names for the parameters that we want the history of. If None, all parameter history is given

• timeSteps (List of (cycle, node) tuples, optional) – The time nodes that you want history for. If None, all available time nodes will be returned.

getHistory(comp: armi.reactor.composites.ArmiObject, params: Optional[Sequence[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[str, Dict[Tuple[int, int], Any]]

Get parameter history for a single ARMI Object.

Parameters

• comps – An individual ArmiObject

• params – parameters to gather

Returns

Dictionary of str/list pairs.

Return type

dict

getHistories(comps: Sequence[armi.reactor.composites.ArmiObject], params: Optional[Sequence[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[armi.reactor.composites.ArmiObject, Dict[str, Dict[Tuple[int, int], Any]]]

Get the parameter histories for a sequence of ARMI Objects.

This implementation is unaware of the state of the reactor outside of the database itself, and is therefore not usually what client code should be calling directly during normal ARMI operation. It only knows about historical data that have actually been written to the database. Usually one wants to be able to get historical, plus current data, for which the similar method on the DatabaseInterface may be more useful.

Parameters

• comps – Something that is iterable multiple times

• params – parameters to gather.

• timeSteps – Selection of time nodes to get data for. If omitted, return full history

Returns

Dictionary ArmiObject (input): dict of str/list pairs containing ((cycle, node), value).

Return type

dict

class armi.bookkeeping.db.DatabaseInterface(r, cs)

Bases: armi.interfaces.Interface

Handles interactions between the ARMI data model and the persistent data storage system.

This reads/writes the ARMI state to/from the database and helps derive state information that can be derived.

Construct an interface.

The r and cs arguments are required, but may be None, where appropriate for the specific Interface implementation.

Parameters

• r (Reactor) – A reactor to attach to

• cs (Settings) – Settings object to use

Raises

RuntimeError – Interfaces derived from Interface must define their name

name: Optional[str] = 'database'

The name of the interface. This is undefined for the base class, and must be overridden by any concrete class that extends this one.

property database

Presents the internal database object, if it exists.

interactBOL()

Initialize the database if the main interface was not available. (Begining of Life)

initDB(fName: Optional[os.PathLike] = None)

Open the underlying database to be written to, and write input files to DB.

Notes

Main Interface calls this so that the database is available as early as possible in the run. The database interface interacts near the end of the interface stack (so that all the parameters have been updated) while the Main Interface interacts first.

interactEveryNode(cycle, node)

Write to database.

DBs should receive the state information of the run at each node.

interactEOC(cycle=None)

In case anything changed since last cycle (e.g. rxSwing), update DB. (End of Cycle)

interactEOL()

DB’s should be closed at run’s end. (End of Life)

interactError()

Get shutdown state information even if the run encounters an error

interactDistributeState() → None

Reconnect to pre-existing database.

DB is created and managed by the master node only but we can still connect to it from workers to enable things like history tracking.

distributable()

Return true if this can be MPI broadcast.

Notes

Cases where this isn’t possible include the database interface, where the SQL driver cannot be distributed.

prepRestartRun(dbCycle, dbNode)

Load the data history from the database being restarted from.

loadState(cycle, timeNode, timeStepName='', fileName=None, updateGlobalAssemNum=True)

Loads a fresh reactor and applies it to the Operator.

Notes

Will load preferentially from the fileName if passed. Otherwise will load from existing database in memory or cs[“reloadDBName”] in that order.

Raises

RuntimeError – If fileName is specified and that file does not have the time step. If fileName is not specified and neither the database in memory, nor the cs[“reloadDBName”] have the time step specified.

getHistory(comp: armi.reactor.composites.ArmiObject, params: Optional[Sequence[str]] = None, timeSteps: Optional[MutableSequence[Tuple[int, int]]] = None, byLocation: bool = False) → Dict[str, Dict[Tuple[int, int], Any]]

Get historical parameter values for a single object.

This is mostly a wrapper around the same function on the Database3 class, but knows how to return the current value as well.

See also

Database3.getHistory

getHistories(comps: Sequence[armi.reactor.composites.ArmiObject], params: Optional[Sequence[str]] = None, timeSteps: Optional[MutableSequence[Tuple[int, int]]] = None, byLocation: bool = False) → Dict[armi.reactor.composites.ArmiObject, Dict[str, Dict[Tuple[int, int], Any]]]

Get historical parameter values for one or more objects.

This is mostly a wrapper around the same function on the Database3 class, but knows how to return the current value as well.

See also

Database3.getHistories

armi.bookkeeping.db.compareDatabases(refFileName: str, srcFileName: str, exclusions: Optional[Sequence[str]] = None, tolerance: float = 0.0) → Optional[armi.bookkeeping.db.compareDB3.DiffResults]

armi.bookkeeping.db.databaseFactory(dbName: str, permission: str, version: Optional[str] = None)

Return an appropriate object for interacting with a database file.

Parameters

• dbName (str) – Path to db file, e.g. baseCase.h5

• permission (str) – String defining permission, r for read only. See armi.bookeeping.db.permissions

• version (str, optional) – Version of database you want to read or write. In many cases ARMI will auto-detect. For advanced users.

Notes

This is not a proper factory, as the different database versions do not present a common interface. However, this is useful code, since it at least creates an object based on some knowledge of how to probe around. This allows client code to just interrogate the type of the returned object to figure out to do based on whatever it needs.

Submodules

• armi.bookkeeping.db.compareDB3 module
• armi.bookkeeping.db.database module
• armi.bookkeeping.db.database3 module
  • Minor revision changelog
• armi.bookkeeping.db.factory module
• armi.bookkeeping.db.permissions module
• armi.bookkeeping.db.typedefs module