armi.bookkeeping.db package

The db package is responsible for reading and writing the state of the reactor to/from disk.

As an ARMI run progresses, this is periodically updated as the primary output file. It can also be an input file for follow-on analysis or restart runs.

This module contains factories for selecting and building DB-related objects.

When updating a db version

The code associated with reading and writing database files may not benefit from Don’t Repeat Yourself (DRY) practices in the same way as other code. Therefore, do not share code between different major versions of the databases. Create a new module if you are creating a new major database version.

Database revision changelog

• 1: Originally, calculation results were stored in a SQL database.

• 2: The storage format was changed to HDF5. This required less external infrastructure than SQL. However, the implementation did not store a complete model of a reactor, but a ghost of assembly, block, and reactor parameters that could be applied to an existing reactor model (so long as the dimensions were consistent). This was inconvenient and error prone.

• 3: The HDF5 format was kept, but the schema was made more flexible to permit storing the entire reactor model. All objects in the ARMI Composite Model are written to the database, and the model can be completely recovered from just the HDF5 file.

  • 3.1: Improved the handling of reading/writing grids.

  • 3.2: Changed the strategy for storing large attributes to using a special string starting with an “@” symbol (e.g., “@/c00n00/attrs/5_linkedDims”). This was done to support copying time node datasets from one file to another without invalidating the references. Support was maintained for reading previous versions, by performing a mergeHistory() and converting to the new naming strategy, but the old version cannot be written.

  • 3.3: Compressed the way locations are stored in the database and allow MultiIndex locations to be read and written.

  • 3.4: Modified the way locations are stored in the database to include complete indices for indices that can be composed from multiple grids. Having complete indices allows for more efficient means of extracting information based on location, without having to compose the full model.

class armi.bookkeeping.db.Database(fileName: PathLike, permission: str)

Bases: object

ARMI Database, handling serialization and loading of Reactor states.

This implementation of the database pushes all objects in the Composite Reactor Model into the database. This process is aided by the Layout class, which handles the packing and unpacking of the structure of the objects, their relationships, and their non-parameter attributes.

See also

doc/user/outputs/database for more details.

Create a new Database object.

Parameters:

• fileName – name of the file

• permission – file permissions, write (“w”) or read (“r”)

timeNodeGroupPattern = re.compile('^c(\\d\\d)n(\\d\\d).*$')

property version: str

property versionMajor

property versionMinor

open()

isOpen()

static writeSystemAttributes(h5db)

Write system attributes to the database.

static grabLocalCommitHash()

Try to determine the local Git commit.

We have to be sure to handle the errors where the code is run on a system that doesn’t have Git installed. Or if the code is simply not run from inside a repo.

Returns:

The commit hash if it exists, otherwise “unknown”.

Return type:

str

close(completedSuccessfully=False)

Close the DB and perform cleanups and auto-conversions.

splitDatabase(keepTimeSteps: Sequence[Tuple[int, int]], label: str) → str

Discard all data except for specific time steps, retaining old data in a separate file.

This is useful when performing more exotic analyses, where each “time step” may not represent a specific point in time, but something more nuanced. For example, equilibrium cases store a new “cycle” for each iteration as it attempts to converge the equilibrium cycle. At the end of the run, the last “cycle” is the converged equilibrium cycle, whereas the previous cycles constitute the path to convergence, which we typically wish to discard before further analysis.

Parameters:

• keepTimeSteps – A collection of the time steps to retain

• label – An informative label for the backed-up database. Usually something like “-all-iterations”. Will be interposed between the source name and the “.h5” extension.

Returns:

The name of the new, backed-up database file.

Return type:

str

property fileName

loadCS()

Attempt to load settings from the database file.

Notes

There are no guarantees here. If the database was written from a different version of ARMI than you are using, these results may not be usable. For instance, the database could have been written from a vastly old or future version of ARMI from the code you are using.

loadBlueprints()

Attempt to load reactor blueprints from the database file.

Notes

There are no guarantees here. If the database was written from a different version of ARMI than you are using, these results may not be usable. For instance, the database could have been written from a vastly old or future version of ARMI from the code you are using.

loadGeometry()

This is primarily just used for migrations.

The “geometry files” were replaced by systems: and grids: sections of Blueprints.

writeInputsToDB(cs, csString=None, geomString=None, bpString=None)

Write inputs into the database based the Settings.

This is not DRY on purpose. The goal is that any particular Database implementation should be very stable, so we dont want it to be easy to change one Database implementation’s behavior when trying to change another’s.

Notes

This is hard-coded to read the entire file contents into memory and write that directly into the database. We could have the cs/blueprints/geom write to a string, however the ARMI log file contains a hash of each files’ contents. In the future, we should be able to reproduce a calculation with confidence that the inputs are identical.

readInputsFromDB()

mergeHistory(inputDB, startCycle, startNode)

Copy time step data up to, but not including the passed cycle and node.

Notes

This is used for restart runs with the standard operator for example. The current time step (being loaded from) should not be copied, as that time steps data will be written at the end of the time step.

genTimeStepGroups(timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Generator[Group, None, None]

Returns a generator of HDF5 Groups for all time nodes, or for the passed selection.

getLayout(cycle, node)

Return a Layout object representing the requested cycle and time node.

genTimeSteps() → Generator[Tuple[int, int], None, None]

Returns a generator of (cycle, node) tuples that are present in the DB.

genAuxiliaryData(ts: Tuple[int, int]) → Generator[str, None, None]

Returns a generator of names of auxiliary data on the requested time point.

static getAuxiliaryDataPath(ts: Tuple[int, int], name: str) → str

keys()

getH5Group(r, statePointName=None)

Get the H5Group for the current ARMI timestep.

This method can be used to allow other interfaces to place data into the database at the correct timestep.

hasTimeStep(cycle, timeNode, statePointName='')

Returns True if (cycle, timeNode, statePointName) is contained in the database.

writeToDB(reactor, statePointName=None)

syncToSharedFolder()

Copy DB to run working directory.

Needed when multiple MPI processes need to read the same db, for example when a history is needed from independent runs (e.g. for fuel performance on a variety of assemblies).

Notes

At some future point, we may implement a client-server like DB system which would render this kind of operation unnecessary.

load(cycle, node, cs=None, bp=None, statePointName=None, allowMissing=False)

Load a new reactor from a DB at (cycle, node).

Case settings and blueprints can be provided, or read from the database. Providing these can be useful for snapshot runs or when you want to change settings mid-simulation. Geometry is read from the database.

Parameters:

• cycle (int) – Cycle number

• node (int) – Time node. If value is negative, will be indexed from EOC backwards like a list.

• cs (armi.settings.Settings, optional) – If not provided one is read from the database

• bp (armi.reactor.Blueprints, optional) – If not provided one is read from the database

• statePointName (str, optional) – Statepoint name (e.g., “special” for “c00n00-special/”)

• allowMissing (bool, optional) – Whether to emit a warning, rather than crash if reading a database with undefined parameters. Default False.

Returns:

root – The top-level object stored in the database; a Reactor.

Return type:

Reactor

loadReadOnly(cycle, node, statePointName=None)

Load a new reactor, in read-only mode from a DB at (cycle, node).

Parameters:

• cycle (int) – Cycle number

• node (int) – Time node. If value is negative, will be indexed from EOC backwards like a list.

• statePointName (str, optional) – Statepoint name (e.g., “special” for “c00n00-special/”)

Returns:

The top-level object stored in the database; a Reactor.

Return type:

Reactor

getHistoryByLocation(comp: ArmiObject, params: Optional[List[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[str, Dict[Tuple[int, int], Any]]

Get the parameter histories at a specific location.

getHistoriesByLocation(comps: Sequence[ArmiObject], params: Optional[List[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[ArmiObject, Dict[str, Dict[Tuple[int, int], Any]]]

Get the parameter histories at specific locations.

This has a number of limitations, which should in practice not be too limiting:

• The passed objects must have IndexLocations. This type of operation doesn’t make much sense otherwise.

• The passed objects must exist in a hierarchy that leads to a Core object, which serves as an anchor that can fully define all index locations. This could possibly be made more general by extending grids, but that gets a little more complicated.

• All requested objects must exist under the same anchor object, and at the same depth below it.

• All requested objects must have the same type.

Parameters:

• comps (list of ArmiObject) – The components/composites that currently occupy the location that you want histories at. ArmiObjects are passed, rather than locations, because this makes it easier to figure out things related to layout.

• params (List of str, optional) – The parameter names for the parameters that we want the history of. If None, all parameter history is given

• timeSteps (List of (cycle, node) tuples, optional) – The time nodes that you want history for. If None, all available time nodes will be returned.

getHistory(comp: ArmiObject, params: Optional[Sequence[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[str, Dict[Tuple[int, int], Any]]

Get parameter history for a single ARMI Object.

Parameters:

• comps – An individual ArmiObject

• params – parameters to gather

Returns:

Dictionary of str/list pairs.

Return type:

dict

getHistories(comps: Sequence[ArmiObject], params: Optional[Sequence[str]] = None, timeSteps: Optional[Sequence[Tuple[int, int]]] = None) → Dict[ArmiObject, Dict[str, Dict[Tuple[int, int], Any]]]

Get the parameter histories for a sequence of ARMI Objects.

This implementation is unaware of the state of the reactor outside of the database itself, and is therefore not usually what client code should be calling directly during normal ARMI operation. It only knows about historical data that have actually been written to the database. Usually one wants to be able to get historical, plus current data, for which the similar method on the DatabaseInterface may be more useful.

Parameters:

• comps – Something that is iterable multiple times

• params – parameters to gather.

• timeSteps – Selection of time nodes to get data for. If omitted, return full history

Returns:

Dictionary ArmiObject (input): dict of str/list pairs containing ((cycle, node), value).

Return type:

dict

class armi.bookkeeping.db.Database3(fileName: PathLike, permission: str)

Bases: Database

Temporary placeholder to ease the API-breaking transition from Database3 to Database.

Create a new Database object.

Parameters:

• fileName – name of the file

• permission – file permissions, write (“w”) or read (“r”)

h5db: Optional[File]

class armi.bookkeeping.db.DatabaseInterface(r, cs)

Bases: Interface

Handles interactions between the ARMI data model and the persistent data storage system.

This reads/writes the ARMI state to/from the database and helps derive state information that can be derived.

name: Optional[str] = 'database'

The name of the interface. This is undefined for the base class, and must be overridden by any concrete class that extends this one.

property database

Presents the internal database object, if it exists.

interactBOL()

Initialize the database if the main interface was not available. (Begining of Life).

initDB(fName: Optional[PathLike] = None)

Open the underlying database to be written to, and write input files to DB.

Notes

Main Interface calls this so that the database is available as early as possible in the run. The database interface interacts near the end of the interface stack (so that all the parameters have been updated) while the Main Interface interacts first.

interactEveryNode(cycle, node)

Write to database.

DBs should receive the state information of the run at each node.

Notes

• if tight coupling is enabled, the DB will be written in operator.py::Operator::_timeNodeLoop via writeDBEveryNode

writeDBEveryNode()

Write the database at the end of the time node.

interactEOC(cycle=None)

Dont write; this state doesn’t tend to be important since its decay only step.

Notes

The same time is available at start of next cycle.

interactEOL()

DB’s should be closed at run’s end. (End of Life).

closeDB()

Close the DB, writing to file.

interactError()

Get shutdown state information even if the run encounters an error.

interactDistributeState() → None

Reconnect to pre-existing database.

DB is created and managed by the primary node only but we can still connect to it from workers to enable things like history tracking.

distributable()

prepRestartRun()

Load the data history from the database requested in the case setting reloadDBName.

Reactor state is put at the cycle/node requested in the case settings startCycle and startNode, having loaded the state from all cycles prior to that in the requested database.

Notes

Mixing the use of simple vs detailed cycles settings is allowed, provided that the cycle histories prior to startCycle/startNode are equivalent.

ARMI expects the reload DB to have been made in the same version of ARMI as you are running. ARMI does not gaurantee that a DB from a decade ago will be easily used to restart a run.

loadState(cycle, timeNode, timeStepName='', fileName=None)

Loads a fresh reactor and applies it to the Operator.

Notes

Will load preferentially from the fileName if passed. Otherwise will load from existing database in memory or cs[“reloadDBName”] in that order.

Raises:

RuntimeError – If fileName is specified and that file does not have the time step. If fileName is not specified and neither the database in memory, nor the cs[“reloadDBName”] have the time step specified.

getHistory(comp: ArmiObject, params: Optional[Sequence[str]] = None, timeSteps: Optional[MutableSequence[Tuple[int, int]]] = None, byLocation: bool = False) → Dict[str, Dict[Tuple[int, int], Any]]

Get historical parameter values for a single object.

This is mostly a wrapper around the same function on the Database class, but knows how to return the current value as well.

See also

Database.getHistory

getHistories(comps: Sequence[ArmiObject], params: Optional[Sequence[str]] = None, timeSteps: Optional[MutableSequence[Tuple[int, int]]] = None, byLocation: bool = False) → Dict[ArmiObject, Dict[str, Dict[Tuple[int, int], Any]]]

Get historical parameter values for one or more objects.

This is mostly a wrapper around the same function on the Database class, but knows how to return the current value as well.

See also

Database.getHistories

armi.bookkeeping.db.compareDatabases(refFileName: str, srcFileName: str, exclusions: Optional[Sequence[str]] = None, tolerance: float = 0.0, timestepCompare: Optional[Sequence[Tuple[int, int]]] = None) → Optional[DiffResults]

High-level method to compare two ARMI H5 files, given file paths.

armi.bookkeeping.db.databaseFactory(dbName: str, permission: str, version: Optional[str] = None)

Return an appropriate object for interacting with a database file.

Parameters:

• dbName (str) – Path to db file, e.g. baseCase.h5

• permission (str) – String defining permission, r for read only. See armi.bookeeping.db.permissions

• version (str, optional) – Version of database you want to read or write. In most cases ARMI will auto-detect. For advanced users.

Notes

This is not a proper factory, as the different database versions do not present a common interface. However, this is useful code, since it at least creates an object based on some knowledge of how to probe around. This allows client code to just interrogate the type of the returned object to figure out to do based on whatever it needs.

armi.bookkeeping.db.loadOperator(pathToDb, loadCycle, loadNode, statePointName=None, allowMissing=False)

Return an operator given the path to a database.

Parameters:

• pathToDb (str) – The path of the database to load from.

• loadCycle (int) – The cycle to load the reactor state from.

• loadNode (int) – The time node to load the reactor from.

• statePointName (str) – State point name at the end, E.G. EOC or EOL. Full name would be C0N2EOC, see database.getH5GroupName

• allowMissing (bool) – Whether to emit a warning, rather than crash if reading a database with undefined parameters. Default False.

See also

armi.operator.Operator.loadState

A method for loading reactor state that is useful if you already have an operator and a reactor object. loadOperator varies in that it supplies these given only a database file. loadState should be used if you are in the middle of an ARMI calculation and need load a different time step.

Notes

The operator will have a reactor attached that is loaded to the specified cycle and node. The operator will not be in the same state that it was at that cycle and node, only the reactor.

Examples

>>> o = db.loadOperator(r"pathToDatabase", 0, 1)
>>> r = o.r
>>> cs = o.cs
>>> r.p.timeNode
1
>>> r.getFPMass()  # Note since it is loaded from step 1 there are fission products.
12345.67

Subpackages

• armi.bookkeeping.db.tests package
  • Submodules
    • armi.bookkeeping.db.tests.test_comparedb3 module
      • TestCompareDB3
    • armi.bookkeeping.db.tests.test_database3 module
      • TestDatabase
      • TestDatabaseSmaller
      • TestWriteReadDatabase
    • armi.bookkeeping.db.tests.test_databaseInterface module
      • getSimpleDBOperator()
      • MockInterface
      • TestDatabaseInterfaceBOL
      • TestDatabaseInterface
      • TestDatabaseWriter
      • TestDatabaseReading
      • TestBadName
      • TestStandardFollowOn
    • armi.bookkeeping.db.tests.test_jaggedArray module
      • TestJaggedArray
    • armi.bookkeeping.db.tests.test_layout module
      • TestLocationPacking

Submodules

• armi.bookkeeping.db.compareDB3 module
  • OutputWriter
    • OutputWriter.writeln()
  • DiffResults
    • DiffResults.addDiff()
    • DiffResults.addStructureDiffs()
    • DiffResults.addTimeStep()
    • DiffResults.reportDiffs()
    • DiffResults.nDiffs()
  • compareDatabases()
• armi.bookkeeping.db.database module
  • getH5GroupName()
  • Database
    • Database.timeNodeGroupPattern
    • Database.h5db
    • Database.version
    • Database.versionMajor
    • Database.versionMinor
    • Database.open()
    • Database.isOpen()
    • Database.writeSystemAttributes()
    • Database.grabLocalCommitHash()
    • Database.close()
    • Database.splitDatabase()
    • Database.fileName
    • Database.loadCS()
    • Database.loadBlueprints()
    • Database.loadGeometry()
    • Database.writeInputsToDB()
    • Database.readInputsFromDB()
    • Database.mergeHistory()
    • Database.genTimeStepGroups()
    • Database.getLayout()
    • Database.genTimeSteps()
    • Database.genAuxiliaryData()
    • Database.getAuxiliaryDataPath()
    • Database.keys()
    • Database.getH5Group()
    • Database.hasTimeStep()
    • Database.writeToDB()
    • Database.syncToSharedFolder()
    • Database.load()
    • Database.loadReadOnly()
    • Database.getHistoryByLocation()
    • Database.getHistoriesByLocation()
    • Database.getHistory()
    • Database.getHistories()
  • packSpecialData()
  • unpackSpecialData()
  • collectBlockNumberDensities()
• armi.bookkeeping.db.database3 module
  • Database3
    • Database3.h5db
• armi.bookkeeping.db.databaseInterface module
  • describeInterfaces()
  • DatabaseInterface
    • DatabaseInterface.name
    • DatabaseInterface.database
    • DatabaseInterface.interactBOL()
    • DatabaseInterface.initDB()
    • DatabaseInterface.interactEveryNode()
    • DatabaseInterface.writeDBEveryNode()
    • DatabaseInterface.interactEOC()
    • DatabaseInterface.interactEOL()
    • DatabaseInterface.closeDB()
    • DatabaseInterface.interactError()
    • DatabaseInterface.interactDistributeState()
    • DatabaseInterface.distributable()
    • DatabaseInterface.prepRestartRun()
    • DatabaseInterface.loadState()
    • DatabaseInterface.getHistory()
    • DatabaseInterface.getHistories()
• armi.bookkeeping.db.factory module
  • databaseFactory()
• armi.bookkeeping.db.jaggedArray module
  • JaggedArray
    • JaggedArray.flatten()
    • JaggedArray.fromH5()
    • JaggedArray.tolist()
    • JaggedArray.unpack()
• armi.bookkeeping.db.layout module
  • Layout
    • Layout.writeToDB()
    • Layout.computeAncestors()
    • Layout.allSubclasses()
  • replaceNonesWithNonsense()
  • replaceNonsenseWithNones()
• armi.bookkeeping.db.passiveDBLoadPlugin module
  • PassThroughYamlize
    • PassThroughYamlize.from_yaml()
    • PassThroughYamlize.attributes
  • PassiveDBLoadPlugin
    • PassiveDBLoadPlugin.SKIP_BP_SECTIONS
    • PassiveDBLoadPlugin.UNKNOWN_PARAMS
    • PassiveDBLoadPlugin.defineBlueprintsSections()
    • PassiveDBLoadPlugin.defineParameters()
    • PassiveDBLoadPlugin.buildParamColl()
• armi.bookkeeping.db.permissions module
  • Permissions
    • Permissions.READ_ONLY_FME
    • Permissions.READ_WRITE_FME
    • Permissions.CREATE_FILE_TIE
    • Permissions.CREATE_FILE_FIE
    • Permissions.CREATE_FILE_FIE2
    • Permissions.READ_WRITE_CREATE
    • Permissions.DEFAULT
    • Permissions.read
    • Permissions.write
    • Permissions.create
    • Permissions.all
• armi.bookkeeping.db.typedefs module