Model metadata#

Directory structure#

The model-metadata directory in a modeling hub is required to contain yaml metadata files, one for each model that is submitting models such as forecasts to the hub. So the contents of the directory will be a set of files named as follows:

  • team1-modela.yml

  • team1-modelb.yml

  • team2-modela.yml

Note that the same team can submit more than one model.


Model metadata describe characteristics of models contributing to a Hub.
General goals for model metadata are that

  1. explicit requirements for model metadata should be minimal

  2. hubs may set fields to be required as necessary for that hub.

Note: Model metadata is a required feature of a hub and describes the characteristics of models contributing to a Hub, while Model abstracts is an optional feature of a hub that is useful for keeping track of round-specific detailed narrative descriptions modeling methods and results.

Template metadata schema file#

The following is a template model-metadata-schema.json file from the Example Complex Forecast Hub, which is itself drawn from the FluSight Forecast Hub. Many Hubs will use the same fields listed in the template metadata schema file, but some may need fewer or more. A Hub’s model metadata schema file should include either the single combined field model_id, or both of the fields team_abbr and model_abbr. Administrators can remove other fields as needed, and can use additional optional metadata fields from this list. We encourage administrators to use the same standard names and definitions to maintain consistency across Hubs.

Optional additional metadata fields#

Depending on the goals and context of a particular Hub, it may be beneficial to collect more information about the modeling assumptions made by models. The following fields have been included within the model_details metadata field in past Hubs. Where applicable, we recommend that hubs draw from this list to encourage standardization of terminology across Hubs. If Hubs develop new model metadata items, they should be added to this list for future reference.