Structure of hub repositories

A hub repository should be structured according to the following guidelines:

¹The directory is required, but the name is flexible. You can use a custom directory path by setting the "model_output_dir" property in the admin.json file. More details can be found in the admin.json schema definition.

1. Code and scripts must not be present in the model-output ^{1 1}The directory is required, but the name is flexible. You can use a custom directory path by setting the "model_output_dir" property in the admin.json file. More details can be found in the admin.json schema definition. directory of a hub repository.

2. If code is included in the hub repository, it should live in a centrally located directory, which we recommend naming src.

3. If code has the potential to disrupt or break other continuous integration operations in the hub (e.g., validation of incoming submissions), it should be moved to another repository.

The directory and file structure of a modeling hub should contain only the following directories, subdirectories, and files:

Required Components

Documentation (README.md)

Component	Location	Description	Hub provides	Modeler provides
Documentation file	e.g., README.md file, located in the top level of a hub and within each directory	File containing info about the hub structure and additional details about each of the directories	X

Configuration (hub-config/)

Component	Location	Description	Hub provides	Modeler provides
Configuration directory	hub-config/	Folder storing configuration files	X
Admin configuration file	hub-config/admin.json	Structured text file containing overall configuration settings for the hub	X
Modeling tasks configuration file	hub-config/tasks.json	Structured text file that defines modeling tasks and therefore implicity defines the assumed structure for any model submitted	X
Model metadata configuration file	hub-config/model-metadata-schema.json	Structured text file that defines the expected format of model metadata files submitted by modeling teams	X

Model Output Submissions (model-output/)

Component	Location	Description	Hub provides	Modeler provides
Model output directory	model-output/	Folder to collect modeling team model submissions	X
Model output subdirectory	model-output/team1-modela/	Model-specific subdirectory for submissions from one modeling team		X
Model output file	model-output/team1-modela/<round-id1>-<model_id>.csv or .parquet	Round-specific model submission file		X

Model Metadata (model-metadata/)

Component	Location	Description	Hub provides	Modeler provides
Model metadata directory	model-metadata/	Folder to collect modeling team model metadata submissions	X
Model metadata submission file	model-metadata/team1-modela.yml	Model-specific metadata submission file		X

Optional Components

The following components are not required for a hub, but may be useful:

Model Abstracts (model-abstracts/)

Component	Location	Description	Hub provides	Modeler provides
Model abstracts directory (optional)	model-abstracts/	Folder to collect optional round-specific model metadata	X
Model abstract subdirectory	model-abstracts/team1-modela/	Model-specific subdirectory for round-specific model metadata		X
Model abstract submission file	model-abstracts/team1-modela/<round-id1>.md	Round-specific model metadata submission		X

Data and Code

Component	Location	Description	Hub provides	Modeler provides
Target data directory	target-data/	Folder storing actual observed (i.e., target) values of an outcome (or links to external open-access sources) and information on how model targets can be calculated from target data	X
Auxiliary data directory	auxiliary-data/	Folder storing any additional data related to modeling efforts	X
Source code directory	src/	Folder storing code that is present in the hub repository	X

• Optionally, a hub may store any files necessary to define continuous integration workflows, for example for the purpose of validating submissions or updating target data.

Although most hubs to date have been housed in GitHub repositories, the proposed structure is more general and can be adapted for use on any shared filesystem.

---