OAFM is a prototype foundation model capturing a range of socio-demographic factors in England and Wales at the 2021 Output Area and Lower layer Super Output Area levels, as a first step towards a population dynamics foundation model for the UK. A masked heterogeneous graph autoencoder is used to create embeddings based on five open datasets available from the Geographic Data Service: the Unified UK Census, Access to Healthy Assets and Hazards (AHAH), Broadband Speed, Dwelling Ages and Prices, and Points of Interest (Overture).
Foundation models can generate general-purpose, rich and concise representations of complex information or systems. In geography, creating general, digital fingerprints integrating and compressing a broad spectrum of data on population, health, housing, mobility, the environment, and the economy of a place can drive useful insights while limiting the amount of preprocessing and complexity of models used in each specific downstream task. This record releases the embeddings produced by OAFM. Each area is described by 256 dimensions derived from the five open datasets listed above, which can be used as inputs to downstream modelling, such as prediction, clustering or similarity search. The embeddings are the raw model outputs. Downstream applications may find it profitable to standardise them (e.g., per-dimension z-score over all nodes of each type) before using them as model inputs.
Content
The model spans two spatial scales across England and Wales: the 188,880 Output Areas (2021 OAs) and the 35,672 Lower layer Super Output Areas (2021 LSOAs).
Two sets of embeddings are published, one per scale, each with 256 embedding dimensions. Both are available as Parquet and as zipped CSV.
Embedding columns are named EMB_index_source_target, where:
- index is a numerical index from 000 to 255.
- source indicates the source dataset embedded.
- target is self (dimensions reconstructing the seed's own features) or the same-type edge relation whose neighbour-average that dimension embeds (near, commute abbreviated to cmmt).
The variable dictionary listed below includes full details on the embedding columns.
Please see the project repository for comprehensive documentation of the model development, training, and input data. The model architecture code, trained-model artifacts and evaluation experiments are also available there.
The input datasets are listed in the related records, and can be downloaded from GeoDS.
Quality, Representation and Bias
The embeddings cover all Output Areas and Lower layer Super Output Areas in England and Wales.
They inherit the completeness, vintage and known limitations of their input datasets.
The development of this prototype posed a wide range of questions regarding the selection of the data, their preprocessing, the model architecture and hyperparameters, and the final evaluation of the outputs. Many choices were driven by data availability, which affected both the scale and the type of data used, and therefore the architecture necessary to achieve satisfactory results. Other choices were guided or assessed using intrinsic (i.e., validation and test losses) and extrinsic (i.e., downstream tasks) evaluation. Nevertheless, the limited time available to complete this prototype didn't allow for an extensive exploration of the opportunities and challenges, nor a thorough assessment of the model. As such, this model is provided "as is", and should be regarded as a demonstrator and a starting point for further development, rather than as a final product.
Licensed under the MIT License and contains public sector information licensed under the Open Government Licence v3.0. Contains OS data © Crown copyright and database right 2023 and 2024.
