The importance of ontological structure: why validation by 'fit-to-data' is insufficient

This chapter will briefly describe some common methods by which people make quantitative estimates of how well they expect empirical models to make predictions. However, the chapter's main argument is that fit-to-data, the traditional yardstick for establishing confidence in models, is not quite the solid ground on which to build such belief some people think it is, especially for the kind of system agent-based modelling is usually applied to. Further, the chapter will show that the amount of data required to establish confidence in an arbitrary model by fit-to-data is (big data aside) often infeasible. This arbitrariness can be reduced by constraining the choice of model, and in agentbased models, these constraints are introduced by their descriptiveness rather than by removing variables from consideration or making assumptions for the sake of simplicity. By comparing with neural networks, we show that agentbased models have a richer ontological structure. For agent-based models in particular, this richness means that the ontological structure has a greater significance, and yet is all-toocommonly taken for granted or assumed to be 'common sense'. The chapter therefore also discusses some approaches to validating ontologies, though the state-of-the-art is far from a situation in which there are established standards. (Abstract from the Preprint of book chapter: Polhill, G. and Salt, D. (2017) The importance of ontological structure: why validation by 'fit-to-data' is insufficient. In Edmonds, B. and Meyer, R. Simulating Social Complexity: A Handbook, 2nd Edition. Cham: Springer. pp. 141-172 (doi: 10.1007/978-3-319-66948-9_8)