Health Systems Simulation Methods Draft

From HPSIGWiki

Jump to: navigation, search

Contents

[edit] Aim

See Aim and Introduction Word Version from Johanna circulated by e-mail

[edit] Overarching Theme

Sustaining Australia's Health: Extending our understanding of the interactions among policy, practice, knowledge, workforce expertise and technology through systems simulation and in silico experiments for designing testing and implementing health systems and organisational change.

[edit] Focus

Impact of CPOE on ED Performance and Acute Patient Flows

[edit] Innovation

Here we develop a way to ensure sustainable ongoing improvement based on initial proof of concept and growing interest in the literature

[edit] Our approach

How we structured the problem consistent with Overarching theme.

  • (farah AMIA2006 paper)
  • Simulation of the dynamics of systems, focussed on explaining a puzzling dynamic (e.g. why do medication errors often increase after inplementing CPOE
  • See Acute Aged Care Interface Work (ask Geoff) for background
  • Basic "physics of the system" scaffolding (facts and structures and interactions accepted by nearly all well-formed and wise people, but beware groupthink)
  • Mental models (debatable)Additional structures (variables and

A dynamic systems simulation is a hypothesis about the structure (variables and their interactions) that is both necessary and sufficient to account for the specific focussed behaviour of interest (a micro-content theory). This is the relevant context and detail to explain the behaviour of interest. Every piece of the model, a loop, or a stock-flow chain or an interaction equation may represent a little theory of how you think reality works. You also hypothesise that these little peices fit together into a larger theory that provides an explanation for the puzzling behaviour of interest. The simulation tests this hypothesis. This is valuable whenever decisionmakers misperceive the system and draw wrong conclusions about behaviour and therefore about the choice of policies.

A theory usually denotes an established thesis or mental view (mental model). A hypothesis denotes a case (often derived from some general theory by deduction), and the simulation can terst the hypothesis against data. In general, though, hypotheses are intuitive and can be explored or abducted or triangulated by searching for key structure (the important stock and flow relationships, feedback loops and non-linearities. THe hypothesis concerns the model structure (equations or equivalent graphical representation). Behaviour can be derived from structure by simulation. Behaviour modes of interest include overshoot and collapse or damped oscillation with a certain period (e.g. health workforce over- and under-supply cycles). Unfortunately critics want to impose their own purpose on the model. For example, We build a model with the purpose of determining the influence of China and India's economic variables on the rate of oil depletion. Interested parties ask "When will the oil run out, that's what's keeping me awake at night". But the model was not built to estimate the full depletion time. That is just too hard for now.

[edit] Justification

[edit] Problem with the usual methods

RCT confounders and counterfactuals in attributing cause and effect in complex interventions in complex systems that take years or are in reality always a work in progress

Usual methods are difficult to use for health services research e.g MET trial clustered randomised not enough power too long too many confounders and counterfactuals

[edit] Recent Turning to Systems Science

[edit] Dearth of quality work

  • In Public Health and Health Service Delivery (see Fone) in general and

also in the ED setting the use of simulation is sporadic with little reported lasting impact and simplistic locally available Commercial tools, such as

[edit] Systems Theory and Practice

Wide range of disciplines (Eng science Qual Quant OR MS OS, with differing philosophical bases for their own systems theory based on differing fundamental worldviews of the basis of of knowledge, in both epistemology (how we know) and ontologies (differences in meaning)

[edit] Theory and Philosophy

Organization Studies 2005 26: 1377-1404.

  • Addition of Pattern oriented modelling to the original table is slide x in Media:Sim_Methods_Slides.ppt
  • Review Science 11 Nov 2005 v310 p987

Pattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology Grimm et al.

  • Mingers J A Classification of the Philosophical Assumptions of Management Science Methods

J. Operational Research Society Volume 54 Number 6 pages 559-570 (2003)

  • Mingers J and Rosenhead J Problem Structuring Methods in Action

Eur.J of Operational Research Volume 152 Number 3 pages 530-554 (2004)

  • A Critique of Statistical Modelling in Management Science from a Critical Realist Perspective: its Role within Multimethodology

J. Operational Research Society 57, 2, pp. 202-219.(2006)

Available online at http://www.palgrave-journals.com/cgi-taf/dynapage.taf?file=/jors/journal/vaop/ncurrent/abs/2601980a.html ( Get it, Geoff can't get access from home!)

  • Barton J and Haslett T Fresh Insights into System Dynamics Methodology-Developing an abductive inference perspective Draft Submission ISDC2006 Nijmegen July (geoff has it)

[edit] Systems Modelling Practice

  • Pidd, Michael ed. Systems modelling: Theory and Practice ISBN 0-470-86731-0 Wiley
  • Key Figures from this book will be slides in Media:Sim_Methods_Slides.ppt done by Farah
    • Slide xx Ch 1 Fig 1.1 p2 combined with Fig 1.3 p8
    • fig 2.6 p39 * Ch 2
  • Jonathan Rosenhead LSE has many references here including Health Apps of Operations Research (OR) (aka Management Science (MS)) mentioned in the health specific section below
[edit] Decision vs Thinking
  • The left hand side of the composite Pidd diagram which is slide xx

Purpose of the model (again summarised in the composite diagram Slide xx

  • Tools for routine decision making (solving puzzles and problems)

Automation of decisions

Routine decision support (with some human interaction)

  • Tools for thinking

high degree of human interaction, used for solving problems and exploring messes or wicked problems

Representing possible system designs and changes

Representing insights for debate

Theory development

[edit] Health Methods Landscape

Importance of the Interaction between developer and user

  • Tools/technologies used (See Andrew's Table Slide x)
    • Quantitative (hard OR (MS)), DES SD AB CAS DS Maths Compartment SPC
    • Qualitative (soft OR) See Midgely for a recent review)
[edit] Prediction vs Projection
  • Time frame of interest See table of the 3Fs slide XX
    • Short term (Forecasting) reduce uncertainty for a precise point in time
    • Long term (Foresighting) scenarios and robust policy design for sustainability
    • Intermediate (Forestalling)scenarios and robust policy design for medium scale change and postponement of collapse
  • Forrester Watching vs shaping the future slide xxx explains Point vs Pattern Predictions
  • Time scale (dt) of model

Note the model of impact of CPOE on ED performance model we propose falls into middle range of forestalling

Prediction states a condition of reality at some future point in time ("It will rain in Sydney tomorrow"). The purpose is to predict the future as accurately as possible.

Projection states a possible condition of future reality based on a set of assumptions (If it is hot tonight, then it will rain tomorrow"). Projection is to demonstrate that a given structure (a necessary and sufficient hypothesis) produces a certain behaviour of interest. The simulation tests this hypothesis.

[edit] Practical Use

  • Collecting and integrating data and evidence (data analysis and mining)
  • Developing options (policy)
  • Choosing options (politics)
  • Optimising
  • Problem structuring

Goal seeking prescription (measurement) vs Collaboration artefact for learning (insights)

Level of detail/aggregation (fine or coarse grain)

Strategic or operational

Diversity and disciplines of participants

Data driven Theory driven Intuition driven

Qual quant, stats, dynamics, linear, circular causation Feedback, delays)

Socially constructed vs Objective reality

Testable in silico

Hypothesis formation

What if explorations

[edit] Research Plan

Based on the purpose of the model we plan to build conceptual models of the four areas of policy, practice, workforce expertise and technology that are deemed relevant to the impact of CPOE on ED performance by domain experts and confirmed by literature review. We will also build a top down SDDE model of the interactions among the four areas in the overarching theme of relevant policy, clinical practice, workforce expertise and technology.

[edit] Dynamic Hypotheses

Provides the point of Focus on explaining Puzzling dynamics from observations (See trends in Barbara Daly's thesis...e.g. lots of improvement initiatives but the problem keeps coming back and model the structure considered relevant to those Structures. THen separate out into physics (agreed widely) and mental models (contested).

[edit] Study Design

[edit] Identify KPIs

Identify a balanced set or scorecard of key measures of ED and Acute PAtient FLow Performance (The term access block implies that access to the ward is blocked, however ththe definition of access block should be the time between ED work completion and ED exit for admitted patients. THere are two confounders here firstly the ED Admit Discharges for which there are no consistent policies across the time of day day of week and different EDs. Also the time of ED work completion, including all paperwork and transport preparations is not well recorded. In our previous ED simulation work we showed you could have perceived access block with the rest of the Hospital empty.

This is indeed a puzzling dynamic and was therefore the point of focus of the previous ED study

This appeared due to problems with consistent data, rework due to poorly supervised inexperienced staff, excessive time in workup esp awaiting Xrays and extenal Specialist review and a perception that the safest place for a sick patient overnight was the ED, not the wards, nor even ICU !

[edit] Identify Key Interventions to improve Acute PAtient Flow

[edit] Develop interaction hypotheses

These are hypotheses about how these interventions interact with each other and how ICT could potentially effect these inteventions and their interactions.

[edit] Build a simulation that captures these hypotheses and the relevant context

[edit] Calibrate with real data from several sites

This includes explaining differences among sites that affect performance

[edit] Perform in silico experiments

to identify key explanatory variables for the impact of CPOE on Performance (first ED then later Acute Patient flow).

[edit] Test the impact of the Simulator experiments on Mental models of a range of relevant decisionmakers

[edit] Activities and Milestones

[edit] Assessing Validity

Explain in a convincing and inoffensive way

[edit] Conceptual Validity

[edit] Simulation Verification

[edit] Simulation Validity

[edit] Reliability

[edit] General Applicability

  • Levy Smoking is a good example of a model that has been used in many different US states and several countries

How to apply it effectively in the right way

[edit] Deployment

  • Other ED/Hosp acute care patient flow sites
  • Apply methods to Other Health & Systems Problems
Retrieved from "http://hpsig.com/index.php/Health_Systems_Simulation_Methods_Draft"
Personal tools