Edith Hall (in the piece I shared six weeks ago) writes about how Aristotle thought about decision making. You might think this reading is better suited to a philosophy class than a network reflecting on how we learn. I will come back to Hall’s chapter and Aristotle’s observations shortly but first let me explain why I think the reading is relevant.
For the last couple of years I have been collaborating with the Center for the Study of Social Policy in Washington DC and the School of Government at the Monk School of Global Affairs and Public Policy at the University of Toronto. Collectively we have been trying to answer the question: ‘what would happen if the relationship between evidence and policy was demand led not supply led’?
What does that mean? It seems to us that the use of evidence is over governed by the type of evidence knowledge organisations are able to supply. But what if the problem being faced by a policy maker demanded different kinds of method from those currently on offer in the major evidence shops?
We asked policy makers in Australia, U.S. And U.K. to: (i) describe the policy decisions they were trying to make; (ii) what kinds of evidence would be helpful in making those decisions; (iii) what kinds of evidence did they end up using?
Beautiful. A logical, elegant design. Except it didn’t work. We found the policy makers – we cannot name them but we are talking about very smart, senior people – were not very good at identifying the decision points. The policies were well meaning but not so well defined. And they mostly used knowledge already on offer in the main evidence shops, and came away as quite unhappy customers.
Why didn’t this design of ours work? What would clear decision points in policy look like? It led me back to the research I had done for policy makers in central and local government and foundations in Ireland, U.K. and the U.S. The following table gives examples of the questions we asked and the decisions made by policy makers as a result.
A lot of the early work was for (and with) senior civil servants who thought about decisions at more or less the same level of generality as we thought about research questions. We were not down in the weeds, and nor we up in the heavens. They told us what challenged them. We came up with a research question. They pushed back a bit. Once we agreed we found a method to answer the question. (We didn’t have, I still don’t have, a set method. Qualitative, quantitative, machine learning, experimental, epidemiology, as long as it is relevant to the question and rigorous then I am happy). They didn’t tell us how to do our work, we didn’t tell them what the policy was. Happy days!
It began to brake down in the Blair years. (This is not a commentary on politics, but on the way in which government began to operate differently, in full blown new public management mode, with civil servant making more and more rapid decisions, and with confidence that they were right. We found ourselves adding evidence like a cosmetic to make it look better).
They asked us to evaluate a programme called Choice Protects, based on the proposition that more placement stability for foster children, better matching of child and placement and more choice for the child would reduce placement breakdowns.
What was different? First there wasn’t really a question. It was assumed from the start that choice would protect. Second, there wasn’t much definition around the policy. What did stability mean? What would matching child and placement involve, and given shortages of supply would it even be possible? What kinds of choice would be given to the young people? Third, the level of generality was wrong. It wasn’t about the relationship between state and family, it was the policy maker deciding the shape of the service at a detailed level from afar.
So we turned down the offer, and switched direction into a new area of research. Someone else evaluated Choice Protects and their graphs went in the right direction. I am not convinced much has changed for children in care, but what do I know?
Coming back to the main argument. A question is fundamental to research. Q-H-M is my mantra to my research students. Question, linked to Hypothesis linked to Method. Don’t start with a method and find a question to fit.
By the same token, decision is, I suggest, fundamental to learning in the real world of policy, or making philanthropic investments, or commissioning public services. I decided to do X. I tried X and it didn’t work. From the learning, I concluded I should next try Y, et cetera.
So what might Aristotle say about all this? Quite a lot. To take a few selections from the Hall piece. Aristotle says that deliberation (the decision making process) is about:
- Means not end (you don’t deliberate about good health, you deliberate about how to achieve good health)
- Uncertainties (you don’t deliberate if you know the action will be 100% successful with no negative side effects)
- Things in our power and that can be realised through action
He goes on to elaborate. We deliberate:
- To act (we don’t deliberate about the laws of nature)
- Future not the past (we cannot undo the past but we can use the past to make a better future
- Knowing our actions have consequences (i.e. that we are accountable).
I also find Aristotle’s rules to be apposite. I have put them in a box in case I am stretching your patience. But if you are beginning to think there is something in this argument, I would encourage you to read Kahneman’s Fast and Slow Thinking which also has a lot to say that is useful.
Am I right in thinking that a lot of what people describe as learning is actually a description of what they believe to be true? That there is a lot of hiding, and a fear of making decisions explicit and being found out?
Being clearer about the decisions we make, finding out if those decisions helped us get to where we wanted to go, and using the learning to make better decisions in the future. Is this core to how we want to think about learning?
