Where we belong - social positioning of the organisation

This fourth prerequisite links to both higher purpose and the ability to influence the system.  It is somewhat tricky to define tightly, but encompasses a range of related issues:

• The data team’s understanding of its role within its organisation or and its relationship with outside organisations. 
• The team or organisation’s understanding of its position insider the broader system, what it can uniquely do, and the context which creates opportunities or barriers for it to further its mission. 
• A range of ethical issues which can be aggregated as an understanding of the risk of doing harm (even the most careful and ethically conscious of organisations may do so inadvertently).

A good way of understanding the first of these issues is to consider how a central agency may differ from a local one. A central agency has several advantages in using data in imaginative ways – in general it has easier access to more data, finds it easier to recruit talent, and has perhaps, more space to experiment to find what may be hidden in the data.  Yet the levers for effecting change are indirect and require partnership with others to have an effect.  Here the opportunities are for greater abstraction and support for policy level thinking.

In contrast, a local agency has the advantages of deeper contextual understanding and direct relationships with “frontline” service delivery, and potentially the ability to respond to change more rapidly.

In both instances there is a sub issue about how to avoid a type of “golden child” syndrome, where the development is seen as a small group of privileged analysts getting to do “cool stuff” with data outside of the mainstream of work, breeding resentment and resistance.  Again, this takes us back to the importance of automating the routine as much as possible in order to allow space for all analysts to experiment.

This space – however creates two further, perhaps paradoxical issues.  First, freedom to experiment requires robust oversight and project management in order to avoid becoming unproductive.  Second, in a context of experimentation, continued success is failure!  If all projects are a success then the likelihood is that not enough risks are being taken, and opportunities for insight are being missed. 

In terms of the ethical issues, the glib answer is that “If it sounds a bit like Philip K Dick, you’re probably doing it wrong”.  Unfortunately, it’s often not that simple.  There are obvious risks of disclosure in using micro-level data through careless presentation of results.  However, perhaps more pressing and harder to solve are analyses that entrench inequity.  A wide range of examples, especially from the US have been chronicled by Propublica[1].

Often, well-meaning attempts to avoid bias by removing variables which could obviously skew analysis, have unintended perverse consequences.  Some of the analysis of the problem and potential solutions identified by the Turing Institute[2] for example are both extremely interesting and potentially of great value.  However, in this context the issue for an organisation using data is to consider the potential risks and implications.  There are some approaches to avoid some of the worst risks of algorithmic unfairness, for example, ACC have made the decision never to reject a claim based on algorithm alone (anything that does not meet criteria for immediate approval goes to claims assessors)[3].  However, whether such an approach could be used by all agencies in all circumstances is not clear.

Another option is to limit how these techniques are used – for example, never undertaking analysis that can be used to manage the delivery of services to individuals (rather than planning them for populations).  Again, though, for a frontline service delivery is this a viable option?