AI Strategy 6 - Judgement

Judgement is a decision made to commit the business to a particular course of action. It relies on the knowledge described in the previous chapter. That might have been the explanation for rising sales. The commitment would be to continue the campaign, that was concluded to have been the contributory factor. That is judgement. Historically, this has always been the domain of the human. With AI, increasingly, it is handed over, to a certain extent, to the machine. That decision itself is also a judgement and so it should not be done in the absence of evidence to support the decision.

A commitment is made at a particular moment, by a particular person or role, and it disposes of something the business could otherwise have spent elsewhere - money, capacity, attention, or some other resource. Decisions cannot usually be reversed without cost. Most of what preceded judgement can be quietly corrected but a commitment produces a consequence that must be more carefully considered.

Judgement is distinct from knowledge insofar as knowledge is sound or unsound relative to the evidence for it, which was the test in chapter five. A decision is good or bad with respect to what was known and available at the time it was taken. Its consequence is favourable or unfavourable relative to the objective it was serving.

The fourth of our maps records the decisions the business unit takes. As usual, we record five components of each:

1. decision - what was committed to, and which explanation it relies on;
2. authority - who takes it, at what level, and whether that authority belongs to a role or to a person;
3. expectation - what was expected to follow from it, by when, and what would have produced a different decision;
4. storage - where it is recorded, and in what form; and
5. review - whether the consequence is ever set against the expectation, and by whom.

Decisions are frequently taken on knowledge that was never written down. Judgement that runs in no system is the first of the five habitually unobserved parts we discover in an audit, listed in chapter two, and the example given there is a basic one - a threshold, set by the person applying it, derived from nothing the business has measured, and applied consistently for years. The decision produced by such a threshold is usually a good one. However, if it is not available to anybody else, it is difficult to determine whether it should endure as it is.

The expectation is the component most often not recorded appropriately. A decision is recorded, in meeting minutes, for example, as what was resolved. What was expected to follow from it, by when, and what would have counted as evidence to support it, is often recorded with it, but not in such a way that it can be readily recovered and applied systematically or programmatically.

Review is the component that closes the loop. The consequence of a decision arrives weeks or quarters afterwards, lands in a different report, is read by different people, and is not always referred back to the decision that produced it.

In one of our audits, a decision to move the business unit off its CRM had been taken and later reversed. Neither the reason for the move nor the reason for the reversal was recorded anywhere, and both decisions had been taken by people still in the business. What the unit had at the end of it was the position it started in, no account of why it had left, and no account of why it had come back, even though both decisions were still very much apparent in the minds of the people who made them.

A wrong decision is made if its consequence was not the one intended. Conversely, a badly-made decision arises if the process that produced it was defective. In other words, the knowledge that the business unit already held was not considered, the expectation was never stated, or it was taken by somebody without the authority to do so. On the other hand, a decision can be soundly made on good evidence by the right person and still turn out to be wrong, because the knowledge it relied upon described a relationship that ceased to hold. This is precisely the condition chapter one described for objectives.

What judgement requires from the stages before it is data whose provenance, format, location and access are known, information whose lineage back to that data can be traced, and an explanation with its evidence recorded with the date it was accepted. Judgement inherits whatever the four rungs of the ladder produced, and our audits establish that.

This whole chapter is important in the context of AI because a machine will commit as fluently as it explains. Give it the information and an explanation is produced. Give it the explanation and a recommended course of action is produced, phrased with the same assurance whether the evidence beneath it was complete or not. The machine has no view on what it was not given, which was the point made in chapter four about synthesis and holds even more so here, because a commitment produces a consequence that is more costly. Where the decision, its authority, its expectation and its basis were recorded at the time, the consequence can be set against them and the business can assess it well in retrospect.

The three As - Assignment, Augmentation and Agency, the subject of the next chapter, take each decision on this map to determine the manner in which AI can be collaborated with in respect to the task at hand. That choice is made before any of the competencies in the later chapters are exercised, and it is made against this record, since judgement whose authority and basis were never formally established cannot be assigned to anybody - human or machine.