AI Strategy 3 - Data

Data is raw facts with no context attached. A name, an address, a quantity, a reference, a date, a status, the identifier of a record in some application: each is a fact and none of them means anything on its own. Ordering those facts, cleaning them and putting them in a context is what produces information, and that is the next chapter’s subject. The distinction is important because businesses habitually describe the wrong thing as data. A report is not data, it is the product of processing data. A dashboard is not data. A summary of the month is not data. Each of them is an insight, and our systems audits need the facts the insight was assembled from, because those are the things that have a source, a format, a location and an owner.

Two things emerge from this distinction. The first is that a single spreadsheet is almost never a single item on the map. It will typically contain raw data, a processed view of that data, and analysis based on it, all on adjacent tabs. The three components have to be recorded separately because they assume different roles in the knowledge pipeline. The second is that data about data (metadata) is also important - when a record was created, by whom, from what, and when it was last changed. If that layer is missing, it is impossible to make best use of the data, with or without AI.

To recap, the data map records four things for each item:

1. its source - where it comes from;
2. its format - unstructured or structured;
3. its storage location - application;
4. its accessibility - who owns it, can write to it and who can read it.

The data map page carries a worked example of all of this, built from an invented business.

We note four different types of data source. The first is a person inside the unit, the second is a by-product of the system, e.g. the timestamps, the audit trails, the message logs and the delivery receipts, the third is data that arrives from outside the unit - by a client, a supplier, or some other party, and the fourth is a fact derived from other facts. Where an item’s origin cannot be established, that is recorded as it is.

Format is the shape that the data arrives in and is recorded in the plainest terms - a field in a structured record, a cell in a spreadsheet, a line in free text inside a document or the body of an email, a scan or a photograph, an audio or video recording, a message in a chat, a written note, or even something with no consistent format at all. The distinction being drawn is between a fact explicitly stored and one that is buried inside prose. A quantity in a field can be read, compared and checked by human or machine but the same quantity written into the third paragraph of an email can only be found by a person who already knows it is there, or by more sophisticated technological means that may not be reliable. In essence, there is a spectrum of data utility and reliability that is a function of its format. That is why it is so important to note, especially when it comes to machines that are inherently less intuitive than humans, even the most advanced AI.

With respect to storage, the map records the application, the account it belongs to, and the device or service holding it. There are two important considerations - is this the record or a copy of the record? And if it is a copy, does anything carry a change in one into the other? Two records can end up describing the same activity with nothing carrying a change made in one across to the other. It’s what can arise when a second record is created to answer a second question, independently of the first and no one ever states which of the two is the canonical record, even if it is tacitly understood. Naming one source of truth for each critical item is what lets two people report the same figure.

Another common data issue we have observed is reference tables or schemas that other parts of the system rely upon that live in a spreadsheet or some other ephemeral data store but with no enforced link to the records that use it. Consequently, an entry in another part of the system can be made against something that does not exist in the reference at all and nothing in either part objects. The defect only becomes apparent downstream, and by then it looks like a mistake by the person who made the entry rather than the absence of any connection between two stores.

That is a failure of the common business language and data dictionary (the ontology): the agreed set of things the unit deals in and how they relate to each other, what counts as a customer, what counts as an order, and which two records are the same customer. Where it is absent, the same thing can end up being recorded under two different labels, or two different things can share one, and nothing in either store declares which is which.

This matters more with a machine than it did when only people were reading the records. A person looking at two customer records with slightly different names can see the ambiguity and intuitively reconcile it. AI models can do the same but when non-determinism is already an issue, mitigating it with a strict ontology is a good step.

Access is recorded on three terms - who can read it, who can write to it, and who or what is the gatekeeper. The third term turns access from a state into a decision, which changes the way the data is treated.

Two categories sit outside the systems entirely, and they are the reason the data map is built from the process map rather than from the applications.

The first is data the business acts on that is not held anywhere. Our audits typically find these at the handovers, i.e. when data moves from one person or system to another. Often, there is legitimate reason for the data to change during this transition but often, even though the change is recorded, the reason is less often done so. Again, when we come to making a choice between what the machine can do and what the human must retain control of, this fact alone can be the deciding factor.

The second category is data that the business holds and never uses. Every audit finds some, and the volume is usually a surprise to the people who own it. It is a peculiarity because, on the face of it, the data process looks robust - a source, a format, a location and an access list, all correctly documented. But no record of what the data is for and therefore it just accumulates and no-one ever notices. In all honesty, this probably describes the state of many CRMs! However, although unused data is not automatically a fault, it is an opportunity to derive value from AI that could be overlooked if the defect was not noted in the systems audit.

Finally, we can never assert that the data map is complete, nor can the coverage be expressed as a proportion, because the denominator is precisely the unknown. Instead, the map is compared to the objectives. The discovery is not the final inventory but the gap between the facts the unit has recorded and the facts required to meet the objectives (see chapter one).

I learned very early in my career, in capital markets, that getting the data wrong can only result in everything else being wrong. Information, the topic of the next chapter, is assembled from this data and no processing reliably recovers data that was never recorded, and no information is useful, if it is synthesised from incomplete data.