ArticlesData Classification vs Behaviour Observation: Thoughts on Converting a Diagnosis into an Actual Strategy

I often find myself giving an overview of how an insider threat management function addresses data protection, to non-technical stakeholders and to new analysts being onboarded. At some point during these speeches, I explain what our tools are for. Not which ones, that part is on the licence invoice. Each one’s purpose, and in what order they are employed. I draw three boxes. Then I hear myself say a sentence I have heard many times. Only this time I am taking the trouble to elaborate further.

The three boxes are the ones every insider risk management program is built from, whatever the vendor. One classifies data at rest: it reads a document and decides what it is, and marks it so that everything downstream can read the mark instead of the document. One applies controls to data in motion: it stands at the exits, mail, browser, removable media, cloud, and decides what may leave. And one watches the interaction between the two and the users, which is to say the people in the organization: who accessed or moved what, when, how much, how unlike themselves. Classification, prevention, observation. At rest, in motion, in use.

It looks like an obvious order to build them in. Classify first, so that the controls have something to act on. Control second, so that the watching has a baseline of what is already prevented. Watch last, for what the first two let through. It is the order the architecture diagrams draw, and it is the order almost nobody follows.

At Microsoft’s Security R&D Days in Paris last May, my fellow colleague Richard Fazio stated this plainly, and the room nodded, including me. He said that organizations very often deploy in the inverse order. They stand up the tool that watches interactions first, because it shows something on day one. Then, from what it shows, they work backwards: this is what people actually move, so this is what we should have classified, so this is what we should have been preventing. Observation first, classification and controls as its consequence.

Is that a defect in the approach, or a better approach than the one on the diagram?

And here is the sentence. I said it myself, I have said it at conferences, and I have heard it said back to me by people whose judgment I respect: there is no better method; both are valid; which one fits depends on the characteristics, the priorities, and the posture of the organization adopting it.

The sentence#

It is true. That is the trouble with it. It is true the way a topographical map is true: it describes a state accurately and tells you nothing about what to do in it.

Both approaches are in use. Both have produced programs that work and programs that do not. Organizations do differ in what they hold, what they fear, and what they will tolerate. Every clause of the sentence survives inspection. And yet a practitioner who says it has answered the question without having thought about the problem, because the sentence is anchored to the present state of things and has nothing to say about where a program is going. It has no view of maturity, present or future. It is a description of where organizations are, offered in place of a judgment about where they should be next.

I think we say it to avoid giving offence. The room contains people who did it one way and people who did it the other, and the sentence lets everyone leave with their choice intact. That is a kindness. It is also a small dishonesty, and cybersecurity has more of these than it admits: sentences that sound like a considered position and function as a truce. A field that wants to be taken seriously by the people who fund it should be able to tell the difference.

The honest version is not to pick a winner. It is to say what the choice actually depends on, in terms specific enough to be wrong.

The variable#

Here is what it depends on. Not characteristics, not priorities, not posture. What the organization already knows about itself.

Classifying first assumes you know what matters and where it lives. You can name the categories of data that would damage you if they left, you can say where they are kept, and you can extrapolate patterns from them well enough that a machine can find them. If that is true, classification is a transcription exercise: writing down what the organization knows, in a form the other two boxes can read.

Watching first assumes you do not know that, or do not trust what you think you know. You cannot name what matters with any confidence, or you can name it but have no idea where it actually is, or you suspect that what the business says is sensitive and what actually moves are two different sets. So you watch, and let the movement tell you.

Those are not two philosophies. They are two states of self-knowledge, and the inverse order described in Paris is what an organization does when it is honest about being in the second state. Read that way, the sentence rewrites itself into something longer and less quotable: each approach has a precise function; used together they form a sequence; and which of the two is deployed first depends on how much the organization knows about itself. It is a mouthful. It is also more precise, and when the thing being engineered is a security system, the mouthful is the required version.

So the order of deployment can be inverted, and usually is, and there is nothing wrong with that. What cannot be inverted is something else, and to see what, the two approaches have to be taken apart properly. That is the step back. The conclusion is already in view: the question was never which goes first.

Classify first#

The case for classifying first is the case for knowing what you are doing before you do it.

A label is a decision, made once, about what a piece of data is. Everything else in the program can then act on the decision instead of re-making it: the control at the exit does not need to understand the document, only the mark on it; the analyst reading an alert does not need to open the file, only to see that it was labelled restricted and left anyway. Purpose is declared before collection. The controls are preventive rather than forensic, which is to say they stop a thing rather than explain it afterwards. And the whole program can state, in one sentence, what it protects, which is a sentence a board, a regulator, or a works council can understand and hold it to.

The case against is the experience of anyone who has run a classification project. It is slow, and slowness here is not a delay but a decay: by the time the taxonomy is agreed and the labels applied, there is a chance that the business has reorganized, the repositories have moved, and a share of the labels describe a company that no longer exists. It is expensive in the currency that is scarcest, the attention of the people who own the data. And it has a structural blind spot that no amount of diligence closes. Labels record what the organization believes is sensitive. They cannot record what actually moves, because nobody is watching that yet. A classification scheme is a theory of the organization’s crown jewels, written by the people least likely to know where the jewels have been left lying around.

Notice what that last weakness needs in order to be corrected. It needs evidence of movement. It needs the third box.

Watch first#

The case for watching first is that it shows something on day one, and that what it shows is true.

Observation does not depend on anyone’s theory of what matters. It records what people do with data: what they open, copy, send, print, upload, and how that compares with what they did last month and with what their peers do. Within weeks it produces a picture of the organization’s actual flows, which is a different and more useful thing than its declared ones. It finds the sensitive data nobody labelled, because it finds the people treating it as sensitive, or the people treating it as nothing at all. It gives a classification project the one input a classification project cannot generate for itself: where to look, and what to look for, ranked by how much of it is moving.

The case against is what happens on day thirty, or sixty, or a hundred and eighty. Observation without a decision rule produces signals, and a signal is not an alert until someone has said what it means. Watching first means the meaning arrives after the watching, in the form of an analyst staring at a large export and asking whether it matters, with no label to answer the question. The queue fills with events that cannot be placed. The tuning debt compounds, because every rule written to quiet the queue was written against a picture nobody had agreed on. The analysts learn the organization by hand, one false positive at a time, which is education, but it is also the most expensive way to acquire a taxonomy that could have been decided in advance.

And there is the legal exposure, which in Europe is not a footnote. I am putting it aside for a moment, deliberately, and will come back to it, because it changes the shape of the argument rather than adding a clause to it.

Notice what this weakness needs in order to be corrected. It needs a decision about what matters, made before the next alert rather than after it. It needs the first box.

Neither goes first#

Put the two catalogues side by side and something is visible that neither approach shows on its own. Each one’s weakness is exactly what the other produces.

Classification cannot know what moves; observation is made of nothing else. Observation cannot say what matters; classification is that saying, and nothing more. So the practitioner’s sequence, observe to discover, classify what you found, control what you classified, then watch again with a purpose, is right as far as it goes. It just does not go far enough, because it describes the loop once and then stops, and the loop does not stop.

Look at what the sequence quietly admits. It admits that the first classification was provisional: written from what observation found, and therefore only as good as the window it was found in. Things move. Data moves, people move, the business moves under both. So the classification decays, and has to be fed by another round of observation. And the observation decays too, in its own way: the rules and baselines that were tuned against last year’s flows describe last year’s organization, and have to be re-aimed by whatever the classification now says matters. Each corrects the other. Neither stays correct alone.

That is the thesis, and it is a smaller claim than it sounds. To be effective, and even to be realistic, the two approaches are not applied in an order. They are applied at once, and one feeds the other. Not as a compromise between two schools, but as a mechanism, with two different things flowing in two different directions. Observation supplies evidence: what moves, who touches what, how much, how often. Classification supplies decisions: what counts, what is permitted, what a movement of that kind means. Evidence flows up into decisions; decisions flow down into what is watched for. Anyone who has built a control system will recognize the shape. It is a feedback loop, simple as that.

Which is why “at once” has to be said carefully, or it collapses back into the sentence we started with. Every organization I know already has all three boxes running at the same time. Simultaneity is not the achievement. The achievement is the coupling: that the output of one is, by design and by somebody’s job description, the input of the other. In practice the three tools tend to be owned in three different places, and the loop is more often broken at the handover than in the technology. I will leave that observation where it is. It is a different argument, for another time.

The entry point#

Now the thing I put aside.

Everything above treats the loop as symmetric, as if it could be entered from either side. Operationally it can. In Europe it cannot, and the reason is the one thing that separates a European program from the diagrams it is usually built from.

Watching how people interact with data is processing personal data about workers. Before it begins, there has to be a ground for it, a purpose it serves, an assessment of what it will observe of the workforce subject to it, and in a number of member states an agreement with the people who represent them. Every one of those instruments asks the same question, in its own language: what are you watching for, and why. Purpose limitation means the answer has to exist before the collection, not be inferred from it.

Which puts the inverse order in an awkward light. “We deployed the observation tool to find out what matters” is, stated plainly, collection without a declared purpose, offered as a method. It is exactly what the regime is built to refuse. The American discourse does not see this cost, because on that side of the water the discovery phase is free. On this side it is not, and a practitioner who quotes the Paris observation without the caveat is quoting half of it.

This does not forbid discovery. It bounds it. A program that needs to observe before it can classify can still do so, with a purpose declared in advance, a scope narrow enough to match it, a duration, and the classification named as the intended outcome rather than discovered as a happy accident. The loop still runs. It has a mandatory entrance, and the entrance is the decision side, however thin the first decision is. You may enter with a small theory of what matters. You may not enter with none.

So the two approaches are not alternatives, and the order is not the question. One feeds the other, continuously, and the program is the loop rather than either half. In Europe, the loop has a door, and it opens from one side. Everywhere, it has to keep turning, or the labels describe a company that is gone and the alerts describe one that never existed. The sentence everyone uses was right about the state of things. It just mistook a state for a strategy, and there was a mechanism underneath it the whole time.

In the recordGV007 Lawful basis register · GV008 Impact assessment before deployment · DP002 Information classification scheme · DP007 Data loss prevention deployment · DP008 Data loss prevention policy and tuning · MD012 User and entity behavior analytics