Discussion Prompt: Is productive engagement on intelligence law, policy and oversight possible between the secret and civilian world and what can be gained from it? Reflections on best practice, lessons learned, and plans for the future.

See all contributions to this question.

As a digital evidence expert witness, government advisory panel member, and specialist adviser for legislation such as the Investigatory Powers Act of 2016, my career has allowed me to actively facilitate the engagement between the public and the secret world for nearly thirty years. Time to take stock, share what I’ve learned, and, hopefully, offer some guidance to other intermediaries.

Those seeking to comment on the powers and activities of the intelligence agencies face a well-known dilemma. Do so solely from the outside and you run the substantial risk of failing to understand the range of risks and tasks an agency may face as well as its actual internal procedures. Get too close to the agencies, however, and you may be ‘captured’ in the sense of relishing access to information that the public at large do not have. Worse still, in this scenario journalists and academics run the risk of being labelled uncritical apologists and propagandists, and at its very worst, can become vehicles for messages and unverified statements that some in the agencies wish to make public.

Most people with an interest in intelligence policy will be able to identify pundits, bloggers, and pressure groups in their countries which fall into either of these extreme categories. For an individual seeking to be both well-informed and uncaptured, the path is far from clear.

So can there be productive engagement between the civilian and the secret world? I think the answer is yes, but there are limits.

By bringing together different stakeholders, formal oversight bodies can play crucial intermediary roles, as can other institutions and think tanks who run sessions using the Chatham House Rule. The rule emerged from the eponymous think tank but is now widely used in many contexts. In practical terms it means that civil servants, intelligence officers, and politicians can interact with others and so help develop policy without being publicly identified and with the opportunity to change their views in the light of discussions.

But rather than looking at the position of organisations, this piece reflects on how I have had to weigh a number of difficult ethical and practical decisions in the hope of producing a set of workable rules for others. I’m not sure whether I have always succeeded, but I am able to describe how I made a series of personal decisions in the pursuit of that goal. In the end, that is what most people have to do — and across a variety of situations.

You will see that opportunities emerged rather than being sought. You also need to know a little about my background: my undergraduate degree was in law; I had a career in book and electronic publishing; I appear to have the “geek” gene in that I have always been interested in technology (and how it was being deployed in society); I teach cyber security at university level but with an emphasis on digital forensics and law; most of my income is from legal expert witness work and broader based cyber consultancy.  The rest you can pick up from the following narrative.


A growing fascination

My interest in the hidden and intelligence world was probably initiated by 1960s publication Spies for Peace. It revealed that the UK government’s plans for a post-nuclear war situation had been to set up a series of bunkers containing Regional Seats of Government. These were to be occupied by civil servants and senior politicians who presumably would eventually emerge to provide order to whatever contaminated individuals and debris remained. Later, notably in Peter Laurie’s book Beneath the City Streets, investigators provided much more detail on this apparatus. From then on this became an interest but I hope not an obsession. Periodically, friends and I would visit parts of the countryside which official maps said were devoted to agriculture but where one could see large steel fences containing notices with warnings about the Official Secrets Acts — and within them massive antennae, radomes, and buildings. We also developed a skill in recognising covert buildings as used by the intelligence services as many had a series of similar ‘looks’. It must be remembered that at this stage of its history, the United Kingdom did not officially acknowledge that it had any active intelligence services; of course spy fiction was being published as were accounts of the successes of intelligence activity during World War II.

Friends such as Duncan Campbell probed this world much more aggressively and boldly — Duncan was charged under the Official Secrets Acts and also subject to police raids.

The motivation was a mixture of technical interest and political concern about unsupervised powers and unannounced use of taxpayers’ money. We asked ourselves: What were the exact technical capabilities of the ‘spooks’?; how many telephone calls and how much telecommunications traffic could they siphon up?; what data search facilities were available to them?;  did they have significantly better speech recognition capabilities than were available on the open market?; and so on. The 1987 publication of Peter Wright’s Spy Catcher, the memoirs of a former MI5 officer, prompted greater curiosity both about agency accountability and the range of technical capabilities. Wright, though from rather different motives, was a precursor to Edward Snowden.


First steps in the world of intelligence

Things changed for me in the mid-1990s when I returned to part-time academic life at the London School of Economics. We were approached by an individual who said he was researching computer security (as we called it then) for the Ministry of Defence. When I met him he came clean and said that he was a senior officer in the Security Service, MI5. Although I was unaware of it at the time, decisions were being made within the ring of secrecy to become more open about its work and to seek dialogue with the outside world. In 1992, then-Prime Minister John Major publicly acknowledged the existence of the agencies, their respective heads had been named by 1993, and an Intelligence Services Act to define their powers was passed in 1994.

At the same time the understanding took hold that national security questions, cyber security in particular, could not be handled solely from within the agencies themselves. Not the least of the difficulties was that much of the United Kingdom’s critical national infrastructure is in the hands not of state-owned entities but private companies, some of them foreign-owned; in addition there was a growing realisation that threats could come from non-state actors as well. Perhaps it was also recognised that although both MI5 and GCHQ recruited people with technical expertise this could never be enough.

So began my personal dialogue with the intelligence world. A little later I was instructed as an expert witness in the defence of a 16-year-old hacker who had made significant breaches into US and NATO military resources. His activities led to hearings in the US Congress. I wrote up my experience in handling the technical evidence in this case for an international workshop dealing with intrusion detection systems (burglar alarms for computers and networks). Afterwards I was approached by yet another MI5 officer and later joined the program committee for the following year’s workshop where a fellow committee member was a declared employee of the US National Security Agency. These interactions developed on an informal basis and from time to time resulted in invitations to closed unadvertised conferences.


Seeing the agencies with my own eyes

Increasingly too, GCHQ and MI5 employees concerned with cyber security found it useful to attend specialist ‘open’ seminars and conferences even if they did not declare the identities of their home organisations.

The discussions at these events included technical matters but also ethical issues. Fairly rapidly I began to appreciate that most cinema and television portrayals of the agencies were misleading. Most UK intelligence staff were not upper-class old Etonians. On the whole, they didn’t all wear dark glasses, travel around in darkened SUVs, and arrange for the extrajudicial assassination of ‘enemies’. (This does not mean that some kinds of covert action of which I do not approve would not occur from time to time.)

What I learned was that many employees of agencies have a strong sense of public duty and are aware of the dangers of invading personal privacy. After all, they could have almost unlimited use of covert surveillance technologies and access to medical and financial databases — and the excuse on being questioned, that everything was a matter of national security. It also became clear to me that within the agencies, as in the outside world, there was a range of views. At one extreme, arrogant self-belief that the public would be grateful if only they knew the problems faced and sacrifices made by agency employees, and at another extreme a conviction that many more safeguards had to be implemented in the agencies to ensure higher levels of public accountability.

We also discussed how far they thought that ministers understood the implications of what they were authorising.  Many employees felt that more had to be done to find out what was ‘acceptable’ to the public. I recognised there was a point beyond which the conversations would not go. Once you make a career with an agency you are committed to concealing your true employment and not discussing it with the outside world. Friendships are made normally within the intelligence world and with people who have also committed themselves to what is a very particular type of life.


Taking on different roles

Another influence was my ever-growing practice as an expert witness. I cover digital evidence in all its forms and situations where older law needs to be interpreted to cover newer technologies and the associated social and commercial structures. An expert witness in the English system is not an advocate or campaigner, still less a juror or judge; they are there to assist the court with specialist technical knowledge and opinion based on experience. I can be instructed either by the prosecution or the defence; I also do civil and international work. Complicated matters must be explained clearly and accurately. But the process exposes you to the practicalities and imperfections of investigations. Of course, the courts want evidence whereas the intelligence agencies produce, well, intelligence. Evidence is disclosed and tested in the court while intelligence is about producing an assessment of circumstances and is neither disclosed nor subjected to external testing. Along the way I became fascinated by the processes by which the raw material of intelligence becomes evaluated into a report. 

During the late 1990s I was asked to act as a specialist adviser to a Parliamentary select committee looking at the U.K.’s electronic commerce laws and in particular whether it was desirable and possible to control the availability of cryptographic products — crypto being essential to ensuring the reliability and confidentiality of electronic commerce. The e-commerce legislation was  being promoted by the Trade ministry but the “crypto” elements were thought to be more appropriate to a Home Affairs ministry and  were moved into another bill which eventually became the Regulation of Investigatory Powers Act 2000.

I had been a long-time supporter, largely passive, of the National Council for Civil Liberties, now known as Liberty, and I became engaged and friendly with a number of other NGOs. Once my role on the select committee ceased, I joined the Advisory Council of the Foundation for Information Policy Research and co-wrote a number of their legal-technical analyses. 

Between 2003 and 2009, following one of those ‘taps on the shoulder’, I was a member of the Scientific Advisory Panel on Emergency Response (SAPER) run by the Government’s Chief Scientific Advisor within the Cabinet Office. The name tells you what its role was — its successor, under a different name, still exists. It covered natural disasters such as pandemics as well as human-induced attacks.   We were regularly addressed by senior serving security and intelligence officers and took part in table-top exercises. I was seeing the process of intelligence evaluation as new threats such as home-grown terrorism were emerging. An important feature of the work was considering potential scenarios for hostile attacks on the UK. The results would undoubtedly fascinate many journalists, academics, and others, but publication would also assist potential attackers. These were situations where secrecy was plainly justified.


The Investigatory Powers Act of 2016

In 2015 the UK government recognised that it needed to consolidate and update the legislation covering the surveillance powers of the intelligence and law enforcement agencies. There was already a significant body of law but it was spread over several statutes; some of the powers were little-known to the general public as they were justified by obscure references in Acts and interpretations of laws about telecommunications. Additional impetus to the need for legal revision was given by the Snowden revelations and the outcomes of various test cases in the UK and EU courts, stimulated by the activities of privacy advocacy groups. In addition, the many forms of communication made possible by the internet could not always be readily dealt with by laws that were designed in the days of traditional analogue telephony. The government decided that such was the size and complexity of the proposed new law that Parliament needed to be given an opportunity for pre-legislative scrutiny. A draft bill and some explanatory notes were published and a select committee of Parliament comprising both members of the House of Lords and House of Commons was set up.

A major practical problem was to translate general objectives — to balance the autonomy and privacy of the citizen against the need of the state to protect itself against its enemies and criminals — into specific unambiguous legislation that would not face future unwanted consequences.  To achieve this required a great deal of legal and technical detail. Among other issues was that the agencies and police naturally wanted laws which gave them access to the material they desired; but it was difficult to map this to the technical terminology which internet engineers use to define the types of online traffic. Engineers don’t immediately know what an “internet connection record” or “secondary data” are, but they appear in the legislation, which requires them to produce evidence to law enforcement. For the UK there was the added complication that ever since 1985 the ‘content’ of a transmission has been inadmissible in legal proceedings though it can be used as intelligence; only ‘communications data’ can be produced in court.

I was appointed one of two specialist advisers with the role of explaining to legislators what was involved; the role was to give advice not to lobby personal views. I read all the written submissions, from privacy NGOs, police, intelligence agencies, lawyers, journalists, trade unionists and the like, and advised on who should be called in to give public oral testimony. In private, I tried to demonstrate what Internet data traffic looks like when viewed through analytic software.  Select committees hear evidence in public — video recordings are available on the Parliamentary website — but they also receive private briefings; in this instance at the highest level. In addition, it became sensible for specialist advisers to open back-channels to the police, the agencies, and the sponsoring government Ministry, the Home Office, in order to discover what they hoped for and why. There were opportunities for informal debates which I believe were very useful. At the end a report with detailed recommendations had to be written — and this was done in association with my fellow specialist advisor and the permanent Parliamentary staff who understood the problems of crafting detailed legislation. In this instance I think all of us would have preferred more time to produce a better report. The views in the report are of course ultimately not mine but those of the committee members of the Houses of Lords and Commons.


Working with oversight & other specialists

This interaction with the intelligence world has continued, but at the moment on a more informal and occasional basis. One of the features of the Act was the setting up of a new oversight body, the Investigatory Powers Commissioner’s  Office,  and I gave the commissioners an early briefing on “equipment interference” (authorised hacking).

My expert witness activity brings me into situations where new surveillance methods, as used by the police and agencies, need to be tested for evidential reliability and legality. Some of this work is before some of the UK’s semi-secret courts and tribunals such as the Investigatory Powers Tribunal and the Special Immigration Appeals Commission. Others involve the surveillance and communications methods of suspected organised crime groups.

Also, digital forensics specialists, including those from the “secret squirrel” community,  frequently meet both informally and in conferences (and online) to discuss new methods for tackling emerging technologies, software and hardware, and associated applications. The rule here is to pose the technical questions whilst rigorously avoiding talking about the specifics of any on-going case. This is not a bad model for managing other sorts of conversation between the secret world and the public.


Lessons

In retracing my professional steps, I hope to have shown that I never sought intermediary roles with the agencies but at the same time welcomed them if the opportunity was presented to me. Although there was never a set of clearly articulated rules at any stage, I would like to think that I thought very carefully about my role. I am glad to report that I was always given the space to do so. Intelligence officers who become agent handlers learn how far to press an individual who could become a ‘source’, but no one ever forced me into situations in which I felt uncomfortable.

Drawing from my experience, I have attempted to formulate a few nuggets of advice which might prove useful to others working between the secret and the civilian world.

1.    One must have a fundamental acceptance that intelligence agencies and their employees need to exist and that significant aspects of their work — both operationally and the infrastructure they use — need to be secret to be effective. But at the same time there must be a framework of law governing their broad activities, authorisation for individual actions that threaten the privacy of persons and/or carry risks with political consequences, and adequate oversight. This is what the final Investigatory Powers Act 2016 tried to achieve though it is worth noting that some further oversight mechanisms have been added subsequently, particularly in relation to authorisations for access to data.

2.    Ultimately, a democratic nation’s decision about the balance between powers for the security agencies and respect for the autonomy and privacy of individuals can only be reached by parliaments. But above laws and codes of practice, informal discussions have an important role in informing parliamentary debate.

3.    For those adopting a career in the intelligence community there is the risk that they only seldom have the opportunity to question their actions and motives with potential critics and so get a view of how the public might view them. At the same time, the public do not tend to have a consistent view of the intelligence agencies. Periods of admiration can come to a sudden halt followed by a 180 degree turn in public opinion after a news story about intrusion of privacy or excessive use of powers has broken.

4.    Intelligence and security agencies need to ensure access to a wide range of views so that they can best serve the public. The people with whom they engage for this purpose are not ‘agents’ or ‘sources’ but those who it is hoped will help formulate policy and get perspective.

5.    Inevitably, briefed individuals in confidential circumstances (which might be quite informal) will on occasion get firm confirmation of what was previously only suspected. They will probably learn the identities of agency operatives and of covert locations and facilities. There is little point in publishing or disclosing secret information just for the sake of it. My own practice has been not to make this type of information public unless it had appeared unambiguously elsewhere. There may be circumstances when this rule has to be broken, but only in combination with a belief in overwhelming public interest and a conviction that many others will feel leaking the information was justified.

6.    There is no clear career path for individuals who get drawn into the role of semi-intermediary. Those who do so should be candid with themselves about their motivations becoming involved. It is all too easy to become ‘captured’ and to develop an unwarranted sense of self-importance.

7.    Journalists are already familiar with the “off the record/on background/not for direct attribution” basis for reporting. It requires the trust of the source but also of the journalist’s readers and viewers. This trust implies that the source will not be betrayed, that the briefing will be honest as far as it goes, and that the journalist’s report will reflect what they believe to be the public interest. People like me hope to operate on a somewhat similar basis. Similar to journalists, intermediaries like myself receive input from academia & civil society and regard it as our duty to honour and protect these ‘sources’. At the same time, we are beholden to executive interests not to divulge classified information we were provided with for our role.

8.  Mutual trust is key in balancing these dynamics while at the same time always pursuing what we believe to be the public interest. Journalists, too, when they receive leaks of confidential information, need a well-developed sense of “public interest” as opposed to “great journalistic coup” — Edward Snowden hoped that this would be the case when he interacted with selected journalists.

9. Alas, I have no easy formula for defining the scope of “public interest” save to say that one important test is to ask, as you apply it to your own circumstances, whether you would be prepared, if challenged, to justify your decisions in public.