Secrecy Wars: National Security, Privacy, and the Public's Right to Know

Secrecy Wars: National Security, Privacy, and the Public's Right to Know

by Philip H. Melanson, Anthony Summers


View All Available Formats & Editions
Members save with free shipping everyday! 
See details


The public and the media are fascinated by U.S. government secrets, real and imagined, yet very few people know how the process of obtaining formerly secret documents works. Secrecy Wars is a look inside the American secrecy system as it is accessed through the Freedom of Information Act (FOIA) and the Privacy Act. With its perspective that of a political legal drama, this important new book will not only entertain and inform but also influence the legal, journalism, and political communities.

Product Details

ISBN-13: 9781574883244
Publisher: Potomac Books
Publication date: 10/31/2001
Edition description: 1ST
Pages: 274
Product dimensions: 6.00(w) x 9.00(h) x (d)

About the Author

Chancellor Professor of political science at the University of Massachusetts (Dartmouth), Philip H. Melanson is an internationally recognized expert on political violence and governmental secrecy. Most of his thirteen previous books relate to political assassination and violence, governmental secrecy, and law enforcement and intelligence agencies. His media appearances have included NPR, the BBC, CBS and CNN news programs; Prime Time Live; Hard Copy; and Inside Edition. Dr. Melanson lives in Marion, Massachusetts.

Read an Excerpt

Chapter One

Your Right to Know

The assumption behind the [Freedom of Information] Act is that taxpayers support the development of information and thus have a right to it. More importantly, the act is built on the belief that control over bureaucracy and government, essential to democracy, is impossible without information about what government is doing.

—Joseph Cayer and Louis F. Weschler, Public Administration, 1988

Secret keeping in the United States has been present since the nation's inception. The Constitutional Convention was held behind closed doors. President George Washington refused to provide Congress with the documents used in the negotiation of Secretary of State John Jay's Treaty with Great Britain. More than a century and a half later, President Franklin D. Roosevelt issued the executive orders creating the modern secrecy system. He also enhanced the political surveillance function of the FBI. After Hitler invaded Poland, Roosevelt instructed Director J. Edgar Hoover to publicly announce that the Bureau would surveil communists, Nazis, and other extremists. This measure was designed to calm the citizenry and frighten "extremists."

    Since World War II, however, the size and complexity of the secrecy system have grown exponentially. As far back as 1964, Secretary of Defense Robert S. McNamara revealed to the Senate Foreign Relations Committee that the top-secretclearance for access to information was already obsolete, as indicated by his testimony:

Secretary McNamara: Clearance is above Top Secret for the particular information involved in this situation.

Senator Albert Gore, Sr.: ... would you please clear up the exact identity of this clearance status that is something superior to Top Secret ...? I would like to be informed. I never heard of this kind. I thought Top Secret was Top Secret.

Secretary McNamara: ... Mr. Chairman, may I try to answer it.... There are a host of different clearances. I would guess I have perhaps twenty-five. There are certain clearances to which only a handful of people in the government is exposed. There are others with broader coverage, and overlapping coverage, and it is not really a question of degree of clearance. It is a question of need to know, and need to know clearances apply to certain forms of data.... There is another clearance, Q clearance, which relates to certain categories of information.

There is another clearance which is the special intelligence clearance we are talking about, that relates to intercept information, and it is this latter clearance in particular that is at issue here, and the staff members of this committee have not been cleared for that kind of information.... I do not want to get into a further discussion until the room is cleared of those not authorized to handle it.

    The "public's right to know" has always existed as an ideal within our democratic system. Defining this right has been a constant and uneven evolution. In 1966 a revolutionary event occurred: the passage of the Freedom of Information Act (FOIA). In the nearly two centuries that preceded it, there was no effective legal right for citizens to obtain information from their government. People could sue the government to compel release of information about them, but only within the narrow legal venue of proving that government was acting maliciously against them. No legally defined right existed to obtain information about government itself: how it operated, what it had done, and how it had made a decision. Whether it was a citizen, a news organization, or a corporation, information could be requested, but government could—and usually did—refuse. No legal recourse existed for such denials.

    There were laws granting limited access to certain records. Some information did flow to the public. FOIA, however, created a process by which citizens were entitled to information unless government could prove otherwise. The public's right to know is shaped by many factors as described in the introduction (the political climate, federal policies, court decision, agency behavior, executive orders, and so on). As powerful as these influences are, FOIA is the centerpiece of public disclosure. It is the focus of the political struggle over conflicting priorities and ideologies.

    While requests for documents are made singularly by individuals or organizations, the process has a cumulative, societal impact. A journalist obtains data on overspending by the Pentagon. Through the resulting newspaper article, the president, Congress, and the public are informed. A historian writes a book based on military and intelligence files, providing valuable insights on Vietnam policies to be read by the public and policymakers alike. A vetting of past CIA covert activities informs the public and Congress concerning what the nation's ethical and policy positions were, and ought to be. In short, FOIA made government more open and subject to scrutiny than at any time in our history. Some releases of information have little or no impact on the political system and its issues and policies; others have a momentous impact. But the cumulative effect is collective and system-wide.

    As with all revolutions, an intense struggle occurred at its inception, the outcomes uncertain both before and after passage of the law. From 1966 to the present, the political conflicts have been a constant element: expand FOIA, kill it, keep the status quo. Not only was the basic concept of legally defined citizen access revolutionary, but so also was the ultimate design of the statute. On the one hand, it was not intended to be a carte blanche entrée to the secrecy system: government could still withhold documents and was given nine legal justifications for so doing. On the other hand, FOIA was not a narrowly defined, esoteric access that few could use or that applied to only a small segment of the secrecy system. Its goals were lofty. As President Lyndon Johnson stated when signing the bill:

This legislation springs from one of our most essential principles: A democracy works best when the people have all the information that the security of the nation permits. No one should be able to put up curtains of secrecy around decisions that can be revealed without injury to the public interest.

    The government "records" to which the law applied were broadly defined. Individuals asking for information were not required to explain their reason(s) or intended use. Requesters did not have to be U.S. citizens. The law provided that "any person" (or organization) could make a request with the same legal rights as anyone else. This is still the case despite strong criticism and opposition over the decades. A countering argument has been that foreign nationals could use our law to obtain information from our government. These individuals included communists, terrorists, and other enemies or political opponents of the United States. And the rub was that we subsidized this nefarious access to our secrecy system, spending tax dollars to find and deliver the data. In other words, patriotism, jingoism, and anticommunism were galvanized by the law's provision of access to noncitizens.

    As part of the political compromise that is typical of landmark legislation, there were exceptions to the law. Those agencies that opposed its passage (and virtually all did) would continue to try to avoid it by having themselves exempted from its purview. Most agencies failed. The CIA would partially succeed in 1984 and the FBI would be partially successful in 1995. Records included and excluded under FOIA are as follows:

Governmental Entities
Whose Records Are Subject to FOIA

• All "agencies," "departments," "regulatory commissions," "government controlled corporations," and "other establishments" within the executive branch.

• Cabinet offices (such as the departments of Defense, Justice, Treasury, and State). This includes the agencies housed within the Justice Department: FBI, Immigration and Naturalization Service, and the Bureau of Prisons.

• Independent regulatory commissions, such as the Federal Communications Commission, the Nuclear Regulatory Commission, and the Consumer Product Safety Commission.

• Presidential commissions. These exist for a fixed term and are usually created to study problems and recommend policies regarding such matters as airline safety and CIA domestic spying (perhaps the most famous is the Warren Commission, created by President Johnson in 1964 to investigate the assassination of President Kennedy).

• The Executive Office of the President. The White House bureaucracy that works directly for the president, including the Office of Management and Budget, the National Security Council, and the Council of Economic Advisors.

Governmental Entities
Whose Records Are Excluded from FOIA

• The United States Congress

• The federal courts

• The president and his immediate staff (chief of staff, press secretary, national security advisor)

• Private corporations

• Organizations that receive federal funding but are not controlled or administered by the government (the American Red Cross, the Corporation for Public Broadcasting)

• State and local governments

    The debate over FOIA's passage was intense partly because the political system was entering uncharted territory. Agencies put forth an apocalyptic scenario of the consequences. Criminals would use the law to evade FBI capture. Soviet spies would pillage CIA secrets. The State Department would be unable to conduct sensitive negotiations with foreign countries. Tax cheaters would flaunt the Internal Revenue Service. Trade secrets for everything from bubble gum to birth control pills would be ripped off from the Federal Trade Commission. Military personnel would be at greater risk in times of conflict. U.S. agents would be assassinated when their covers were blown. No one would dare be an informant for a federal agency. The costs of implementation in terms of human resources (then referred to as manpower) and dollars would break the bureaucracy. Organized crime would run amok while FBI employees labored over frivolous requests. Government would cease to function effectively, paralyzed by a deluge of invasive demands for information.

    The other side of the debate conjured up an impressive litany of democratic ideals: the public's right to know, governmental accountability, effective checks and balances, a more informed citizenry. It also asserted that the law's safeguards would protect government functions. Agencies still controlled their secrets: they could refuse requests using one of the nine exemptions. Outsiders would never see a secret unless the agency released it or a court compelled its disclosure after determining that there was no harm. The costs in resources would be well worth it, creating a revitalized democracy. Because of the increased scrutiny, agencies would become more efficient and more responsive to Congress and the public. Everyone would win. Moreover, it was time for this change after 190 years of keeping the public estranged from its own government. The secrecy cloak had too often been abused to cover up malfeasance and stupidity.

    After the bill was passed, both sides nervously assessed the outcome. Apart from the hyperbole about democratic ideals on the one hand and bureaucratic disaster on the other, no one really knew exactly what the impact would be. The first few years of FOIA produced middle-of-the-road results. Government did not grind to a snail's pace. Criminals and terrorists did not reign. Bureaucracy and democracy were not radically transformed for the better. Agencies and requesters alike floundered in the act's vague definitions and unclear procedures. Secret keepers and pro-disclosure interests competed for advantage in fleshing out the new process. By and large, the agencies resisted the law—some mildly, others fiercely. In 1966 and 1967, the FBI exerted maximum pressure on the Justice Department for a complete exemption of its files.

    Significant information was released. The public, Congress, and the media gained more insight into government operations than ever before. Media reports citing "documents obtained through the Freedom of Information Act" became increasingly common. But the law remained a relatively elite, if not arcane, tool used predominantly by four groups: citizens curious about their own files, reporters working a story, academics conducting research, and lawyers working a case.

    One prominent media spin was that Congress had created a windfall for the legal profession—an extremely useful tool provided free or with a taxpayer subsidy (this was in an era in which lawyers were not the target of negative stereotypes that sought to portray them as a societal scourge). As a Washington Post article assessing FOIA's debut described:

The boxes [29 boxes at the Federal Trade Commission] were in response to a single Freedom of Information Act request from a lawyer on a fishing expedition, a symbol of a law that has not worked out exactly as planned.... "Everybody over there [lawyers requesting data from the Food and Drug Administration] is on a big fishing expedition for new information on pharmaceuticals," says one freedom of information specialist with a big law firm.

    By the early 1970s it had become clear that FOIA was not working as Congress had intended and supporters had hoped. Bad press was the least of its problems. Some agencies were conducting administrative guerrilla warfare against it. The Congress, energized by Watergate, by executive branch deceptions about Vietnam, and by revelations of disturbing (if not illegal) activities by intelligence and law enforcement agencies, was ready to strengthen the act. In 1974, to save the law from becoming a hollow symbol of public disclosure, Congress took action.

    Agencies had been subverting the process by taking months, or indefinite periods, to respond to requests. They often charged a dollar per page for copying documents when the going rate was a few cents. Congress's ire was piqued by the discovery that some agencies kept files on its members.

    Colorful New York Congresswoman Bella Abzug, wearing her signature wide-brimmed hat, read excerpts from her CIA file into the congressional record. She had obtained it using FOIA. It went back twenty-two years to her days as a lawyer and political activist. Congressman Charles Porter of Oregon found that the CIA had a file on him as well. He was given seventeen documents under FOIA. One included a report on his attendance at a meeting of the Congress of Racial Equality (a civil rights group). An irate Porter responded: "What the hell does that have to do with the CIA? They're treating me like a security risk."

    Congress strengthened the act. Agencies must reply to requesters within ten working days. Copying costs were to be at normal commercial rates. Agencies were required to devote resources to administering the law and to report annually to Congress on their FOIA activities and compliance. Perhaps most significant, the federal courts were written into the law as the arbiters of the disclosure process. President Gerald Ford vetoed the changes; Congress overrode him with a two-thirds majority. The reforms were law.

    The inclusion of the courts was a major change. Requesters now had a specifically defined, legal right to appeal an agency's rejection. Congress put the burden of proof squarely on the agency to show that the withholding was justified. In essence, Congress was bringing the court system to bear on the bureaucracy, to enforce the public disclosure process. This would have a profound and lasting impact on the secrecy system and on the public's access to it.

    Congress's specific inclusion of the courts stemmed in major part from a confrontation with the Nixon administration (for which Congress had developed a deep distrust, culminating in presidential impeachment hearings and Nixon's resignation in August 9, 1974). In 1971 the administration approved underground nuclear tests in Alaska. Experts gave conflicting assessments of the environmental risks. The National Security Agency (NSA) compiled a report on the safety issues, but it was classified. Congress was refused access.

    In federal district court, the NSA's refusal was upheld, simply on the basis of the agency claiming "national security." A federal appeals court partially overturned the decision and ordered the release of pages not related to national security (the appeals court had actually seen the report; the lower court had not). The government appealed to the Supreme Court and won: the high court ruled that the appeals court had erred in ordering the release of any of the report. In its sweeping opinion, the Supreme Court stated that courts do not have the authority to review information classified by government. Courts could only review the agency's compliance with the procedure for disclosure, not the substance of what was withheld.

    Essentially, FOIA was eviscerated. Agencies could now refuse disclosure and no court could stop them, provided the procedures of the law were followed. Congress responded to the high court's decision by giving the courts a substantive as well as procedural role in monitoring FOIA. In addition to interpretations of legal precedent and practice, judges were now empowered to read documents and decide if their release would threaten "national security," or whether the public's right to know was overriding.

    Time magazine glowingly reported that the 1974 reforms were working and that FOIA requests had drastically increased throughout the federal government; the FBI was experiencing a "sixteenfold increase": "As a result [of the reform]," said Time, "officials are speedily granting many of the requests for information, a mass of formerly withheld material is being turned over to academic researchers, reporters, and other citizens."

    This perceived renaissance in the public's right to know would be relatively short-lived. The setting in which all political issues exist changes over time; interest group alignments and political values shift. In the late 1970s and early 1980s, the ideological climate changed, and the political pendulum began to swing toward secrecy. Congressional distrust of the presidency and of intelligence and law enforcement agencies had diminished. Key congressional liberals who had championed accountability and disclosure, like Idaho Senator Frank Church, had retired or been defeated. A "conservative revolution" held sway in Washington—pro-defense, pro-intelligence, pro—law enforcement. Ronald Wilson Reagan was president.

    Media criticism of FOIA escalated. A quintessential example is the 1980 Reader's Digest article written by author John Barron, who had written a book on the Soviet KGB: "Congress passed the Freedom of Information and Privacy Acts with the best of intentions, but criminals and spies have perverted that intent to hobble the work of our law-enforcement agencies." A series of anecdotal horror stories alleged that criminals, terrorists, and spies were using the public disclosure laws to undermine our government: "A laughing Soviet KGB officer told a confidant (actually a U.S. spy) that the Russians systematically exploit such laws to extract information...."

    If this was so, it was the fault of the secret keepers for not effectively administering the withholding process. It was not the fault of the law or a consequence of who decided to use it. The insider nature of Barron's piece was revealed by his claim that "some 500 persons affiliated with the Communist Party last year demanded information from FBI files." This is an interesting assertion since requesters are not required to state their purpose or affiliations. Did the Bureau run a records check on the names of those seeking information?

    In the early 1980s, the Reagan administration launched a full-scale assault on FOIA. Executive Order 12356 ended the thirty- to fifty-year limit on classification. No longer did agencies have to show identifiable damage resulting from release, as President Carter had required. Documents could now be classified in perpetuity, so long as required by national security considerations. Government could reclassify previously released documents at its discretion. Secret keepers were ordered to classify material at the highest possible level, reversing the order of President Carter that it be at the lowest level possible.

    President Reagan proposed to Congress a series of FOIA reforms that were not passed: allowing agencies eighteen months to respond to requests, limiting requests to one per year per person, increasing fees ten times over. Law enforcement agencies, led by the FBI, stepped up the pressure for broadening their exemptions from FOIA, claiming that their effectiveness was being severely handicapped. In 1983 the Senate Judiciary Committee approved a compromise bill (S774) that would have drastically altered FOIA's timetables, fees, and scope to the detriment of the requester. That bill did not pass. In 1986 Congress did pass the FOI Reform Act, which made it considerably more difficult for requesters to qualify for fee waivers of the costs of finding and copying documents. In the opposite direction, Congress expanded FOIA to include electronic records, which was a major pro-disclosure change. Overall, FOIA had been considerably weakened by this onslaught compared with its invigoration by Congress's 1974 changes. But it had escaped the extreme measures proposed by its most strident opponents, such as Republican Senator Orrin Hatch—measures designed to severely narrow and weaken it.

    The political ebb and flow of the public's right to know, and of its legal centerpiece, continue with each new presidency. A highlight of the efforts of George H. W. Bush's administration to reduce available information concerning government was a proposed "ethics bill." It sought to make it a crime to reveal nonsecrets. In 1989, one month after President Bush signed a proclamation celebrating "Freedom of Information Day," this proposed law was sent to Congress. The law made it a criminal offense to reveal nonclassified information that "would [emphasis added] be damaging to the interests of government" (such as insider information on trade negotiations with foreign countries). Restrictions applied even after a person had left federal employment. This open-ended gag order would preclude the disclosure of millions of pages of documents releasable under FOIA. It would also make it very difficult for former policymakers to write memoirs about their government experiences. The bill did not pass.


Excerpted from Secrecy Wars by Philip H. Melanson. Copyright © 2001 by Brassey's, Inc.. Excerpted by permission. All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.

Customer Reviews