January 1, 2006  

Measuring success

How can we know if we are succeeding in Iraq? This is one of the central problems facing the Bush administration, and America, today. The counts of “enemy” bodies that characterized the Vietnam-era approach to this question have fortunately vanished — the military properly and decisively rejected giving significance to body counts from the very beginning of the struggle. But the search for numerical measures of progress continues unabated, and in the absence of body counts, similarly distorting substitutes have emerged.

Many in the media focus on American casualties and the numbers of attacks on U.S. or Iraqi soldiers, police, contractors and civilians. U.S. Central Command (CentCom) briefings focus heavily on the numbers of raids, cordons-and-searches and other types of military activity conducted. The Bush administration and the military have also been providing detailed briefings about the numbers of “trained and equipped” Iraqi forces available each week. This last number has largely replaced the Vietnam body counts as the single most important numerical measure of success the administration offers.

None of these numbers, however, provides any clear notion of whether the American experiment in Iraq is succeeding or failing. U.S. casualties, while tragic at any level, are comparatively low for recent struggles of this variety — the approximately 840 U.S. troops who die on average each year in this conflict is about half the 1,500 to 2,000 Soviet soldiers who died annually in Afghanistan, and about one-eighth of the 6,000 Americans who died annually at the height of the Vietnam War (even adjusted for the size of the overall forces deployed, annual casualties in Iraq are running at about half the rate of Vietnam and Afghanistan). Neither their absolute magnitude nor that historical comparison, however, is any measure of the degree of coalition success in Iraq at this point.

The statistics about the numbers of “incidents” or “attacks” against coalition targets in Iraq are slightly more meaningful. They are also seriously misleading, however, because they normally include all incidents, whether initiated by the insurgents or resulting from coalition offensive operations, and because they do not discriminate against massive bombings and failed attempts to emplace roadside improvised explosive devices.

More discriminating statistics provide an ambiguous reading. Attacks on the Oct. 15 referendum, for instance, were far lower than attacks on the Jan. 30 elections. The insurgents launched fewer attacks in August 2005 than they had in August 2004. The rebels are generally placing more IEDs, but the coalition is defusing more of them before they explode, and the complexity of the IED attacks is lower than it had been.

The Bush administration’s favorite statistic, the number of trained Iraqi forces, is even more misleading. Measuring the training level of military forces is always a complicated business. Trying to assess the capability of Iraqi forces rushed quickly through abbreviated training and then sent out to partner with American units in on-the-job training is almost a fool’s errand. Numbers based on equipment and personnel fill, training hours and so forth say little about the unit’s actual capabilities. The real evaluations are inherently subjective based on the trainer’s view of the situation — and so are far from being the concrete data points usually presented.

Numbers of “independent” operations conducted by Iraqi Security Forces (ISF) troops are also hard to evaluate, since an “operation” is a term without any clear definition, as is “success” in many of those operations. At best, numbers related to “trained” Iraqi troops measure the potential for success in transitioning to Iraqi control over the counterinsurgency. They say little about the actual use of those forces or the consequences of that use.

The fundamental problem with all of these numbers is that they represent things that are easy to measure rather than phenomena that actually indicate the progress of the war. Senior leaders occasionally admit as much. For instance, in response to a recent question about whether the number of foreign fighters in Iraq was increasing or decreasing, Marine Gen. Peter Pace, chairman of the Joint Chiefs of Staff, responded, “I don’t know that that’s definable. I think what is definable is the amount of territory that is being controlled, for example, by the Iraqi armed forces. Today we have one division headquarters, four brigade headquarters and 24 battalions that are Iraqis, who are in fact controlling areas of their own country, providing protection for their own citizens, and that will continue to grow, which will squeeze out the insurgents.”

The media are counting U.S. casualties. The military counts Iraqi soldiers. Both are measures of convenience, reflecting the ease with which data can be collected and presented rather than its inherent importance. Neither tells us whether we are winning or losing in Iraq. In order to consider and address that more fundamental problem, we must return to the definition and basic nature of an insurgency and reconsider what it is that we really need to be measuring in the first place.

The primary characteristic that distinguishes insurgency from conventional warfare is its trinitarian nature. In conventional conflicts two sides engage in what Carl von Clausewitz so aptly called a duel, as each seeks to break the will of the other to continue fighting. An insurgency is no such matter. In revolutionary struggles, two (or more) sides compete to gain the support of the mass of the population. In general, the government struggles to retain or establish its legitimacy in the eyes of the people while the insurgents attempt to destroy that legitimacy. The real battle is, therefore, within the minds of the people of the unfortunate country thus torn.

This situation is no less true or important in Iraq than in any historical insurgency. The most fundamental problem the coalition faces today is that a significant (although unknown) percentage of the Arab Sunni population remains unwilling to accept a subordinate role in governing a country it had dominated for centuries. This unwillingness is manifested in attacks on coalition and, increasingly, Iraqi forces and in tacit support for the terrorist networks of Abu Musab al Zarqawi and others who are pursuing different agendas.

The elimination or dramatic reduction of Sunni rejectionism would not put an end to all attacks. Zarqawi and his ilk will probably continue to try to undermine any Iraqi government for their own purposes regardless of changes in the sentiment of Sunni Arabs. But a significant decrease in Sunni rejectionism would also reduce the degree of that community’s support for the rebellion, upon which Zarqawi relies, would reduce the number of attacks launched by Sunnis not aligned with Zarqawi, and would in general lower the level of violence in Iraq dramatically. The Multinational Forces West commander, Maj. Gen. Stephen Johnson, recently noted that “the insurgents in Al Anbar province and north Babil province are largely locally based insurgents; that is, the insurgent we fight here is from here, he’s from those communities in which we are engaging them.” Clearly, controlling Sunni Arab rejectionism would go a long way toward reducing violence against the coalition. It would also lead to a much greater role for Sunni Arabs in the Iraqi political process, pointing the way toward the establishment of a stable Iraq.

How can we measure progress toward this objective? In many past insurgencies, it was not possible to do so directly. The insurgents frequently controlled important parts of the most vital areas and could prevent the government or its agents from conducting polling or gathering useful intelligence about the attitudes of the population. This problem is significantly lower in Iraq because of the relative weakness of the insurgency itself. The insurgents are, with few exceptions, unable to conduct meaningful guerrilla attacks against coalition forces. Their recent focus on targeting Iraqis reflects not only the success of the Iraqi political process, but also a recognition that attacks on coalition forces are likely to be relatively ineffective.

The coalition’s failure to keep the bulk of the Sunni Triangle secure by occupying the population centers has had numerous baleful consequences, moreover, but the policy of continuing periodic raids throughout the area, rather than pursuing an “oil spot” strategy that abandons much of the country to the insurgents, has had one important benefit: The insurgents have not been able to set themselves up in clear control of any single area for a long time. They have been able to deny the Iraqi government the ability to establish a stable and coherent reign in the Sunni Triangle, but they have not been able, as the Viet Cong, the Afghan mujahideen and many other insurgent groups were, to establish their own de facto rule except in short-lived enclaves.

This fact has two important and related consequences. First, it means that coalition forces, including the ISF, have access to the greater part of the Sunni Triangle and can interact with the local population throughout the area. Careful intelligence collection programs designed to use those interactions to gauge attitudes should yield part of a picture of the state of the Sunni Arab mind-set that is at issue in this conflict. Second, it means that traditional polling is possible in much of the region. These two facts offer the hope of establishing a meaningful intelligence cell in CentCom tasked with tracking the feelings of at least the Sunni Arab community, if not all of Iraq, toward the critical questions of the insurgency. Such an effort would provide an infinitely more meaningful series of measures of coalition effectiveness and of progress toward victory.


There appear to be five major sources of information about attitudes in the Sunni Triangle that are or could be made accessible to CentCom collection efforts: polling; crowd observation; coalition forces; contractors, nongovernmental organizations (NGOs) and other civilian entities in Iraq; and the Iraqi Security Forces and police forces. Each of these techniques or agencies has the potential to paint part of the picture of Sunni Arab attitudes so desperately needed to understand the state of progress in the counterinsurgency effort.

n Polling A number of organizations have undertaken periodic efforts to ask Sunni Arabs directly how they feel. CentCom can undertake some of these efforts using its own resources and it can also contract external groups to conduct polls on its behalf. It would also be worth exploring the possibility of assisting the Iraqi government to establish its own polling methods, although the problems of internal Iraqi politics might reduce the accuracy of any such attempts. Polls are inherently inaccurate, of course, and polling in Iraq is a time-consuming activity, since many of those polled are not content simply to say yes or no even to simple questions. Even so, extensive polling and careful evaluation of techniques and results could refine this approach and help correct for its difficulties, and the combination of such results with information drawn from other sources could be very valuable.

n Crowd observation When an explosion goes off and a crowd gathers, what are the people chanting or yelling? How do the crowd’s responses to tragic events differ from moment to moment and region to region? Skilled observers can discern much not only from what people are saying, but also from how organized the crowd response is. Data about such responses should be collected and systematically analyzed.

n Coalition forces The relative weakness of the insurgency allows coalition forces to move throughout the Sunni Triangle, and the coalition practice of striking terrorist targets as they are identified means that coalition forces are touching a large portion of the area, even if only briefly and sporadically. Enlisted soldiers and officers trained to observe local reactions can report those reactions in a meaningful way, especially if military intelligence personnel are embedded in the units for precisely such a data-collection effort. But training is required. There is little benefit in a lieutenant surrounded by an armed patrol asking terrified locals through an interpreter if they like Americans, as happens all too often.

n Civilians A large number of contractors and NGOs have personnel in Iraq, many of whom interact with the local population periodically (although most commonly outside of the Sunni Triangle, to be sure). CentCom should establish mutually acceptable procedures to debrief these personnel periodically about popular attitudes, which would provide a degree of balance to reports generated by coalition soldiers. Many NGOs might be resistant even to this sort of involvement; others might be more responsive. There is no reason why most contractors should not be willing to participate. The bottom line is that any sober observations of Sunni Arab attitudes would be helpful and CentCom should explore all avenues of obtaining such data.

n The ISF and Iraqi police These are, naturally, the very best sources of information about local attitudes, since they can interact with the population without benefit of interpreters. Both the ISF and the police will generate distortions of their own, of course, particularly when units of one predominant ethnicity are deployed in areas of a different ethnicity, as is commonly the case in the Sunni Triangle. Collection of this sort of data should be twofold. Embedded intelligence professionals should operate within each Iraqi unit, again dedicated solely to the acquisition and analysis of data about population attitudes. In addition, unit commanders and senior NCOs should be debriefed regularly for their own perceptions. The training of Iraqi units should also include preparation for generating the most accurate possible picture of popular feeling, and techniques for reporting it in a reliable and systematic way.

Much of the infrastructure is already in place for a great deal of this reporting, if CentCom were willing to divert its attention somewhat from the task of chasing terrorists to the task of measuring success within the population. In addition to the justifications offered above for this diversion of effort, there is one more: CentCom cannot really know if particular approaches, tactics, techniques and procedures are working if it has no meaningful measure of the only quantity that really matters — the attitude of the Sunni Arab populace. Without measuring that attitude on a continuous basis, the feedback loop essential to military success is broken — it is impossible to know with confidence and on a wide scale when a given operation has furthered the counterinsurgency effort or hindered it. In the absence of such information, it is all too easy to fall back on our own preconceptions of what should work and what should be a problem, forgetting that in an insurgency, not only does the enemy get a vote, but the population does, too.

Sadly, it does not appear that CentCom has been taking this approach. From the beginning of the conflict, CentCom briefers have relied instead on the handful of independent polls of Iraqis that appear in the media, repeating simplified versions of their conclusions, and adding no information of their own to the discussion. It is possible, of course, that CentCom prefers not to release such information, or would prefer not to do so even if it began collecting the data. There is the danger that weeks would go by in which there was little progress or even regress in the Sunni Arab mind-set, and the Bush administration can ill-afford to generate any bad news from Iraq.

Such a view is shortsighted, however. The bad news from Iraq continues daily without the administration’s participation, since the media’s measures of choice are American casualties and “attacks” or “incidents.” It is quite true that these numbers have limited significance in measuring American success, but in the absence of any other metric, the administration has gained little traction in undermining the sense of impending doom created by these inherently depressing statistics. Adding another measure with occasional downturns would probably have little overall effect on American public support, but the prospect of developing a measure that might actually portray the success the coalition has been having is very enticing.

Politics aside, it is actually essential to any prospect of overall success in Iraq. Without measuring changes in Sunni Arab attitudes, it is simply impossible to know whether we are winning or losing at the theater level. Without knowing that, it is impossible to know whether to pursue current policies or to change them, or to evaluate how to change them with any basis in reality. Above all, the American people deserve to know how their government is actually doing. In all previous wars save one, the American public has been able to track military success: in the Civil War, both World Wars, Korea, and both Gulf wars, units moving over maps and guesses about the positions of enemy forces and their strengths allowed even the average American to understand how close victory actually was. The only major war that did not provide such measures was Vietnam, and their absence played no small role in the collapse of American will that led to defeat.

CentCom must stop unwittingly repeating the critical mistakes of Vietnam — always while trying to avoid them — and find a way to track and present progress in this war that is meaningful. It is hard to imagine, otherwise, overcoming the current macabre measures of violence and death that highlight pain and point to failure.