Public Administration in Today's World of Organizations and Markets

Herbert A. Simon, Carnegie Mellon University

It is a special pleasure for me to give a lecture named in honor of John M. Gaus. My files still retain several gracious letters from him to a young colleague, one after a session in which we had participated at the 1950 APSA meeting; another commenting on a paper I had published in APSR in 1953.

When I encountered the world of public administration 65 years ago, John Gaus was just undertaking, with Leon O. Wolcott, their remarkable study, Public Administration and the United States Department of Agriculture (1940). Don Smithburg, Vic Thompson, and I drew heavily on that work when we wrote our textbook, Public Administration, in 1950. Gaus and Wolcott's work today continues to throw valuable light on how organizations come into existence and grow, often under the influence of technological advances that provide them with the potential for important new activities. The innovations in the Department of Agriculture that their book examined included forest conservation, vaccines to combat animal diseases, and the parity principle to deal with agricultural overproduction during the Great Depression.

In fact, Gaus and Wolcott's analysis of the interaction of organizational and technological change resonates strongly with today's interest in the processes and rates of growth of public and business organizations.

Research on Complex Organizations

This brings me directly to my topic for this evening: the mechanisms that make complex organizations effective instruments for carrying out human purposes. And even closer to the theme of John Gaus's study of the interactions between technological innovation and changes in governmental organization, I should like to ask what kinds of organizational structures facilitate change and innovation. I will talk about both private economic institutions and government.

Recently, the Russell Sage Foundation sponsored several conferences involving some Nobel Prize winners in economics who have strayed into political science, often either to show how economic analysis could explain political phenomena (i.e., "Public Choice") or to discuss the merits of markets and private enterprises as ways of getting our society's work done (Alt, Levi, and Ostrom 1999). I was invited to participate in the conferences as a Nobel economist, but I treasonably defected to my political science origins in order to defend our political institutions against the imperialism of utility maximization, competitive markets, and privatization.

Neoclassical economics created a unified framework for "explaining" virtually all human behavior as produced by an Olympian process of utility maximization that recognizes no limits to the knowledge or thinking powers of the human actors. The neoclassical framework assumed a static equilibrium and, as soon as serious attention began to be paid to dynamic phenomena and uncertainty in large, complex social systems, the structure began to deteriorate, and continues to crumble today.

Today, economics is in an increasingly chaotic and productive state of disorganization, searching for an alternative picture of economic mechanisms and human rationality--that is, of the genuine bounded rationality of which people are capable. There are theoretical proposals galore; what is still in short supply is detailed empirical research (of kinds that are well-known in political science) to determine how human beings actually go about solving problems and making decisions.

I do not intend to reopen the whole range of questions posed by bounded rationality, but will direct my remarks to just one institutional aspect: Why, in a modern society, do we have markets, and why do we have organizations, and what determines the boundary between these two mechanisms for social organization? These questions go to the heart of the roles of our diverse political and administrative institutions, public and private, in contemporary society.

Markets as Coordinating Mechanisms

If we were to take an extreme libertarian view, both markets and organizations would be unnecessary. For the libertarian, human beings are Leibnitzian monads: hard, elastic little particles that bounce off each other without any other interaction, certainly without either responding to or influencing each others' values. Libertarians can hold to their faith only on the absurd assumption that my exercise of freedom never affects your ability to exercise yours. Quite the opposite: The freedoms and the fates of all six billion of us who occupy this globe are inextricably interwoven.

Markets and organizations allow human beings to do together, through interchange of information and the ensuing coordination of activity, things they could not do independently. Coordination simply means organizing activity in such a way as to handle the problems that arise because the behavior of each participant depends in some ways on the behaviors of the others. I hardly need explain why such dependence is often valuable; if you absent-mindedly drive in the right instead of the left lane while visiting Britain, you will find out all too soon. Organizations, some quite large, especially armies, have been with us since the earliest historic times and earlier. Perhaps for that reason, we take them for granted, and they excite in us less wonder than do markets, which developed somewhat later, first locally, then over increasingly long distances. The most peculiar characteristic of markets, Adam Smith's "invisible hand," is their ability to secure coordination without obvious central planning, and without a common interest among their members, for each buyer and seller is supposed to be pursuing independently his or her own private interest.

But this invisibility of mutual dependence is deceptive. The usefulness of markets depends on a shared knowledge of the prices and the characteristics of goods that are being traded, the absence of serious third-person effects (so-called "externalities") that are not reflected in prices, and sufficient stability of products and manufacturing practices so that both sellers and buyers can plan their activities rationally and make rational decisions to sell and buy at the prices at which the markets equilibrate. They also depend critically on the safety of transit routes. The effects upon buyers and sellers of agricultural products of prolonged drought, or the effects of closing a strategic strait in a major trade route between India and Europe provide vivid examples of the fragility of markets in the face of various forms of uncertainty, and the social and human distress that can be caused by their malfunctioning.

In order to use markets to provide oil for the lamps of China, oil well owners must know that there exists a land, China, where oil will be used in certain volumes at certain prices for at least the proximate future (the relevant time horizon depending on the time required to produce the oil and amortize the investments). And the Chinese buyers will acquire oil lamps only if they believe that oil will be purchasable at a price that makes oil lamps competitive with alternative light sources. Substantial stability of manufacturing, consumption, and trade is essential to markets' working effectively. And, of course, social institutions, and governmental organizations in particular, play an essential role in maintaining (and occasionally destroying) that stability.

On another dimension, where there are many competing commodities, similar but not identical, price information may have to be supplemented by product quality information offered by organizations like Consumer Reports so that buyers can compare competing brands, or by governmental regulations to protect them from injurious products. If we wish to understand how complex markets can be, we can turn to building-construction contracts, or contracts for manufacturing large custom-built machinery, and count how many pieces of information have to pass between designers and builders before a contract can be sealed, and how much daily interaction takes place between seller and buyer while the transaction is being completed. Such contracts might almost better be viewed as agreements to form temporary organizations for the duration of particular construction or manufacturing jobs.

In summary, markets are, indeed, remarkable coordinating mechanisms in the parsimony of their requirements for information. But they are far less parsimonious than they appear at first blush, for they require a high degree of economic stability and a low level of externalities in order to operate. Moreover, in important classes of market transactions, much product information must flow in the negotiation of the exchange and the subsequent manufacturing process. Adam Smith's invisible hand is often highly visible. Consequently, when the qualifying conditions for stability of markets are not met, as, for example, in wartime, we see a rapid movement toward centralized planning as the preferred coordinating mechanism for many activities.

From a Market Economy to an Organizational Economy

We are so accustomed to hearing our society described as a market economy that we are often surprised to observe that, since the time of Adam Smith, markets have steadily declined, and business (and governmental) organizations have steadily grown as the principal coordinators of economic activity. In Adam Smith's time, almost the only economic organizations beyond the scale of individual families were agricultural estates directly managed by their owners or through stewards, and relatively small shops owned by guild masters. The putting-out system was a market system, not an organizational system, though one with a special coordinating role for the capitalist who contracted for the successive stages of manufacture of the products--from flax to yarn to cloth to a peasant's blouse. The contractor did not operate as an employer managing a factory. Adam Smith took a dim view of large organizations where management became separated from the direct oversight of the owner. Looking around for examples of such organizations, he found mainly universities like Oxford and Cambridge, which he described as inept, inefficient, and corrupt. (One could claim that Smith anticipated our golden parachutes for salaried executives. Perhaps he was forewarned by the not-infrequent speculations by stewards of the estates of the gentry and aristocracy.) But in spite of Smith's skepticism, organizations have grown until the vast bulk of our economy's activity takes place within the walls of individual large business corporations, not in markets. This growth had already begun to root itself, in the coal mining, iron, ceramics, and textile industries, at the time that Smith was writing his great work, and entered into land and sea transportation a generation or two later. (He foreshadowed it just a bit in his tale of the efficiency of specialization in the manufacture of pins.) It was triggered in large measure by technological advance, especially the invention of the steam engine and its applications as a centralized power source for a factory or mine and, later, for a ship or train. footnote 1 Today, in consequence of these developments, we do not live in a market economy, but in an organization economy, or at most, in an organization/market economy, with a predominance of organizational over market activity. It is ironic that one of the first industries to move toward this new kind of organizational society was transportation, where the railroads enabled an enormous rise of market exchanges over long distances, with correspondingly large factories to produce the goods that were exchanged. Electronics is now completing the comparable transformation of communication.

Now, before going on to my next topic, I must issue one caveat. Current developments in electronics, notably the development of the World Wide Web and e-markets, and the enhanced abilities of organizations to manage geographically dispersed activities, provide new opportunities of unknown magnitude for coordination at a distance. Today, we have very little experience with these new developments, both their current forms and their potential. Hence, there is as yet little basis for judging whether markets or organizations will be best able to make use of the new opportunities and whether, as a consequence, we will see a continuation or acceleration of the current trend towards concentration of productive activity within organizations, or will see that trend slowed or even reversed in favor of markets.

How Organizations Coordinate

To understand this growth in organizations, business and governmental, we must understand organizations' ability to coordinate complex activities efficiently, and at far higher levels than markets can attain. As organization theory has long taught us, coordination is not a good but a necessity. Coordination is costly and imperfect, and we wish to introduce no more of it than the structure and intricacy of our goals calls for. Stated a little more positively, organization design focuses on balancing the gains from coordination against its costs. The first step in designing an effective organization is to determine what kinds of interdependencies in its activities will benefit from coordination, and then to minimize the amount of coordination required by partitioning activities in such a way that a much lower rate of interaction, on a more leisurely time scale, is required between subunits at any level than is required within each subunit. This is the familiar division of work. The same issues of balance between the benefits and costs of coordination that guide organizational design also play a major role in defining the boundaries between organizations and markets, which are defined by the decisions of when to make things or perform services within the organization and when to buy them from outside vendors.

In any case, the basic reality of the division of work is that high rates of rapid communication are required among people who perform activities that are highly interdependent, much less frequent communication is required among those carrying out activities that are independent, and this distinction should be clearly reflected in organization structure. Systems whose structure reflects these properties are referred to as "nearly decomposable." And a formal mathematical theory exists today that describes them and makes important predictions, as we shall see, about their behavior.

Organizational Identification

A second component of organizational design is the special contracts between the organization and its participants: for example, employment contracts with those who work in it, stock and bond ownership contracts with those who contribute capital, and sales contracts with suppliers and customers. A key feature of organizations is the employment contract, which "buys" the employee's efforts during working hours so that they can be applied to the organization's goals. Of course, ordinary economic motives play an important role in securing employee acceptance of employment, but far more is involved. Once installed in the organization, the employee is surrounded by information and influences quite different from those that would surround him or her in another setting, inducing in the employee a strong identification, not only motivational but also cognitive, with the organization and its goals. It is this mechanism of organizational identification that Adam Smith missed when he concluded that large organizations with hired managers could not be efficient.

Organizational identification is a powerful motivator, rooted both in people's values and in their need to build a simplified model of the world that focuses upon their particular responsibilities and work environment. It is distinct from the self-interest which, of course, also plays an important part in organizational behavior.

It is the organizational identification of members, more than any other of their characteristics, that gives organizations their remarkable power to secure coordinated behavior of large numbers of people to accomplish organizational goals, thereby playing a major role during the past two centuries in the rise of modern organizations and in their successful competition with traditional market mechanisms. footnote 2 Identification has received too little attention in our research on organizations.

Organizational identification does not depend on profit motives; it can work within governmental and university organizations as powerfully as within profit-making businesses. Such studies as have been made (not as numerous as one would like) show that, on average, profit-making and governmental organizations that produce the same products, both operating in markets, attain about the same levels of efficiency--the profit motive appears to give no visible competitive edge to private business. So the increasing tendency in recent decades for government agencies to contract out many of their activities evidently is not driven by considerations of efficiency--or, if it is, there is little solid empirical evidence for this preference.

What are the implications of this picture for the role of organizations, and especially governmental organizations, in our society?

Organizational Innovation and Adaptation to Change

The interest in recent years of many sciences in complexity and complex systems has drawn attention to the fact that most of the complex systems seen in the world are nearly decomposable systems. They are arranged in levels, the elements at each lower level being subdivisions of the elements at the level above. Molecules are composed of atoms, atoms of electrons and nuclei, electrons and nuclei of elementary particles. Multicelled organisms are composed of organs, organs of tissues, tissues of cells.

Why is this principle of organization so universal? The answer to this question has two parts. The first I have already discussed: Near-decomposability is a means of securing the benefits of coordination while holding down its costs by an appropriate division of labor among subunits. So, if we design complex systems to operate efficiently, we must incorporate near-decomposability in the design.

But most of the complex systems we see in nature were not designed; they evolved through the processes of natural selection. In evolution, there is a different, but closely related, reason why near-decomposability prevails. In an environment of evolutionary change and natural selection, nearly decomposable systems will adapt to the changing environment and gain in fitness more rapidly than systems without this property. I can indicate roughly why this is so.

If increases in fitness of one organ of an organism do not affect the fitness of other organs (except possibly for adjustment of relative size) and if increases in organ fitness are reflected in the organism's overall efficiency, natural selection will home in rapidly on the favorable changes that occur. Thinking of the evolution of organs as a problem in design, designing each organ to adapt to changing requirements will be much easier if the design of any one organ has little effect on the efficiency of the others; if the heart can be designed without redesigning the lungs, for instance. With a higher degree of dependence, the continued "favorability" of any change in one organ will depend on what changes occur in the other organs at the same time or in the future. There is, then, no simple basis for selecting those changes that will continue to benefit the whole system, creating a dependency among the structures of the separate organs that is highly inimical to change.

If complex systems must operate in a constantly changing environment, or in competition with other systems that are changing, they must modify their structures at a corresponding pace. The need for close coordination, even in the presence of strong identification with the organization's goals, places a very heavy burden on a system's capacity to evolve toward greater effectiveness under changed conditions. For although identification reduces the need to police self-interest and to ensure its compatibility with organizational objectives, it also causes excessive influence of existing organizational practices and identifications upon decisions that should be adapting to a changing world. This is the major cause for the difficulty that organizations, even very successful ones, experience in trying to respond to rapidly changing opportunities and challenges, and why they are often outpaced by new organizations that do not carry the same burden of outmoded knowledge and habit.

Near-decomposability affects the growth potential of market systems as it does the potential of organizations. Although markets are weakly coordinated systems, permitting a considerable degree of independent change in each component, the change must not be so rapid as to destabilize the expectations of participants that, in stable markets, make large short-term flows of information unnecessary. When markets must compete with organizations as means for securing the benefits of progressive change by rapid adaptation, the former are likely to succeed in this competition only in highly stable environments.

In light of these considerations, we might read the history of the past 200 years of industrializing societies as showing that, with the growing advantage that large organizations secured from advanced technology, and our increased skills in designing large organizations that achieve high levels of coordination which maintain a reasonable approximation to near decomposability of their components, we have enlarged greatly the area within which organizations are more effective than markets. This conclusion applies to governmental as well as to business organizations, for both have followed similar paths in the development of their designs.

Social Implications

Let me turn now from these rather abstract and high-level pictures of social systems and the course of their development and ask why it matters. I will focus on two issues: first, the distribution of power in modern society, and, second, the distribution of the social product. I'm afraid that the picture my crystal ball displays is still abstract, but it does refer to matters that are of very practical concern to all of us, individually and collectively.

The Distribution of Power

Lord Acton said it very well, and I don't think I can say it better: "Power tends to corrupt, and absolute power corrupts absolutely."

A central problem in democratic institutions at all times and in all places has been and is to create a broad distribution of power, and to keep that distribution in stable equilibrium. A basic tenet of democratic theory, well supported by historical and other evidence, is that self interest is such a strong motive that no fraction of members of a society can be safely entrusted with the freedom and welfare of others who do not participate in the decision-making processes.

During the twentieth century, two massive experiments were carried out, as well as numerous smaller ones, to determine whether major changes in political and economic institutions could produce the "new person" and the new institutions required to realize basic human goals within a highly centralized system of power. It is now widely agreed that the new institutions didn't work well and, especially, that the "new person" didn't appear. The personal motives that emerged in Soviet Russia and in Maoist China appeared to be no less selfish or more virtuous than the motives of people in other parts of the world--in fact, they were discouragingly familiar.

Among neoclassical economists, the outcome of these experiments was widely interpreted as a clear vindication of markets being the bulwark of freedom and productivity. The years since the dissolution of the Soviet Union and the complex developments in an industrializing China have shown that matters are not so simple. As to Russia, it has become painfully clear that the introduction of markets without the coincident introduction of socially enforced rules of the game for their operation and the simultaneous creation of viable and effectively managed organizations cannot create a productive economic system. Nor has a stable equilibrium of diffused power been established in Russia. In China, there remain substantial deficiencies in the social enforcement of market rules and, at the same time, continuing governmental interference with normal market operations, as well as equally evident deficiencies in organizational skills and steadfast resistance to the decentralization of power.

Nothing that we have seen in these two histories challenges the thesis that diffusion of power requires, in addition to markets, a multiplicity of effective organizations to perform a society's productive and service tasks. In fact, it can well be argued that the most important role markets play in a modern society is to diffuse power by holding organizations, through competition, to the tasks of providing efficiently the things demanded in the markets, thus preventing them from using their resources as power bases for extending their social influence and control by direct influence upon government. A multiplicity of organizations competing vigorously in markets is a strong protection against diversion of resources (by either for-profit or not-for-profit organizations) to political objectives.

By the same token, diffusion of power calls for governmental organizations that maintain a reasonable balance between the effectiveness that large business organizations can sometimes provide and an avoidance of concentration of power in a few places. The many experiments with privatization of services that had previously, for good or indifferent reasons, been supplied by public agencies, are beginning now to show us that switching to the market/business-organization system is not a sovereign remedy for all administrative ills.

To illustrate what I have in mind, I need merely mention the complex mixture of gains and losses that deregulation of the air transportation industry has brought to its customers (in spite of rosy reports of fare savings). The same can be said of deregulation and privatization of energy distribution, education, and communications, all of which are faced today with perplexing economic and organizational problems. I could add other examples, notably the prison industry, which has not become a magical cure for criminal tendencies as a result of experiments in privatization.

Nor can we say that we have solved all of the organizational problems posed by public goods and by such externalities as those associated with preservation of the environment. Experience has indicated that a wide range of essential services can be provided better by government than by any private business arrangement thus far invented, or, as in the case of basic research, will be undersupplied because of public goods aspects, if left to competitive markets.

We do not need to reinvent government. Governmental organizations are needed, as they have always been needed, to enforce the rules of the game (including the rules of market contracting), to facilitate coordination of private organizations, and to perform services that are unlikely to be performed effectively by the private sector. The legal institutions must be vigorous and independent enough to curb corruptions of the rules of the game by bribery and other illegal activities. And the rules of the game themselves (e.g., rules for political campaign contributions) must themselves not enable influence buying. footnote 3 In performing these functions, government agencies themselves, of course, become centers of power that help balance the power exerted by the private sector in its own interests.

Distribution of the Social Product: Levels of Employment and Production

There is little consensus in economics today about how to maintain high levels of employment and production, even though, at the moment, we are enjoying these conditions; many economists, when asked why, look toward Heaven and sigh. Among those of my friends and acquaintances who are macroeconomists or specialists on money, I find only one point of general agreement (and even here there is dissent from diehard believers in the gold standard). Most agree that money is neither a solid substance, nor a liquid, nor a gas; it is simply a state of mind. More precisely, money's value is a collection of states of mind of all the people who use it. These states of mind, as history shows, can change in a short time from utmost confidence in a currency to utmost skepticism, and vice versa. On the role of government spending and monetary policy in determining the level of activity, "expert" opinions range from the laissez-faire of rational expectations to Keynesianism and beyond.

Having identified this important topic and the disarray of expert views about it, there is only one thing about it that I can say with confidence: Maintaining economic equilibrium cannot be left to the invisible hand of the market, it requires governmental attention. What economists call Say's Law guarantees that the economy can be in equilibrium at any level of activity from 100% employment of workers and capital to zero percent. At both of these levels, and all between, the income from selling goods will just balance the costs of producing them plus the profits of owners. So production produces total incomes that are just sufficient to purchase the marketed product. When markets are not at full employment equilibrium, as they often aren't, neoclassical theory does not explain why, nor what to do about it.

Distribution of the Social Product: Fairness

Just as competitive markets cannot, by themselves, guarantee an adequate distribution of power in a society or full employment, so they do not guarantee that markets will distribute income and wealth in a way that will satisfy our notions of fairness. "Fairness" is, of course, not a question of fact, but of values. What is fair cannot be settled by our science. Hence, I will limit myself to showing the consequences of a particular definition of fairness that has wide currency in our society today: that it is fair that people receive and be allowed to retain what they earn. I am not advocating your adoption of this definition, I'm simply using it to illustrate the complexities of the design of social systems that take the question of fairness into account.

Let me pose a simple question. Consider the income that you or your family now earn as members of American society (which most of you are) and compare it with the income that you would expect to earn if you were equally hardworking members of Chinese or Indian society, or the society of any other Third World nation. I expect that for most of you, the difference between the two incomes is one or more orders of magnitude, at least 10 to 1 and perhaps even more than 100 to 1.

Now, I would like you to consider the causes for the gap between the 10 and the 1 or the 100 and the 1. How much of it do you wish to attribute to your superior energy, motivation, and application of effort as compared with your Third World counterparts? And how much do you wish to attribute to your good luck or good judgment in being born in, or joining, the highly productive and democratic American society?

If we are very generous with ourselves, I suppose we might claim that we "earned" as much as one fifth of it. The rest is the patrimony associated with being a member of an enormously productive social system, which has accumulated a vast store of physical capital, and an even larger store of intellectual capital--including knowledge, skills, and organizational know-how held by all of us--so that interaction with our equally talented fellow citizens rubs off on us both much of this knowledge and this generous allotment of unearned income. Again, I have no specific proposal for allocating the "unearned" income of an affluent society. That, of course, is one of the important things we will be voting on in November. Clearly, the allocation of income in a society is a matter of values to be decided by political processes.

What I wish to emphasize is simply that public attitudes about the fair allocation of income are necessarily and justifiably a major factor in determining the scope and nature of public organizations in the society. There is no way in which the proper allocation of the social product can be left solely to the market in a private-organization/market/public-organization society, or solely to considerations of productive efficiency. Society is demonstrably not a collection of Leibnitzian monads. Much more flows between the members of a society, in the form of exchange of information and cooperation, than the simple interchange of momentum by impact. Market equilibria that are Pareto efficient will often be inferior to other equilibria, Pareto efficient or not, when criteria of fairness are applied.

Conclusion

It is not too fanciful to think of writing a history of human civilization in terms of progress in the means of human cooperation, that is, of organization. In that history, hierarchical and nearly decomposable systems would play a central role. Almost from the beginning, the division of work into component tasks and the assembly of the components into a hierarchy were discovered to be powerful means for achieving efficient coordination of effort. In a later period, markets entered as a means of coordinating certain kinds of transactions that required very limited communication, and therefore can take place over long distances. Then, long before the Christian era, nation states and empires emerged that show there is almost no upper limit to the sizes to which organizations can aspire.

The system, after more than a millennium and a half, entered a new stage when the social store of knowledge and innovation in technologies begin to accelerate, with markets again at first playing a central role in enabling these developments. Gradually, increases in the demands for, and in the advantages of, more coordination in economic activity, together with the accumulation of skills of organizing, brought into existence ever-larger corporations that begin to emulate in size the administrative organizations of the nation-states--and we were launched into our modern world.

Both private and public organizations have played essential roles in these modern developments, complementing each others' functions, learning from each other, and, at the same time, competing for power to steer and manage the systems that have emerged. That process has not reached its end, and political science and economics must continue their mutual education, with each discipline learning from the other.

The education must be symmetric. Its goal is not to convert political science into a theory of the "public choices" of a mythical, utility-maximizing "economic person." Its goal is to understand how human behavior molds and is molded by the complex structures it employs to secure the coordination needed to accomplish most of its goals, and how this can be accomplished while preserving the wide dispersion of power that is consistent with democratic institutions.