Informing the Welfare Debate:
Perspectives on the Transformation of Social Policy

Note: To download Adobe Acrobat (.pdf) documents, version 3.0 or higher is required.

This document is the introduction to a full report, available in its entirety in IRP Special Report 70; (pdf, 167 pp). The full report contains:

• Informing the Welfare Debate: Introduction and Overview, by Thomas Corbett
• Welfare Policy and Caseloads in the United States: Historical Background, by Thomas Kaplan
• Welfare Reform in the United States: A Background Paper, by Michael Wiseman
• Time-Limited Public Assistance Benefits: State Options under Federal Welfare Reform, by Rebecca Swartz and Thomas Kaplan

Introduction and Overview

by Thomas Corbett
Assistant Professor, School of Social Work,
and Acting Director of the Institute for Research on Poverty
University of Wisconsin-Madison

April 1997

Introduction
Reform and Information
The Era of Policy Discontinuity
Is the Welfare Debate Over, or Is It Just Getting Started?
The Academic and Evaluation Communities: Can They Contribute to the Future Debate?
Summary

Introduction

The Institute for Research on Poverty (IRP) began a project in 1995 titled Informing the Welfare Debate. The project has been funded by the Helen Bader and Joyce Foundations. The hope behind the project was that an academic entity such as IRP might mute some of the hard edges of the then rancorous welfare debate by contributing broader analysis and historical perspective to a dialogue sometimes characterized by opinion and partisan positioning. The goals of the project were:

1. To produce objective, credible, and readable documents that summarize current knowledge in critical areas of the welfare reform debate.
2. To disseminate these documents widely among policy makers engaged in welfare reform efforts at the federal, state, and local levels.
3. As necessary and feasible, to conduct workshops and conferences to discuss the material and obtain feedback on how best to present the information and maximize its usefulness.

Over the past two years, the Institute has sponsored or facilitated several events directed toward these goals. Congressional briefings were held in Washington, D.C.⁽¹⁾ Seminars for state legislators and agency personnel were organized.⁽²⁾ Conferences and workshops have been held.⁽³⁾

The welfare reform debate has remained contentious and emotional, because it touches the most sensitive of societal issues: work, family, sex, abortion, personal responsibility, and community integrity. Welfare reform has become a proxy for fundamental questions about quality of life and how to allocate personal and public responsibilities. We are uncertain as a society whether assuring economic security for children is more important than providing clear consequences to parents who fail to play by the rules. We argue whether individual irresponsibility or societal barriers are at the root of welfare dependency. We continually debate the efficacy of available interventions: can economic sanctions and rewards effect changes in behavior? Can the human capital of recipients be raised to a competitive level? Would the most disadvantaged respond if work were really a rational economic alternative? We even debate the fundamental goals of reform--to save money or save people; to reduce poverty or minimize welfare dependency?

No one would credibly argue that information, by itself, will resolve the most intractable of these disagreements. But information, when used properly, can narrow the scope and range of prevailing confusion and identify remaining points of contention more sharply. In some instances, it can temper the most blatant expressions of unsubstantiated opinion. If information is to make a difference, however, at the least it must be seen as objective and credible.

The papers included in this Special Report are designed to provide context and information for discussions of welfare reform that will now occur in the 50 states. The paper by Thomas Kaplan reviews the history of welfare in England and the United States, giving special attention to the establishment of the Aid to Dependent Children program in 1935 and its subsequent evolution. Michael Wiseman focuses on the convergence of political developments that led to passage of the Personal Responsibility and Work Opportunity Reconciliation Act of 1996. The paper by Rebecca Swartz and Kaplan discusses perhaps the most contentious substantive issue of the recent reform debate--limits on the amount of time participants can receive public assistance benefits. The remainder of this paper reviews selected contours of the long debate about welfare and suggests some strategies for reasoned analysis. The four papers in this document are thematically connected but were written as separate essays. Consequently, readers will notice some intentional redundancy.

Other papers will follow, but these set the stage.

Reform and Information

Reforming welfare has been a concern virtually from the moment public responsibility for the poor was codified in the Elizabethan Poor Laws of 1600. Welfare in the United States was largely patterned after the British Poor Laws and remained a local responsibility until passage of the Social Security Act in 1935. Even in the nineteenth century, calls for reform of welfare in the United States were periodically heard.⁽⁴⁾ The welfare reform dialogue focused on the Aid to Families with Dependent Children (AFDC) program has been on the public agenda for over thirty-five years, having begun in 1962. In that year John Kennedy held the first presidential press conference devoted exclusively to AFDC, announcing a major federal investment in social services to weaken the grip of dependency and move recipients into mainstream society.

Kennedy's proposal was based more on faith than evidence. Although the administration lacked empirical support for the proposition that service technologies could reduce welfare dependency, considerable resources were expended on hiring and training social workers to counsel clients toward self-sufficiency. Nevertheless, the AFDC caseload doubled by the end of the 1960s.

In the years since the Kennedy proposal, federal, state, and local governments in the United States have developed, debated, and sometimes attempted a variety of reforms: social service strategies (addressing individual and family problems); institutional strategies (attacking community problems); human capital strategies (improving labor market competitiveness); labor demand strategies (creating or subsidizing jobs); income strategies (giving poor people money); child support strategies (making both parents responsible); social contract strategies (conditioning aid on behavior and obligating welfare agencies to do more than give out checks); "make work pay" and "make them suffer" strategies (either make work more attractive or welfare less attractive); and, most recently, the "Gordian knot" strategy (end welfare because it cannot be reformed).

The War on Poverty that emerged in the mid-1960s was a response, in part, to the fact that neither poverty nor AFDC disappeared despite a vigorous economy and intensive social service strategies. A radical community empowerment philosophy (an extreme form of devolution) prevailed in this early period, as federal dollars were often being funneled past governors and mayors to the smallest governmental and quasi-governmental units--neighborhoods. But the task of changing people and communities taken on by the early poverty warriors proved technically difficult, politically problematic, and expensive. The second wave of the War on Poverty strengthened the federal role, at least for a time.

This second wave, which emerged between the mid-1960s and mid-1970s, fostered an "income definition" of poverty. Poor people were perceived as differing from the rest of society primarily in their lack of money, and the apparent solution was to correct the income shortfall in a simple, efficient, and standardized manner heavily directed by the federal government. Social services were separated from cash assistance and no longer funded on an unlimited basis. Flat grants (as opposed to individualized budgets based on each household's measured costs) were introduced. Client protections were strengthened. AFDC became a federal entitlement, with benefits based almost solely on categorical status (single parenthood) and income and assets that fell below a threshold. Much of the machinery of the early War on Poverty was dismantled. Government was getting out of the people-changing business, except for change that could be obtained through altering economic incentives.

In the entitlement program that emerged, the efficiency principle predominated--officials provided benefits in a simple and standardized fashion that was easily routinized and thus easy to design and operate centrally. The federal government took the lead in proposing improved methods of distribution of benefits. The result was increased federal control over the calculation and disbursement of benefits and over measuring of the accuracy of state and local determinations.

Today, public systems for providing income support for poor children are undergoing a profound transformation. Reform now calls for truncating the federal role in providing cash assistance to low-income families with children--for devolving design, management, and some financing responsibility to state and local jurisdictions. Many reformers argue that governmental units situated closer to the problems are better positioned to reflect local preferences and circumstances and to respond to the challenges with greater imagination and insight. It is widely argued that federal welfare policy has failed, so the responsibility should be turned back to the states.

The character of the devolution that evolves will generate a new set of dynamics, the consequences of which cannot be known in advance. The block grants built into the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 have capped federal contributions (so that they will decline in real terms) and have created an uncertain federal role in ensuring accountability for performance. Because states will now incur full costs of social investments at the margin, fear of welfare migration and concerns over limited funding may, over the long term, constrain state choices severely. Paradoxically, the move to enhance innovation may (again, over the long term) have the opposite effect--flexibility without resources may not be flexibility at all. Whether the current era of reform is, in fact, an unparalleled opportunity or an impending apocalypse remains to be seen. But virtually all agree that it is a dramatic break with the past, an era of policy discontinuity.

The Era of Policy Discontinuity

Why has the welfare world been turned upside down? Have the key issues been decided? Is the debate over? In fact, no one can really answer these questions. But discussions in the last few years concerning who should control public assistance and why welfare is so unpopular provide useful insight.

Who is in charge? Aid to Families with Dependent Children (AFDC), the signature program of welfare in the United States and the focal point of discussions about reform, was terminated in 1996 with the Personal Responsibility and Work Opportunity Reconciliation Act of 1996. This legislation effectively replaced AFDC, Emergency Assistance, and the JOBS program--and the open-ended federal appropriations that supported them--with the Temporary Assistance for Needy Families (TANF) block grant. In effect, the federal government began turning responsibility for welfare back to the states.

The federal role in providing income assistance to poor families with children had been established under Title IV-A of the 1935 Social Security Act, which provided fiscal resources enabling states to assist needy children without fathers. The states defined "need," set their own benefit levels, established within federal limitations the income and resource limits for eligibility, and administered the program. Title IV-A was not particularly controversial when enacted, as it accorded fiscal relief to hard-pressed states while allowing substantial state flexibility and control over the program. But increased federal regulatory oversight eventually followed, particularly in the late 1960s and 1970s, when stronger rights were attached to beneficiaries and the benefits themselves came to be viewed as an entitlement. Though states always retained some program control--to set welfare guarantees, for example--control over policy development clearly had shifted to Washington by the early 1970s. Full federal control almost became a reality in the form of President Nixon's Family Assistance Plan, which would have instituted a federally guaranteed income floor under all families.⁽⁵⁾

The decision concerning where to locate program control did not remain settled, however, as Congress gradually gave states more flexibility to seek waivers of AFDC provisions for the purpose of testing welfare reforms. The Omnibus Budget Reconciliation Act of 1981 offered states additional flexibility in designing programs to move recipients from welfare to work. In 1986, the Reagan administration began explicitly to encourage states to pursue waiver experimentation.

The increasing use of waivers began a new process of devolution of program authority over welfare from Washington back to the states. By the 1990s, waivers were no longer restricted to programs with the prospect of offering new knowledge that could contribute to national policy deliberations. Waivers had instead become vehicles for accommodating state preferences concerning policy goals and program strategies. The transformation of purpose unleashed a torrent of state activity, directed not just at work-related policies and programs but also at other aspects of recipient lives. Many new waivers tried to influence personal decisions about marriage and cohabitation, family stability (for example, divorce and other family composition changes over time), and fertility, and some waivers tried to address the quality of parenting. The thrust of reform activity was thus increasingly directed toward a new strain of social engineering: employing welfare innovations as strategies for influencing a broad range of behaviors society deems important.

Why the obsession with reform?⁽⁶⁾ One reason may be concerns about costs. Federal outlays for poor children and families were estimated at about $90 billion in FY 1994. Of that, however, only $14 billion was spent on AFDC, less than 1 percent of the federal budget and about the same amount that was spent some two decades earlier, in inflation-adjusted terms. While recent congressional reform packages were projected to reduce the federal deficit over the next seven years by at least $40 billion, most of those savings derive from changes in the Food Stamp program and reduction in benefits to legal aliens. AFDC reform is likely to have trivial impacts on the federal budget over the next five years.⁽⁷⁾

Another plausible reason lies in rising AFDC caseloads and concerns about deleterious effects of public dependency. The national AFDC caseload increased by about one-third between 1989 and 1994, when the number of families receiving benefits surpassed 5 million for the first time. (However, proportionately fewer poor children received benefits in the early 1990s than they did two decades earlier. AFDC participation rates for female-headed families with children actually fell from 60 to 49 percent between 1977 and 1992; the rate for blacks fell more dramatically, from 78 to 51 percent.) AFDC caseloads began to drop sharply in 1994 (from over 5 million cases in mid-1994 to 4.87 million cases in 1995, according to preliminary estimates), but national interest in welfare reform did not abate.

Arguably, a more important motivation for reform may be the perception that welfare, especially AFDC, promoted irresponsible, self-destructive behaviors. Welfare benefits limited to those with income below a particular threshold inevitably create troublesome incentives. By design, they provide benefits to those who do not work and then (as the programs must, if they are to serve only low-income people) reduce benefits drastically as earnings rise. This inevitable feature of welfare results in marginal tax rates, "notch" effects (a point at which one dollar of increased earnings results in the loss of all welfare income), and penalties for savings that society would never impose on other income classes.

AFDC in particular targeted benefits on vulnerable groups that society generally wishes to protect--such as single-parent families and children. While sensible from a resource-allocation perspective, the targeting has unintended consequences: at least theoretically, AFDC may discourage marriage, because benefits are more easily obtained by one-parent families. It may encourage a teenager to have a child to qualify for benefits or a woman to have additional children to increase her income. It may create incentives for absent parents to avoid their responsibilities, particularly if they assume that AFDC will take care of the children with whom they do not reside. When absent parents do contribute, all but a modest amount of the contribution goes to offset AFDC costs and not to the support of the children.

So why are we shifting responsibility to the states? National policy makers have increasingly despaired of national solutions to these problems of welfare. A transfer of virtually full control to the states in the form of block grants apparently offers an attractive alternative.

Are the new reforms more of the same or something really different?The evidence suggests that the new reforms are different, both in character and, perhaps more important, in the ends they seek to achieve. As states became more comfortable with the exercise of their waiver authorities, the demonstrations they developed became increasingly bold. States used waivers for multiple ends (addressing income poverty, labor supply, family formation and stability, fertility decisions, and parenting skills); multiple target populations (the resident adult caretaker, the nonresident parent, the children and sometimes grandchildren, the public and community-based institutions that assist the poor, and the communities within which the poor predominantly reside); and multiple strategies (from reforms that improve the opportunities faced by recipients, to reforms that impose more obligations, to reforms that replace welfare altogether).

Perhaps the most important trend evidenced in recent state demonstration activity is a shift in program objectives: the demonstrations increasingly stress changes designed to alter personal and interpersonal behaviors. The theme of welfare reform in many states has been to concentrate on helping (or obligating) people to play by society's rules: get a job, get married, make responsible fertility decisions, be a good parent, and obey the law. This change in direction occurred only recently. As late as 1992, only a couple of states has shown interest in the family cap concept, under which benefits remain constant when a mother has an additional child while on AFDC. By the summer of 1996, about 40 percent of all states had actual or proposed family cap provisions and 18 states had (or had proposed) provisions requiring minor parents on assistance to live with their parents or in a supervised setting as a condition of eligibility. Also by 1996, about two-thirds of all states had (or had proposed) reforms explicitly linking welfare benefits with good behaviors, such as regular school attendance and maintenance of up-to-date immunizations. Social engineering, in the sense of the use of AFDC policy to inform and shape a variety of behaviors, had become the rage.

Is the Welfare Debate Over, or Is It Just Getting Started?

The outcomes of devolution are not transparent. Despite widespread (and often justified) dislike for welfare programs, they have endured because changing welfare raises very difficult policy choices. Every dialogue on welfare reform confronts two policy challenges--how to alleviate poverty, particularly among children, and how to minimize welfare dependency among their adult caretakers. This dilemma makes the debate hard to resolve. Society is concerned about the condition of poor children, to whom no blame is assigned for their plight. At the same time, society has mixed feelings about the parents, to whom some responsibility for the family's economic situation is assigned.

Welfare systems like AFDC endured despite widespread criticism because, in part, policy makers had multiple ends in mind. If the issue were only reducing welfare dependency, the policy response would be obvious to all: simply end welfare. Generally, however, society is concerned about the degree of economic insecurity experienced by some children. As AFDC guarantees (the amount of benefits that a family with no other income would receive) have steadily declined, the number of poor children has climbed to levels not seen since the War on Poverty was launched thirty years ago. If reformers were concerned only with reducing dependency, then cutting benefits would be a simple policy solution. But when child poverty becomes part of the equation, the policy challenge becomes more daunting.

The real reform challenge is to reduce dependency and poverty at the same time, although past and current reformers have typically focused on one or the other of these goals. Repeated contests between those who worry about poverty and those who worry about dependency have led to frequent shifts in the basic goals of welfare reform. The welfare debate has wallowed in confusion partly because the underlying question of interest keeps shifting among the following areas of focus:

• Child well-being. The major concern of the architects of what became AFDC was the well-being of children. In the early days of the federal program (mid-1930s through the early 1960s), and during the decades when state-run Mothers' Pension programs prevailed (1910s through the early 1930s), the well-being of children was the focal concern. Earlier, poor and other "at-risk" children were routinely taken from homes and placed in institutions or in other, more "appropriate" environments. It was now thought that deserving mothers should be supported in their caretaking role and that this would better serve the interests of the child. A great deal of emphasis was placed on assuring that the home was "deserving" and the behavior of the adult caretaker was rigorously monitored.
• Income deficits/poverty. In the 1960s, the concern shifted to income poverty. Economists replaced social workers as the definers of the policy issues. Being poor became somewhat more closely associated with income deficits. After a brief dalliance with the human capital and community empowerment strategies of the War on Poverty, solutions focused more on "incomes strategies," in which the financial shortfall was directly addressed. The Negative Income Tax and welfare as an entitlement were direct expressions of this.
• Labor market participation/private earnings. Beginning in the 1960s and growing in importance over the next three decades, the labor market focus emerged. Mothers increasingly were expected both to care for the children and to work in the labor force. Nonwelfare mothers increasingly were juggling this dual role, and motherhood as a full-time avocation lost support. By the 1980s, policy innovation, program evaluations, and basic research had accepted this welfare-to-work theme as the core direction of welfare policy.
• Child support strategies/absent parent involvement. In the early 1970s, welfare officials realized other parents were not doing their share. The child support strategy initially focused on improving the absent parent's financial involvement with their children. More recently, the objective has been broadened to include time and energy as well as money.
• Fertility/teen births/nonmarital births. As researchers and policy makers slowly transitioned from a poverty and income support focus to a labor market emphasis, other policy concerns emerged. Changing demographics drove many to concentrate on a dramatic transformation in the profile of the American family, particularly those at the low end of the income distribution. The poor were disproportionately found among single-parent families, many of which were formed when the mother was a teen. It was not lost on observers that the cost of remedying the problems associated particularly by nonmarried teens was substantial. Slowly, fertility became a focus of concern.
• Family formation/stability. The ever-changing process of redefining what is important continued. Concern about the integrity of the family certainly can be traced to the creation of the AFDC Unemployed Parent program in the early 1960s and the strengthening of child support enforcement provisions in the early 1970s. Still, preserving and enhancing the family came to center stage by the late 1980s. Increasingly, observers talked about bringing the father more fully into the family in ways beyond ensuring economic contributions.
• Child well-being, again. We are, perhaps, in the 1990s returning to the beginning. Discussions in 1994 assumed forms similar to the "child-saving" debates of earlier times, including discussions of orphanages. Although the discussion of orphanages was brief, the focus on children has been more enduring, emphasizing questions of how to ensure that all children grow up to be productive members of society. The demands of full participation in society are increasing, and this generation of kids must be better prepared than any previous cohort. Success will be measured in downstream benefits, failure in downstream costs.

We do not know what the next generation of reforms will look like with any certainty. The multiple goals embedded in the reform debate will not be easily achieved. Achieving many of them will require a different way of thinking about the organization and delivery of services; as well as innovative ways of crafting new participant experiences through radical institutional change. If policy and program trends evident under waiver policy are any guide, we might speculate that we will see the following in state-designed responses to devolution:

• The new welfare systems will be behavior-oriented rather than income-oriented. Among the new generation of reforms, there is a marked, though not total, shift away from income support goals to behavioral change objectives. Dominant treatment modalities will shift from income support strategies to service technologies, broadly defined to extend beyond traditional social work services but also including case management, crisis management, and counseling activities resembling social work functions that marked the provision of public assistance some three decades ago.⁽⁸⁾
• The new generation of policy/program interventions will be complex and introduce numerous changes simultaneously, including in some cases increased benefits to people not previously eligible for public assistance. ⁽⁹⁾ The new interventions will also be characterized by increased policy volatility (continuous change across time) and variability (significant differences across jurisdictions or agencies within a state).
• The new welfare systems will be dynamic rather than static. They will be oriented toward change. They will take a family at a baseline status and actively work toward changing behavior and attitudes. The welfare program in Wisconsin (Wisconsin Works, or W-2), for example, explicitly talks about participants ascending the multiple tiers built into the system. The assumption is that participants will progress or move up the tiers before "graduating" into the labor market and mainstream society.
• The conceptual and temporal bases for thinking about what is going on will be dramatically altered. Traditional public assistance programs employ monthly accounting units, each month representing a new temporal period of interest. The new generation of reforms tend to be change-oriented: the participants are expected to evidence different behaviors as a function of their experience in the system. That is, the conceptual basis for looking at a case is transformed from a point-in-time perspective to apoint-in-process or longitudinal perspective.⁽¹⁰⁾ The status of the participant at any given juncture is interdependent with prior experience in the program and future expectations. All participants will be subject to time-limits, both within program components and in an overall sense. The worker, or system, must remain sensitive to where they are relative to these temporal constraints.
• The new reforms intend to affect a more diverse set of actors, and thus have multiple units of analysis. They address several target populations: adult caretakers in participant families (the usual suspects); children, and noncustodial parents. Some reforms are also designed to change the attitudes and behaviors of those operating the program (agency culture change) and of selected populations in the community (e.g., employers, child care providers, etc.).
• The new welfare systems will be multi-dimensional rather than uni-dimensional. An extension of the principle of individualization is the notion that all participants will not proceed through the welfare experience in a lock-step fashion. They are likely to be tracked along different paths. Differential tracking suggests that important decision points exist where key choices are made--the assignment of participants along distinct program trajectories that implicitly recognize diversity within the welfare population.
• The new welfare systems will be characterized by complex decisions that will require a good deal of professionalism and involve much discretion. The new welfare regimes will be craft-oriented rather than routinized. Some of the decisions under the old welfare system were fairly complex, but the basic intent of the system was to treat all participants the same. This was a practice of rough justice--treating all alike. The new generation of reforms are designed to treat participating families as individuals, or in individualized ways. Many involve the negotiation of individualized "social contracts" or "individualized employment plans."
• The new welfare systems will be labor intensive. The old welfare involved repetitive, routine decision making. The emphasis was on efficiency and accuracy. Participants who wanted help were referred to other systems. Not surprisingly, administrative costs often equaled less than 10 percent of benefits issued. W-2 and similar reforms will require intensive case management and a very active rather than passive participant-worker interaction.

The old welfare was thus routinized, repetitive, limited, static, and unidimensional. The new welfare is dynamic, complex, pro-active, change oriented, longitudinal, and labor intensive. This paradigm shift inevitably is accompanied by differences in management and oversight. Standardized, routinized rules and procedures lend themselves to stable policy environments and vertical, hierarchical management structures. The new welfare policy environments are neither very stable, nor do they lend themselves to control from the top. Policy and program strategies are volatile, and tend to be continuously in flux. States will have more difficulty in centrally managing and prescribing agency operations. Consequently, intra-state variation may begin to rival inter-state variation.

States face hard challenges in institutionalizing the new public assistance. They must grapple with the structural flaws of welfare that have bedeviled national experts for decades. They must reform a system in which the expectations of what reform will accomplish have multiplied over the years. And they must do this with a federal contribution that will shrink in value over time. In this context, what can the academic and evaluation communities do to inform the future welfare debate?

The Academic and Evaluation Communities: Can They Contribute to the Future Debate?

Under the new legislation states theoretically will have greater flexibility in designing and managing their support programs for poor families with children.⁽¹¹⁾ Greater flexibility brings increased responsibility and, given the nature of the new legislation, additional risk, because under the new welfare bill the federal contribution is fixed and will certainly decline in value over time.⁽¹²⁾ This suggests that states and local governments will bear the full fiscal burden of policy decisions at the margin, after the federal contribution is exhausted. Thus, states (or whatever level of government program responsibility ultimately is vested in) now will have a greater stake in obtaining a priori knowledge about the likely consequences of their policy decisions. If policy makers assume certain behavioral responses to a reform and guess wrong, they could incur substantial costs or be required to ration services and benefits as budgets come under increased scrutiny.

In this environment of change and challenge, program evaluations and other forms of analyses become critically important. Knowledge is more critical in a devolved policy environment, since the decision making vested in local governments is both more complex and more consequential. But though the value of knowledge is greater, the price of knowledge also has increased. The federal government will neither mandate that evaluations be done, nor ensure that certain methodological standards be maintained.⁽¹³⁾ Many jurisdictions will hope to be free riders, letting others incur the fiscal and other costs of doing evaluations while still taking advantage of the results.

Even if costs were not a problem, evaluations of the next generation of reforms will be challenging because of their character. Many state welfare reforms emphasize varying combinations of what Lawrence Mead has termed "new paternalism" programs. These programs--which often require school attendance, work, or a daily activity that leads to work--are bundled into varying packages designed to discourage dependency on public assistance, to influence family-formation decisions, and to promote work. State officials implementing these packages expect their reform strategy to yield greater impacts than would be assumed from the sum of the impact of each individual reform, and they worry that separate evaluations of program components may understate overall impacts. They also fear that critical social messages embedded within "new paternalism" initiatives will be less effectively communicated in an environment of random assignment of individuals to varying treatment, compared to reforms that are universally implemented.

In addition, the next generation of reforms are designed to change behavior rather than provide income support, and thus require a whole new relationship between agency and client. They are complex, subject to continuing change, and often call for radical changes in agency culture which, among other things, decentralize decision making, rendering management control more problematic. Though classic experimental methods unquestionably remain the most powerful tool for identifying causal associations, they may not always be feasible or warranted in this emerging policy milieu.

In short, the conventional approach to evaluation has been to change one or two parameters of the existing welfare program; randomly assign participating families into either an experimental or a control group; and examine "net" outcomes on a limited number of measures that virtually everyone agreed was important. None of that holds any longer. The changes are broad in scope and complex in character; the target groups of interest are numerous; the outcomes of interest are even more numerous. Policy designers want to radically alter agency cultures and communicate a whole new set of messages to low-income communities. If anything, random assignment would mute the message and make changing agency culture more difficult. In effect, it is a new ball game.

These and related challenges to the evaluation community and to the methods that have come to dominate program evaluative strategies have often been described.⁽¹⁴⁾ A responsive strategy remains elusive, however. Over the past year, interest in developing a coherent response to what might be called the evaluation challenge has emerged and become more pressing.

In February 1996, the National Center for Children in Poverty (NCCP) and the Institute for Research on Poverty (IRP) brought together representatives of the academic evaluation community, top evaluation firms, policy research organizations, foundations, and various federal agencies to discuss the challenges posed by the anticipated "devolution revolution." The results of that session were reported in Focus (newsletter of the IRP), Vol. 18, no. 1. A concern reflected throughout the conference was that little might be learned from the anticipated explosion of innovation and investigation anticipated in a devolved policy world. It was feared that, absent a strong federal role, program evaluations would be too idiosyncratic, underfunded, poorly designed and managed, or politicized to create a stock of theoretical and practical knowledge essential to a devolved policy world.⁽¹⁵⁾

One suggestion was to build the notion of a "Guild," to develop a commonly accepted set of understandings, practices, and methods for how to do acceptable program evaluations.⁽¹⁶⁾ The discussion also touched upon the need for improved mechanisms through which evaluation results might be collected, organized, interpreted, integrated, and disseminated. A follow-up session of major foundations interested in welfare reform and devolution was held in New York in March 1996. Again, the idea of a guild and related strategies were discussed. Further dialogue on these concepts was not sustained, in part because legislated devolution of national welfare reform faced an uncertain future.

In November 1996, the Institute for Research on Poverty held a working conference using the Wisconsin welfare reform proposal as a laboratory for exploring methodological and other challenges associated with evaluating the new reforms. The conference devoted a half day each to three issues: a counterfactual for an evaluation of W-2, potential contents of a process evaluation of W-2, and potential contents of an impact evaluation of W-2. The presentation of papers was interspersed with group dialogues and review and comment by panels of conference participants.

Reactions from participants after the conference indicated that it was a very helpful step, but that more work was needed to:

1. Expand the discussion to the reforms of states beyond Wisconsin.
2. Give more thought to the variables which evaluations should consider--the core issues that should be known about the impacts and processes of a reform and, perhaps, secondary issues that are less critical but merit attention if resources are available.
3. Allow discussions by state program officials responsible for managing welfare reforms. These discussions might include specific variables that they would want evaluations to examine; the constraints they need to address in doing evaluations; and the primary research and management issues that must be addressed.
4. Use academic researchers, policy research organizations, and federal officials to discuss criteria necessary for cross-state comparisons and the potential of federal data-gathering efforts.

Future steps suggested by conference participants focused on broadening the evaluation discussion to encompass a more national perspective, to ensure more participation by evaluation consumers (e.g., state officials), and to give more attention to what is needed to facilitate cross-state comparisons and generalizable knowledge-building.

The academic and evaluation communities are at the crossroads as devolution becomes a reality. There are encouraging signs that states remain interested in learning from their innovations. About 60 percent of the states submitted proposals to the U.S. Department of Health and Human Services to secure funds provided under the welfare bill to continue existing evaluations or initiate new evaluations. At the same time, good intentions are not enough, and the challenges and difficulties faced by states in conducting good evaluations are unprecedented.

States are likely to require much support in answering the following kinds of questions: How does one establish a counterfactual? How are the correct criterion variables selected? How does one agree upon which target groups to examine? What is the appropriate unit of analysis, individual or case or agency or county? How does one go about determining overall and component effects? Should the implementation analysis be used in a formative way, if that increases policy instability and confound the impact analysis? Should local discretion and flexibility be curtailed, so that the character of the intervention might better be understood?

Evaluation concerns can derive from two quite different interests: (1) the broad social and economic problems of concern to policy makers, and (2) the specific goals of current programs and policies. The former set of questions is much broader and more complex, and most of the current demonstration evaluation research, particularly that being funded by the states, addresses only the second concern. But any agenda to inform the welfare debate should cover not only an assessment of the success of the reforms in meeting their central objectives, but also other possible consequences and the mechanisms through which both intended and unintended consequences occurred.

Several generic strategies might be proposed to enhance the roles of research and analysis in future welfare reform discussions:

1. The Guild strategy--replicate and extend the work begun at the February 1996 NCCP/IRP conference and continued, in part, at the November IRP evaluation conference, but with a broader purpose and agenda. No single event will suffice. We intend to initiate a set of activities to develop the following:
   
   • Consensus regarding acceptable methods. There are two main types of evaluation--process and impact. Both are difficult to do and both are important. There are not agreed-upon methods for doing process analyses; consequently, studies permitting cross-site comparisons are not common. Since the classic experimental design may no longer be the sine qua non for doing impact analyses, what alternatives are reasonable and acceptable for establishing causal inferences? Establishing minimal standards may not be a reasonable goal; moving toward such standards may be a necessary objective.
   • Common terms and definitions. Even where everyday terms are used--such as case, or successful outcome, or full employment, or noncompliance--great variation in meaning may occur in a devolved policy world. Without agreement on the meaning of terms, cross-state comparisons will be difficult.
   • Common research questions. All evaluations start with research questions concerning management and theoretical issues about which empirical information is desired. The questions posed inform all other features of the evaluation--choice of outcomes, choice of methods, and so forth. If a consensus (or movement toward a consensus) could be developed around a set of management and research questions, more comparability across evaluations would be likely.
   • Common outcomes of interest. Defining success in a devolved policy world will be a major challenge and a likely point of contention. Those who select the outcomes of interest and choose how to operationalize those outcomes greatly influence the political agenda of reform. Although local preferences should not be ignored, a core group of common outcomes, consistently operationalized, is needed to anchor state and local evaluations and permit analyses across jurisdictions.
   • Common reporting of findings. Several evaluations conducted by the Manpower Demonstration Research Corporation have been quite influential, in part because they reported findings in an understandable manner. More varied evaluations by an increasing number of evaluators are likely in the future, and standards for reporting results would be helpful.
     
     
2. The information and technology diffusion/utilization strategy--the dissemination of information and "institutional" behavior modification. Over time, we would like to inform and shape the way selected individuals and institutions do their business. We are not looking for miracles, just progress. Some target populations and behaviors might include:
   
   • state officials--develop increased appreciation for the necessity and value of evaluation that employ a common core of research questions and outcomes that facilitate cross-site comparisons, use methods that permit the articulation of causal inferences with some confidence, and actively explore the learning possibilities in the creative use of integrated administrative data sets.
   • evaluation firms--move toward common terms, definitions, methods of reporting results; toward common ways of doing process and implementation analyses, again to facilitate comparability and the creation of a stock of knowledge.
   • academics--get more involved in applied policy analyses and program evaluation activities at the state and local level. Develop increased ability to report findings in such a way that allows researchers and their work to become more influential and useful to policy makers.
   • foundations--move toward common standards in making awards, improve the technical knowledge of program officers, increase interest in research and evaluation activities.
   • federal officials--increase technical assistance and continue to play an active, though perhaps less official, role in enhancing the quality of evaluations. Increase the use by federal officials and organizations of neutral, third-party experts to perform review and monitoring functions that can no longer be performed internally.
     
     
   Influencing these target populations involves continuous communication and contact. Dissemination in this context should not be confused with one-way communication of "fact." Rather, we are talking about a semi-coordinated effort to keep the evaluation agenda alive, move toward the common understandings and tools suggested as part of the Guild, and communicating information that may be useful to policy audiences in ways they can use.
   
   Each of the steering-group institutions is connected with the target-group members in some way. Each of these organizations has, or is developing, venues for communicating with relevant parts of the policy and evaluation worlds.⁽¹⁷⁾ We intend to use the various communication vehicles that each possesses to increase and maintain this contact. The premise is that continuous dialogue and attention through a variety of communication outlets will influence the evaluation culture and move to a functioning guild concept.
   
   
3. The "expert" review panel strategy--evaluations of reform initiatives will inevitably vary in quality and utility, despite efforts toward common standards. Consequently, we propose to establish a pool of expert reviewers to examine more important evaluations for the following purposes:

   a. To isolate which results and findings from an evaluation might be accepted with some degree of confidence. This would be an effort to sort out credible findings from those that are questionable owing to methods or management.

   b. To identify and relate credible findings across diverse evaluations in order to build up a body of knowledge based on common findings in diverse program and situational contexts.

   c. To identify findings that seem contradictory, and thus may constitute research and management issues that require further study. This is an attempt to identify the cutting-edge issues requiring analytic attention.

   d. To articulate those issues that require additional study, and possibly suggest ways they might be examined. That is, the panel should advance the art of issue identification and resolution.

   The composition and sponsorship of the panel is critical. Academics drawn from affiliates of the two national poverty centers, IRP and the Joint Center for Poverty Research (Northwestern University and the University of Chicago) would be likely participants. But methodological input is only one dimension of interest. We also want the consumer's perspective, and perhaps even input from the most common producers of program evaluations--the top evaluation firms. Thus, we potentially see panels drawn from academics, state evaluation and policy units, evaluation firms, and the federal government. Mechanisms for ensuring cross-panel communication would be developed.
   
   There may be some urgency to moving on this particular strategy. Many evaluations of waiver-based reforms will be issued over the next 12 months. A critical review of these evaluations would both enhance their utility and help in the process of improving the next round of assessments.
   
   
4. The state peer-assistance network strategy--expand and replicate the use of regional state networks to facilitate communication among welfare officials representing contiguous states. It is assumed that, under devolution, peer consultation will become the best way to identify common problems and the best source for developing solutions. A prototype for such a network is being developed among upper Midwest states,⁽¹⁸⁾ called the Midwest Welfare Peer Assistance Network (WELPAN). Welfare officials from Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, and Wisconsin have been meeting since October 1996 to share problems and search for common solutions.

The premise of WELPAN is that, under devolution, horizontal communications (across states) will increasingly replace vertical communication (from the national level to the local level). The WELPAN group chose, as its first substantive topic, the issue of defining success in welfare reform. The next topic will involve sharing insights about how to effectively alter the institutional culture of welfare agencies. To date, the meetings have been a combination of general information and problem sharing, coupled with in-depth discussions of selected topics. Participating members control the agenda and determine who may attend the meetings. The Family Impact Seminar and the Institute for Research on Poverty provide staff support for the sessions.

Experimenting with some variant of WELPAN in the Midwest or another region promises a means for creating an environment through which common evaluation strategies might develop and through which new knowledge can effectively be disseminated. The participating members in the group are upper-level management and not individuals with particular expertise in evaluation. Experimentation with a parallel network of researchers, evaluation experts, and policy makers drawn from participating states may also prove necessary. The substantive agenda would focus on the following: how best to develop and manage effective evaluation strategies in the participating states; how best to interpret findings from such studies; and how to assess the significance and utility of findings from studies and evaluations carried out in nonmember states. Member states might also explore the possibility of cross-state evaluations or combining resources to monitor performance and outcomes.

Summary

Although welfare reform has passed at the national level, it is unlikely that the debate about welfare is over. The problems are too intractable, the challenges too awesome. The debate, however, is now distributed more broadly, lacking a central audience and stage. In response, we have begun to think through a set of strategies for continuing the work of informing the welfare debate. The papers in this special report set the stage. The generic strategies laid out at the conclusion of this paper indicate some possible future initiatives. Clearly the work has just begun.

Endnotes