José Pacas on the Puzzle of Measuring Rural Poverty in the Supplemental Poverty Measure

Dave Chancellor [00:00:01] Hello, and thanks for joining us for the Poverty Research and Policy podcast from the Institute for Research on Poverty at the University of Wisconsin–Madison. I’m Dave Chancellor and for this episode I talked with José Pacas about data challenges of measuring rural poverty, especially in the Supplemental Poverty Measure. Dr Pacas was a 2019–2020 fellow in the IRP Scholars in Residence program, and he’s currently chief of data science and research at Kids First Chicago and previously worked as a research scientist at IPUMs at the University of Minnesota. I learned a ton from him in this interview, and I really hope you enjoy the conversation. So, Dr. Pacas, thank you so much for being here, and you’re currently serving on the National Academy of Sciences panel on evaluation and improvements to the Supplemental Poverty Measure or SPM. And one of the reasons you became connected with this panel is, as I understand it, is because of a couple of papers that you coauthored that look at how the spam handles rural poverty. So can you just kind of start us off? Can you tell us a little bit about this panel that you’re on and some of the work that you’ve been doing?

José Pacas [00:01:12] Well, thank you so much for having me, Dave. It’s a pleasure to join you here from my dark basement in Minnesota. Now regarding the NAS panel. Well, the National Academy of Science, Sciences, Engineering and Medicine. Their goal is to provide objective advice to inform policy with evidence, et cetera. And this particular panel is looking to assess and evaluate the supplemental poverty measure, which I think we’re going to talk about a lot more in depth. But it’s kind of an interesting thing in that the supplemental poverty measure, as I’ll explain more later, that takes root in the NAS panel that they assembled in 1995, they brought that together to talk about improvements to the way we measure poverty in the U.S., a lot of the deficiencies that we have with the official poverty measure. So they set that up in 1995, and Connie Citro and Robert Michaels led that panel, and they basically established what was the framework for what is now the supplemental poverty measure. And importantly, they said, we need to reassess the Supplemental Poverty measure, and they suggest that I think every 10 years and while it’s been about 25 years since we’ve done that. And so this is a panel that’s now looking at how the supplemental poverty measure is working and what are the improvements that we can make to that measure. And the big part, it’s to look at how we can accurately give a picture to how our policies are having an effect on poverty. So we’ve assembled a big group. And one of the really exciting parts about that is that Connie Citro’s the National Academy science liaison to this group. So you have this really important person in the room guiding us through this. But you also have a bunch of people, affiliates, IRP affiliates on this panel. So our task is to look at the suffering, the poverty measure and see how we can, well, evaluate. I don’t want to say necessarily improve everything because some things are working great other parts we might need to improve a little more. And ultimately, the Census Bureau will decide what sort of improvements they can actually implement from this panel.

Dave Chancellor [00:03:21] So you’ve been talking about some of this history of the development of the Supplemental Poverty measure, and I think a lot of people on here that are listening to the podcast are probably at least somewhat familiar with the SPM and maybe some of the differences it has with the official poverty measure. But it seems like this might be a good kind of starting off place for thinking about how these measures developed and in the case of the Supplemental Poverty Measure, might continue to see refinement.

José Pacas [00:03:46] The way I start this, this conversation is typically thinking about how would you if you were given the task of measuring poverty in the 1960s, how would you set that up? And a lot of that depends on the data that you have available and Mollie Orshansky basically had the brilliant methodology of saying, Well, we need to figure out what it means to meet basic needs. And so she basically figured out that we know that about a third back in the 1960s, a third of a family’s budget was spent on food, and we could figure out a thrifty food plan and we could say, Well, we know how much that costs, so we multiply that by three. Then we know how much your family needs for meeting their basic needs. And then they adjusted by family size, and back then they did farm versus non-farm. That’s something they don’t do anymore. But then they said, that’s going to be our poverty threshold. So if a family has enough income to meet that, then they’re not in poverty or below that they’re in poverty. And the other data limitation, which is the lens that I typically bring, is like, what can we actually measure? And this comes from my background. I used to work at the Census Bureau, the Poverty Statistics branch. And so the current population survey was all they really had or the best data available that was nationally representative that had data on income. And so when you get a family’s pretax income, which is how the official poverty measure counts income, they then compare that to a threshold that corresponds to that family structure. And they say above that line, you’re not in poverty. Now that’s how we’ve measured it for the past 60 years now. We’ve all known that that may not properly or accurately reflect every aspect of what it means to be in poverty, which is in itself a very difficult. Concept to answer. But when we try to bring that to data to give you an idea as to where there might be pockets of poverty, the official poverty measure is rather it tells us a lot, but it doesn’t tell us a lot of things about what is happening with government taxes and transfers, right? Which is our main the main way the government can give back to or to help poor families. So given those limitations of supplemental poverty, measure improves on the official poverty measure across several dimensions, and there is many.

Dave Chancellor [00:06:12] Can you walk us through some of those?

José Pacas [00:06:14] So the main way to think about it is one let’s define who shares resources. So the official poverty measure takes the resource sharing unit as a family, as people living together related by birth, marriage or adoption. But then the SPM adds coresident unrelated children, foster children and unmarried partners and relatives, which is in essence, are cohabiting partners. Unmarried people living in the same household. That’s a major limitation of the official poverty measure. And so we consider those people as a resource sharing unit. The next part is basically saying, what is that basic that that threshold, what is the need, the basic need for a family? And instead of saying, you know, taking the Mollie Orshansky method, we say, Well, let’s look at what people have actually been spending on. And so they we measure food, clothing, shelter, utilities. Lately, they’ve been including internet and telephone expenditures. And they get the median of that across the entire country using the consumer expenditure survey. And then they take a percentage of that and say that’s going to be our poverty threshold is it’s kind of this quasi relative threshold. And finally, well, this is more related to the rural poverty question is we have these geographic adjustments, right? The official poverty measure says whatever you need in rural Idaho is the same thing you need in New York City, which obviously does not fit the. Just from a practical standpoint and what just makes sense to people that just doesn’t seem to fit the bill for how we should measure our needs across places. So instead of doing that, they take these thresholds and they adjust them by housing tenure, whether you own with or without a mortgage or whether you rent, because the spending patterns are different across those. And then finally, they adjusted by the median rent of a two bedroom household. So. That’s how they adjust your graphically. And that’s where a lot of my research has focused. And finally, there’s also like what should we include in cash income? That’s the other huge free tax. That’s a very clear thing that people think of as like, well, you still got to pay taxes or in many cases, you receive credits, earned income tax credits, child tax credits. Those things are not included in the official poverty measure and non-cash benefits aren’t included. So those are included in the scheme. And that also subtracts out childcare work expenses. And one of the biggest ones is medical out of pocket expenses. So those are the three main ways. And overall, once you do that right, the SPM is typically higher by about half a percentage point. So the Supplemental Poverty gives us a much more complete picture of what our government policies are doing to alleviate poverty. And one of the best ways to think about this, or one of the best examples, is this past year. So for the past, since we’ve been producing or publishing the SVM since 2011, we’ve seen that spam rates are about 0.5 half a percentage point higher than the OPM. And that’s been true until last year with COVID in the COVID stimulus package. Where we saw these really big headlines were that the spam was finally lower than the official poverty measure. So the official poverty measure is about 11 and a half percentage points. But the spam was around nine percent, so 11, 11 percent for OPM and nine percent for SPM. And all that reflected was the government benefits. So we saw that the help that we were giving people without the spam, we would not know that right. So that’s one of the strengths of the spam.

Dave Chancellor [00:10:10] So I want to go back to thinking about geography in the context of rural poverty. And it seems like so often the first question is, well, you know, how do you define rural?

José Pacas [00:10:20] Well, they’re really in the paper. We talk about three different dimensions or two different dimensions, and neither the non-metro metro definition or the urban rural definition really line up with one another. But I think what you’re getting at is this metro non-metro question. Right? So in the current population survey, when we look at rural urban and you can see this, what is it at a glance? There’s a publication every year, the USDA rural something at a glance.

Dave Chancellor [00:10:48] I think you’re right. I think it’s just rural poverty at a glance.

José Pacas [00:10:51] They like one of their first pages. They say we use rural as synonymous or interchangeably with non-metro, and that is what we have in the current population survey. Is this metro non metro delineation. It depends how we define rural. Right. So let me let me step back in the American Community Survey, for example, where we can get some geographic areas that are more well defined. If you were to just look at metro non-metro differences, you’d see a higher poverty rate for non-metro. But if you jump and get into urban rural definition, which is a much more closer to the USDA rural continuing definition, you see that rural poverty rates are lower than urban poverty rates. And this is using the official poverty measure, right? So already baked into poverty and how we think about rural versus non-metro, we already have this weird interplay of, well, what does it mean to be rural? What does it mean to be non-metro either to the same? And are we getting the same picture? The answer’s no. Just based on the definitions, when we jump over to the OPM and the SPM, we’re getting into murkier spaces because now we’re limited to the current population survey, which only has a metro non-metro delineation. And right there is where we start to get into issues of how does the supplemental poverty adjust for these geographic differences in median rent? And it’s actually this interplay between the operationalization of that adjustment and the hits head on with the data limitations and privacy issues that are baked into the current population survey. I mean, this is also a paper that my colleague at IPUMs, Jonathan Schroeder, he’s a geographer. We were trying to see how we think about defining rural life based on concentration, population size, whatever it is. So we were trying to figure out how to provide data in the American Community Survey that gave people more granular, a better way of understanding and defining rurality, which we’re still not sure is technically a word. But we’ve used it a lot, and that’s how we pose that question. It’s like, well, aren’t real spaces more poor than urban? And that’s where we ended up realizing, well, it actually really depends how you define it. And then actually, even more importantly, depends on what’s available in the data.

Dave Chancellor [00:13:29] So in your paper with David Rothwell, the two of you kind of set it up by saying that, you know, conventionally, for most of the time that poverty has been measured in United States, we have thought of rates of world poverty as being higher than urban rates. But for the most part, that’s not what we see under the SPM so that you sort of set out to see what was going on. And because it seems like there’s you know, as you say, there’s potentially a different narrative here, depending on how you measure it. It’s all right.

José Pacas [00:13:56] Like you said, it depends on how we measure it. But under the official poverty measure, if we just take that back to the 1960s, we’ve seen that rural poverty. Let’s, let’s say the non-metro poverty has been higher than the metro. And one of the difficulties in figuring out, well, if the SPF is an improvement on the OPM in terms of reflecting the reality of low income populations, then we need to figure out what that look like going back in time. And so that’s where the Columbia Group did this work to create the historical supplemental poverty measure. And what you see with that is that certainly rural poverty non-metro poverty was much higher than urban poverty back into the 1960s. But at some point there is this giant inflection point where that doesn’t become true anymore. So if you were to limit your view to when we have the official supplemental poverty measure, which is 2011 on, it’s always been true that supplemental poverty has been lower in rural places than in urban spaces. But that’s hasn’t that wasn’t historically true. And what we’re gaining with this is as you look at historical data, we get these different views of poverty and really when we think about the supplemental. Poverty measure, what you’re gaining is this view of like how has a social safety net been helping people in urban versus rural areas? So it’s always been conflicting how these two are working on an important part of this historical understanding of rural poverty as defined by the Supplemental Poverty Measure is a Nolan et al. paper. I think it’s 2017, and they point out there that it’s the geographic adjustments that play a large role in that pattern. Right. And that’s a that’s a key thing in the research that we’re doing with Dave Rothwell because we met at a conference for RUPRI. So he and I met at the Rural Policy Research Institute, and he was presenting a paper with, I think, Bryan Thiede. And what I was looking at there was the persistence of poverty across two years. So that’s a whole different set of research. But what I was finding was under the Supplemental Poverty Measure, families who stayed in poverty two years in a row was that percentage was lower than in urban areas, and that was surprising to many people. In the audience, I think it was Daniel Lichter who said that’s a function of the geographic adjustments. And I think Jim Ziliak, who’s leading up the next panel, the autumn period, probably the geographic adjustments. And so later on, David, they started talking about this and we said, Is that true? Is this a statistical artifact or is this actually representing some sort of reality of poverty in rural spaces? And that’s the core question that we ask is did we mess up the implementation of the geographical adjustments in the way that’s giving us this weird view or a different view or a, let’s say, incorrect view of rural poverty?

Dave Chancellor [00:17:01] So I want to talk about those geographic adjustments because in your paper with Rothwell, you cover three main areas where the SPM may be a concern for the way it treats rural poverty and makes those adjustments, and I’m just going to lay them out here. So first, you say it focuses on housing costs but doesn’t account for things like higher transportation cost and other things that might be different between rural and urban areas. Second, there may be differences in family economic lives associated with living in rural versus urban places. And then third, there’s probably a big one. It treats all areas in a state the same. So, you know, we’re here in Wisconsin, even though costs of living in our state are fairly modest even here. There are really large differences in housing costs if, say, you’re close to Madison versus, say, in northern or central Wisconsin, so. So these are challenges, but what do you do about them?

José Pacas [00:17:52] Yeah, yeah. For the most part, it’s I want to bring that back to the data availability. That’s kind of where my mindset always focused. And it’s that when you think about how can we accurately adjust for differences in cost of living in different areas and say, different geographic regions? One of the limiting or binding constraints is basically, well, what geographic areas do we have available in the data anyway? And so when you couple that with the American Community Survey, the consumer expenditure survey of the current population survey, or even data from the BLS, right, or BEA regional disparities, really, even if we have that available, the question is can we put that into the CPS and can the geographies match up to the CPS geographies? The answer for the most pressing no, because the current population survey has such limited geography, relatively speaking to other datasets. Some people might argue that there’s actually a lot more geography, given that it’s a larger sample size, but it’s still for the purposes of rural poverty. Are rural spaces are sparsely populated, and so it’s hard to draw a circle around a group of people large enough to get past that privacy standard. Right. So. When you do that, you say what the American Community Survey has got some data that’s got more geographic refinement. And one of the more reliable cost of living adjustments is based on housing or rent. And so certainly it’s not going to fit everything right. It’s not going to be the exact way of doing it. But the 1995 NAS panel basically said this is one of the best ways to do it right. And and without getting into too many of the conversations that have led up to the NAS panel, it’s always been one of those frustrating inversions that occurs in the data when you adjust geographically. So it’s certainly not complete, but it is a good way of doing. Adjusting it somehow is better than not adjusting. And so it’s it actually picks up that question that you’re bringing up is another paper that we just had published with Tom Mueller and Matthew Brooks, the myself on. We answered. We tried to look at that exact question. We said, is housing costs substantially different across urban and rural areas? We find that they are. And they’re actually, I mean, without getting into the nuances that that’s actually a pretty good way of adjusting for cost of living. But then we ask the next question, like, what about transportation costs and the data that we have in the American Community Survey, which is what we’re trying to look at, was how much time do they spend commuting as a proxy for commuting costs? And we didn’t find that much differences across urban, rural or metro and non-metro areas that they’re pretty much on par. But we were lacking, which is a big thing that I think you’re bringing up is this feels like. But getting to work is not. It’s different about the realities of people living in rural areas or rural versus urban areas. When we started looking at the differences across transportation costs, which is in the American Community Survey, you get the transportation time to work. When we looked at those differences across metro, not metro areas, we didn’t find that much of a difference, which kind of is the counterintuitive, right? It doesn’t fit, right? But the people at the travel such distances to get to work in rural areas, but they’re also they don’t have to deal with as much traffic, they don’t have to sit in in traffic all the time. But then there’s a question of access to public transportation, which is not. No matter how we cut it, that we didn’t find much differences. But the big things are the unmeasured parts, right, which are things like access or how far to the nearest hospital, how far to the nearest grocery store, how far to the nearest child care. And maybe people have to double their commuting time, but that’s not included in this particular measure. So overall, the data that we have are showing, well, maybe median rent isn’t the worst way to do this. And by the other measures, we’re not seeing other differences. So that’s under the current way or the best way that we could identify. And transportation costs using the American Community Survey.

Dave Chancellor [00:22:33] So given that background to the idea that maybe rural poverty, as it’s currently measured, is lower than what it really is. How did you go about trying to figure out what might be driving this? And was this consistent across the United States? Or were there particular states or areas where these patterns were more pronounced?

José Pacas [00:22:54] Yeah. So this gets into basically the decomposition that we did in our paper, which was basically in many ways, you know, sometimes I don’t. I want to say this is the editorial part of this is, you know, economists want to get to this really strong causal identification. And this paper really just said, we just want to do something an accounting exercise as to to isolate the exact aspects of this geographic adjustment that’s driving this inversion. So the very first step to all of this is just sort of implementing every single difference that the SPM has to the opium cutting that out instead of implementing that by itself and really identifying that the inversion happens in the geographic adjustment, you can add in change family units, taxes, expenses, benefits, tenure. Nothing is going to invert it. But when you add median rent, that’s when you get the inversion finally. And then what we did is we said, All right, well, how much of that inversion, how can we isolated to different states? And so what we started finding was, let me bring that up. Yeah, here we go. Is it actually depends on saying this, it depends on how many years of data you use. So we use 2015 to 2017 CPS and in that so. So here’s where I’m going to give you this caveat is that for people, I suppose, listening to the podcast and they say, Let me go find this baby that they’re talking about. It’s titled Why is poverty lower in rural America? According to the Supplemental Poverty Measure. But technically, it was published under the title. Was poverty higher in rural America, according to the second the poverty measure, and we discovered that right after we sent it to I believe it’s spring and we couldn’t undo that. So there’s an erratum on that. So for people who know, like people like, the similar question of like, is poverty higher? According to the scheme, people see that title like, that’s not right. And so it’s one of my it’s my first publication and already I’ve got an item on the first location that’s been super sad. But anyway, it’s a funny story. But what we find, right? So when we decompose, what states are really pushing this in version to the overall poverty and poverty rate, we find that six states alone drive that inversion. And that’s Alabama, Georgia, Kentucky, Mississippi, which is all technically, I suppose, in the Appalachia region of depending on how you’re thinking about it. But they have that part of it. And those are the main places that we think about poverty in the Appalachian region. But then we also have North Carolina and Ohio. When we saw North Carolina and Ohio, we started thinking about the geography of these places and how what we really consider it to be, there’s a lot of cities and how close are these counties, these non-metro counties to these cities we started thinking about while the cost of living is going to be completely different in southwestern North Carolina compared to northeastern North Carolina. And yet they all get the same geographic adjustment. And that’s how we jump into the paper with Tom Mueller and Matthew Brooks, which is, well, let’s really start to think about that geographic distribution or variability in median rent across. Really, we use the rural urban continuum codes and we say, let’s look within the state. We actually, I think Tom and his beautiful graph and there that plus every single state in the variability, in median rents in rural areas, and you can see that it’s very different across within the state and across all states. And so that immediately was sort of the bigger, more what’s the bigger principle driving the geographic adjustment is saying, well, it doesn’t make sense that the same geographic adjustment gets applied to these different rural areas. But again, this is where it meets the privacy concerns and the CPS. And I don’t know how much you want me to jump into that, but it’s super important, at least in my view, to understand why we can’t really get beyond that right now.

Dave Chancellor [00:27:08] Yeah. Tell me about that.

José Pacas [00:27:09] So the current population survey writes a survey, at least the annual social and economic supplement, which is where the income, poverty and health insurance data come from every year. It’s a sample size of about 200000 people, and that already includes just over sample that you don’t have relative to the basic monthly current population series. So when you look at that, they suppress a lot of geography. And when you try to match that up to the American Community Survey, which is where the median rent data come from, even when you use five years of American community survey data, we’re still limited by the current population survey geographies, which just I think it’s 45 percent of counties are identified in the current population survey. And we can’t bridge this divide because if we were to bring in more granular data from any source and put it onto the current population survey, at that point, the Census Bureau has to make a decision as to what to release and its public use feels right. Because, for example, let’s take Minnesota and Wisconsin. We technically have Hudson as part of our metropolitan area in Minneapolis, St. Paul, Bloomington, and you can look at the data and you can see you can identify the people from Wisconsin that are living in this metro area. But the way we construct the geographic adjustments is within the state by these michon’s regions. So if I then published in the public use files, a geographic adjustment that is specific to Wisconsin’s region, then at that point we can figure out exactly who we’ve assigned that geographic adjustment to living in Wisconsin, which will introduce some privacy issues. And so if we were to do that with more granular, you just think about, for example, North Dakota. And if we want every rural county in North Dakota to have its own geographic adjustment first, we don’t have every single county in North Dakota identified in the American Community Survey. We only have technically five Pumas in the American Community Survey, and then we can identify the county that Fargo is in.

Dave Chancellor [00:29:18] For anybody that doesn’t know what’s a PUMA here.

José Pacas [00:29:20] Yeah. And the Puma rate, the public use micro data area, which comprises a geography that has 100000 people total population in that area, like in New York City, you can get eight just in Manhattan listeners.

Dave Chancellor [00:29:35] We checked on this right after the interview and it’s actually 10 Pumas in Manhattan. Not eight, there are 10.

José Pacas [00:29:41] But then over in in North Dakota, you have, I think, five total. So you can’t identify individual Pumas. And these are primarily rural counties. So they’re all going to have even if we were to say, let’s use the American Community Survey, let’s use Poumons and let’s do the Pumas that we can identify in North Dakota. There’s not that many, but already none of those are going to be identified in the current population survey. So no matter how we try to triangulate this with public use data, we have this issue of not being able to get granular enough data about rural areas in order to apply these geographic adjustments. So given that we, we still don’t know what the effect would be if we were to apply geographic adjustments that are based off of, for example, a rural county, every single county gets its own geographic adjustment. What would the Supplemental Poverty Rate be with that sort of approach? We don’t know the answer, and partly it’s because does that research has to be done within the Census Bureau or the Federal Statistical Research data centers that will give you access to those data, but you need to get a project to prove and it’s a long, cumbersome process. So we still haven’t been able to identify or answer this exact question, which is the SPM higher or lower in rural areas where we to apply different geographic adjustments to the SPM? We don’t know that answer yet.

Dave Chancellor [00:31:13] OK, so you know, I think you’ve shown us that the subtleties in the data and how you treat it are so much of the story here. But stepping back. How does this all matter what you know? Where do we go next from here?

José Pacas [00:31:26] So what we should strive for in this? This data exercise in defining poverty correctly is what the right answer is. And by that, I mean, imagine there’s a world where we didn’t have to worry about privacy constraints, and we just had data that reflected the true lived experience based on a poverty definition of everyone in the US. What would that rate be in rural areas versus urban areas versus metro versus not metro? If we were not inhibited by the data constraints and we should try to get to as close of an answer as that, even if it means that the Census Bureau might be the only group that knows the true answer, and they can publish a rate, but then deal with how do we get public use data that is reflective of that reality? And that’s a very difficult space to navigate, especially right now. And this is like a paying differential privacy and everything that’s happening at the Census Bureau around that it this is where it all meets. How? It on right in saying researchers need to know the truth, really about poverty, when we need to have an accurate measure of poverty out there and we don’t right now, we’re limited by the data. I think I can’t reveal the conversations that we’ve had on the next panel, but I think it would be very clear, at least in the paper that we wrote with Dave Rothwell saying we got to figure this out. We need to know what the right answer is. But then that also gets into the question of, well, if the SPM is a better measure of poverty than the opium, then what do we do about that? Some of the dicier trickier questions, I think, is should the opium exist? Should that be something that we continue? And I think. Of course, it should be statistical. As a statistical measure, it’s important that we have that trend or that yet that poverty rate continue to exist. But where it gets tricky for me is the relationship between those thresholds and the federal poverty guidelines. And when you couple how they construct the federal poverty guidelines, which is very similar to how we do the official poverty measure, then we get into these trickier questions of, well, if we don’t, the just for cost of living somehow, then rural areas are going to be hurt in the sense that some people aren’t going to meet these thresholds and we need to revisit that. We at least think about how that could have an effect on the lived experiences of low income people in rural areas. So this really subtle thing of poverty thresholds, I think has a lot of real world implications, a lot of policy implications.

Dave Chancellor [00:34:23] Dr Pacas, thank you so much for being here. I really appreciate your time.

José Pacas [00:34:27] Appreciate getting invited and get in a chat about this.

Dave Chancellor [00:34:30] Thank you again to Dr José Pacas for sharing this work with us. You can follow him on Twitter at @JoseDPacas. The production of this podcast was supported in part by funding from the US Department of Health and Human Services Office of the Assistant Secretary for Planning and Evaluation. But its contents don’t necessarily represent the opinions or policies of that office, any other agency of the federal government or the Institute for Research on Poverty. Music for the episodes by Martin de Boer. Thanks for listening.