Saturday, June 30th, 2012

Misreferencing Misoverestimated Population by Chris Briem

[ Chris Briem works at the Program in Urban and Regional Analysis at the University of Pittsburgh’s University Center for Social and Urban Research. He’s one the best I know anywhere at urban data, and his personal blog Nullspace is a fantastic, data wonkish take at Pittsburgh. I happened to see this article he wrote there which deflates a lot of the recent enthusiasm created by the latest census population estimates for municipalities. This stuff was news to me, so I’m grateful Chris allowed me to repost it here – Aaron. ]

I know the media confusion story of the day is all about the momentary misreporting that got the story of the Supreme Court ruling yesterday backwards. Yet there was some real misoverestimating across the nation over the latest census numbers that were released yesterday on municipal population estimates for 2011.

Here are some headlines yesterday: 

LATimes: U.S. population in cities growing faster than in suburbs

Chicago Tribune: Census sees Chicago’s population inching up

Boston Herald: U.S. population in cities growing faster than in suburbs, figures show

AP: Big US cities boom as young adults shun suburbs, census estimates show

Lots more just like those. Guess what… Pretty much all of those stories are wrong, or at the very least baseless when you really look at the data.

The census data reported was the 2011 population estimates for incorporated places across the US. So basically cities, towns, boroughs, and townships. We went through this yesterday, but if one read the actual census methodology for this particular data they were quite clear. The subcounty (i.e. municipal) population estimates are mostly based on an estimate of the change in housing units at the municipal level.  The census changed their methodology on how they computed housing unit change for this particular data and as they explain:

To produce subcounty housing unit estimates, we distributed the extrapolated county estimates down to each subcounty area within a county based on 2010 Census proportions.” (emphasis added)

Which means basically that there was very little 2011 data that went into these numbers. Without using new information it begs the question of how much the results should be interpreted. They basically took the estimated county level population data and allocated it to smaller municipalities based on the 2010 Census. They also just assumed that all the growth was even within counties. That assumption, that center cities grew the same as their immediate suburbs, produced the results being reported on everywhere. There appears to be no other supporting analysis for the assumption, it is just an assumption. Other than that, there is no new information here to lead to the conclusions making their way into the headlines.  It may have even tripped up the experts out there because the Census folks explain they changed their methodology just for this particular data release, and are likely to change it again before next year’s update.   But you have to read into their methodology notes to realize the changes for just this year. This is all probably an example of why some of us have the bad habit of reading footnotes first. 

Was there any new growth in cities? Not at all. Or at least there is no data in any of this to tell us one way or another. The Census basically took the growth that likely continued to be mostly in the suburbs and just assumed it was spread evenly between center cities and suburbs within counties across the nation. The result was that it all of a sudden appeared cities were growing faster (or in some cases shrinking less) than they have been in other data. In reality, the new patterns were no more than an artifact of the temporary change in the Census Bureau’s methodology for this data. If they had ever used the same methodology in the past, namely taking county-wide population changes and distributed growth evenly across municipalities the results would have come out the same.  If these municipal estimates had been calculated this way over the last decade, they would have wound up being very much different from the eventual decennial census enumeration.

So the headlines may be ok if there is data on ‘cities’ that are in themselves counties, but those areas are few; or in the case of New York City, multiple counties.  For most cities are only parts of larger counties. Other than Allegheny County I looked at Cook County which includes Chicago and indeed both the city of Chicago and most all of its Cook County suburbs are being reported as having nearly identical growth rates since 2010. I bet that is no more true there than it isn’t here.

The only caveat to any of that is that the data reported does seem to have some new 2011 data on group quarters population incorporated into it, as their methodology says it should. So where there was a recent change in the population of college dorms, military barracks, prisoners are related types of institutions then you are seeing population changes different from the county-wide averages. That appears to me the main source of the disproportionate growth the 2011 data is showing for the City of Pittsburgh. So real growth for sure, but I would be careful in explaining its causes.

So this all may not be as egregious an error as the news cycle we once had in 2000 when  population ‘growth’ Downtown was attributed to a big new influx of young people living in the Golden Triangle in the 1990’s. The truth was that the Allegheny County Jail was rebuilt and expanded in the 1990’s and that expansion more than accounted for a nominal reported increase in Downtown’s residential population. The eventual increase in Downtown’s population would come mostly a decade and several hundred million dollars in subsidies later. Nonetheless, this misuse of Census data is certainly more widespread and likely be misreferenced for years to come.

This post originally appeared in Nullspace on June 29, 2012.

Topics: Demographic Analysis

16 Responses to “Misreferencing Misoverestimated Population by Chris Briem”

  1. This is a valid point, and indeed the massive discrepancy between the estimates on city-level population in 2009 and the Census 2010 results are indicative of the problem. For example — and this was the worst case — in 2009, Atlanta was reported to have a population of 540,921, up from 416,474 in 2000. But then in 2010, the Census revised the city’s population down to 420,003, to the great consternation and confusion of everyone.

    But, as I reported back in April, looking at core counties (and ignoring the municipal level) is a valid way of comparing changes between 2000-2010 and 2010-2011. As I showed, the core counties of the Seattle, New Orleans, New York, Miami, Atlanta, Minneapolis, Philadelphia, Portland, and Washington metropolitan areas are all seeing an increasing share of the regional population, which does indeed indicate central-city resurgence, even if we do not know the exact figures for the cities themselves.

  2. Yonah, the Atlanta thing had me fooled badly. It was the most over-estimated of any big city in America. Huge growth in the estimates followed by nearly flat population. I hope they are able to get more of a handle on municipal level estimates this time around.

  3. Rob Steuteville says:

    Thanks for pointing this out, I was not aware of this. On the other hand, it doesn’t discount the significance of these figures. In more than a few cases — Philly, DC, milwaukee among them — the county is the city. In many other cases, like Cook County, the county is the core county of the metro region. The core county is mostly comprised of the core city plus inner-ring suburbs that are walkable urban rather than drivable suburban. Comparing growth in the core county with middle and outer ring suburbs says a lot about where people are choosing to live. It may be a better measure than suburbs versus core city. The big caveat in my mind is that this is only one year of data.

  4. Chris Barnett says:

    This “core city” issue is vexing. Clearly, for each metro, there is some core of census tracts that represents a fixed and unchangeable metro core area. This core is most likely the pre-1950 developed rail-served area (i.e. a set of 1940 Census tracts or block groups) of any metro.

    In older landlocked cities like Philadelphia, some of the core tracts might actually be outside the core city-county municipality (such as the lower Main Line, parts of Upper Darby, Camden), while some of the inside tracts (Philly’s Levittown in the Northeast) would not.

    Some coordinated experts and crowdsourcing might be able to produce or refine “standard” core-city maps for the largest 50 metros, and thus remove that variable from this whole “cities-are-growing-no-they’re-not” argument. Tools are increasingly accessible: For example, Indy has digitized aerials back to 1937

    Maybe the Census Bureau (or urban reseach wonks) would see the utility and begin reporting on “standard core city” data and estimates.

    Surely someone has thought of this before and produced something useful.

  5. Alon Levy says:

    Aaron, it boils down to which methodology you think is more accurate: trying to count as many people as you can and treating that as the right number, or applying statistical methods to extrapolate from a representative sample. It boils down to Gallup vs. Literary Digest.

    Also, what Yonah said re core cities and counties. If the issue is purely that the ACS estimates are only accurate at the county level, then we can still look at cities where the core city is a county-level entity or has nearly all of its county’s population: New York, Philadelphia, Washington, Boston, San Francisco, Denver, Baltimore, St. Louis, Indianapolis. As it turns out, the ACS had Boston outgrowing its suburbs by a factor of 2 between ’00 and ’09, and New York and San Francisco outgrowing their suburbs by a small amount.

  6. Matthew Hall says:

    excellent suggestion, chris. Was atlanta overestimated or undercounted, aaron?

  7. The ACS methodology is different from the population estimates program, and the census bureau says to use popest as of now. In any case, there are no post-2010 ACS estimates available yet.

    There are definitely city-county entities. The challenge is that most of them are very unrepresentative. NYC, DC, and SF are booming as everyone knows, and are very different from other cities. I haven’t looked at Philly, but I did check St. Louis and Baltimore and both showed 2011 core city declines. I suspect this is more representative of bigger cities generally.

  8. Matthew, I find it hard to believe that the Census Bureau missed over 100,000 people in a city the size of Atlanta when it went out and did a count. I’m guessing it was very over-estimated for some reason.

    If you look at the cities that had an egregious discrepancy between estimates and the census count, unsurprisingly you’ll find many that successfully challenged estimates to get artificial estimate increases. Cincinnati for example.

    That’s why I don’t buy into Alon’s sampling argument. It may be more technically accurate, but the Census Bureau has already made it known through the estimates challenge process that it subject to political pressure to produce favored results and willing to give in to that pressure.

  9. Matthew Hall says:

    I ask everyone I know( more than 100 people) if they’ve had contact with the census bureau and they all say they haven’t. Estimation must a tremendous challenge. That is why I look to labor force and non-farm wage and salary numbers to help understand metro economies.

  10. Chris Barnett says:

    Matthew, remember that in any very large group a sample size of 800 is usually statistically sufficient to produce a small margin of error with high confidence level. So it’s relatively unlikely that any one person would know any of the 800-person sample in his/her city.

    The real issue today is that populations are so diverse, and some (educated, intelligent people) are so guarded about polling and personal data collection, that one must wonder about the basic premise of universal availability and the actual representative-ness of any sample-based statistics. Just look at how inaccurate political polling is today. (This is a serious social-modeling issue, not a mathematical or statistical one. One can model for lying, but not for withheld info.)

  11. urbandata says:

    The news analysis seems credible given that many cities (and many urban cores) have been growing faster than their suburbs from 2000 to 2010, for the first time since the early 20th century. Also, you have the outliers such as DC. This wouldn’t be the first time that stories have misinterpeted the data but as “trend” stories still been more or less correct.

  12. Jim Russell says:

    Garbage in, garbage out. Cities may be growing faster than suburbs. We can’t be sure either way. Longer time series data are more reliable. You can see the urban core bouncing back just about everywhere. Sometimes that rebound gets lost in the overall numbers.

    Policy cycles are measured in months. Demographic cycles are measured in decades.

  13. Chris Barnett says:

    If we are to assert that “cities are growing faster” than suburbs, I think it is fair to expect that statement to mean “more people are moving to cities than to suburbs”.

    In most instances, suburban population is larger than the core city’s, so a straight percentage of growth comparison is misleading because it refers to different bases. IMO the only good ways to compare are “share of metro growth” or absolute numbers.

    “City share of metro growth” is something that could be tracked meaningfully over a long period of time to glean a trend. It is also comparable across metros. Absolute numbers should accompany such a stat, so that small numbers in smaller metros don’t distort a national pattern.

  14. Patty Becker says:

    The Census Bureau’s excuse for this nonsensical publication of 2011 subcounty estimates is something to the effect that they’re evaluating their past methods and so did this as an interim measure. Did they think no one would notice? I think that they simply don’t care enough about the subcounty numbers to put the effort in. I am telling our users here to ignore these estimates.

  15. James says:

    I understand that this site likes to troll urbanists to gin up comments and traffic. But if the census numbers are only reliable for counties then you must agree that this quote holds true:

    “The nation’s No. 1 fastest-growing county from 2000 to 2010, Kendall was part of an exurban wave that more than doubled Kendall’s population…By 2011, Kendall County’s annual growth had stalled further at 1 percent, dropping its county growth-rate rank to 236th”

    So then really you are just playing semantic games over terms like core city vs city core, right?

  16. @James, without a doubt, the low end far exurbs have suffered a huge reversal in recent years. Many of these communities never built any intrinsic attraction beyond new and cheap. I worry what is going to happen to a lot of the low end subdivisions in these places.

The Urban State of Mind: Meditations on the City is the first Urbanophile e-book, featuring provocative essays on the key issues facing our cities, including innovation, talent attraction and brain drain, global soft power, sustainability, economic development, and localism. Included are 28 carefully curated essays out of nearly 1,200 posts in the first seven years of the Urbanophile, plus 9 original pieces. It's great for anyone who cares about our cities.

About the Urbanophile


Aaron M. Renn is an opinion-leading urban analyst, consultant, speaker, and writer on a mission to help America’s cities thrive and find sustainable success in the 21st century.

Full Bio


Please email before connecting with me on LinkedIn if we don't already know each other.


Copyright © 2006-2014 Urbanophile, LLC, All Rights Reserved - Click here for copyright information and disclosures