Comment Permalink
Stuart Buck3y1

Those are all fair points, and I might have phrased things a little too strongly in the original post. 

I do think the education example is interesting, though, because both "sides" (if you will) are convinced that they are the only ones who truly care about improving children's education. The problem is that they're confusing means and ends. 

To me, whether it's progress studies or education or whatever, there needs to be a significant number of academically-minded folks who agree with the end of improving progress or improving education, but who are resolutely agnostic about the means of doing that, and who are willing to follow the data wherever it leads (including being willing to admit when something doesn't work, or backfires, or has other unintended consequences or tradeoffs). 

Reply
See in context
by Stuart Buck1 min read5th May 20222 comments

3

Frontpage

How can "progress studies" scholars be objective? 

From the original Atlantic article: Progress Studies "would study the successful people, organizations, institutions, policies, and cultures that have arisen to date, and it would attempt to concoct policies and prescriptions that would help improve our ability to generate useful progress in the future."

It's one thing for folks to "study" what worked in the past, and another thing entirely to "concoct policies and prescriptions" for the future. 

It seems dangerous to combine these two activities. As can be seen in other areas that are just a particular subject matter with no agreed-on methods (e.g., education), people whose real interest is in prescribing policies aren't the most objective analysts. Instead, they are often tempted to skew their scholarship so as to support their preferred policy.  

Hence, you find "scholars" of education who argue vehemently that charter schools do [or do not] create valuable opportunities for inner-city children. What's true? Who knows. It all depends on whether you ask someone who already has their mind made up in favor [or against] charter schools.

Combining scholarship with advocacy for particular policy positions ends up corrupting the scholarship. I hope that doesn't happen here. 

2 comments, sorted by
top scoring
Click to highlight new comments since: Today at 4:36 AM
New Comment
Write here. Select text for formatting options.
We support LaTeX: Cmd-4 for inline, Cmd-M for block-level (Ctrl on Windows).
You can switch between rich text and markdown in your user settings.


[-]elidourado3y 3

Stuart, I'm not seeing how it is a conflict per se to prescribe actions that would generate progress and also study what has worked to generate progress.

In the example you give, certain advocates of charter or public schools skew research on what creates valuable opportunities for children when they give advice. But the problem is that their true objective function isn't aligned with their stated objective function. They claim their goal is to advance opportunities for children, but what they really want is to promote or impede charter schools. So they skew their advice in service of their real motive while couching it in terms of their stated motive.

This could happen in progress studies as well. Suppose a progress studier's true commitment is not really to progress, but rather to libertarianism or leftism. Then they might put forward policy solutions in the language of progress studies that actually advance their other political commitment at the expense of progress. We should be wary of this.

Yet to fully separate the functions of research and advocacy will never work. It may be fine if there are at least a few people working in ivory towers who are solely studying what works to produce progress. But things immediately get muddier when you introduce any advocacy at all. Suppose I tried to be a pure advocate, and that my true goal is to advocate for policies that advance progress. Where am I supposed to find these policies? Probably I will need to read a lot of stuff on what works to drive progress. Even if there is a well-functioning pure academic progress studies community, I would at minimum need to read a lot of the papers and evaluate the literature, essentially doing my own meta-analysis. That's basically research. Given the current state of the field, advocates probably have to go even deeper than just this minimum level of research.

I think I could accept some softer claims.

1. It would be good if there were at least a few people who as pure academics study what works to produce social progress. These people could serve as a check on advocates misconstruing research to advance some other agenda.
2. We should be attentive to the fact that as a movement becomes politically successful people might use the language of that movement to push for other goals. This matches the charter schools example, using the language of opportunities for children to drive an ideological victory for or against charter schools.

Curious to know if you or anyone thinks I'm missing anything important here.

Reply
[-]Stuart Buck3y 1

Those are all fair points, and I might have phrased things a little too strongly in the original post. 

I do think the education example is interesting, though, because both "sides" (if you will) are convinced that they are the only ones who truly care about improving children's education. The problem is that they're confusing means and ends. 

To me, whether it's progress studies or education or whatever, there needs to be a significant number of academically-minded folks who agree with the end of improving progress or improving education, but who are resolutely agnostic about the means of doing that, and who are willing to follow the data wherever it leads (including being willing to admit when something doesn't work, or backfires, or has other unintended consequences or tradeoffs). 

Reply