The NYPD Took This Dog Into Custody Because His Owner Filmed the Police

[Dill]

(05-21-2023, 03:53 PM)Belsnickel Wrote: As much as we try in program assessment, it is almost impossible to get to a point where our experimental design would allow for truly causal statements. The real world is messy and accounting for confounders is tough. I have never found a study on this sort of stuff that I could not go through their design and find issues with it where I could say that it would potentially impact the results. It is the nature of these sorts of assessments.

This isn't to say that it is the same for all of social science, but in these sorts of assessments it is just generally the case. But keep in mind that program assessment is also relatively small when it comes to the social sciences. It hasn't had the sort of researcher attention that many other fields have had for much longer.

Late getting back to this, but well said and I appreciate your appreciation of the difficulties in research design. And you've always exhibited that researcher's instinct to examine texts/issues closely first with a built-in null hypothesis, and put analysis/understanding before evaluation/politics. Even when the design appears successful, you understand that still just means "for now"; other parts of that blind man's elephant may soon appear to change its shape in our understanding. That's a healthy anxiety.

We may understand "truly causal" a bit differently, though, in the sense that I am more comfortable with induction and probability. I have a close friend who is a theoretical mathematician, and for her, nothing is "knowledge" until we put it into an algorithm. The only social science that can really come close to that is linguistics. If that were the standard, we'd never be producing RELIABLE knowledge about, say, the school to prison pipeline or the difference between left and right populism. The science of polling could not be said to have improved since the Literary Digest fiasco of 1933. I've always viewed our major foreign policy fiascos--the Vietnam and Iraq Wars--as direct consequence of ignoring a great deal of "soft" knowledge about the countries in question, including intel. 

My greater comfort comes partly from recognizing how much we have learned from SS failures, how failure is "productive." The early 19th century saw the first attempts to seriously model a social science after the natural sciences, only to learn in the succeeding decades that, in contrast to the object of the NS, the object of the SS was so constituted by meaning, history, and social standpoint that it was permanently unstable. But this also affected the natural sciences, as SS gradually undermined such "natural" concepts as race and gender.  That teaches us much about the limitations of theoretical and methodological "instruments" in the NS as well as the SS, how even NS is "within" the social world, and that world is socially constructed from the ground up, even where we think the social has been excluded.* It also provides much guidance as to how "soft knowledge" can be reliably be deployed when we NEED social knowledge but cannot get it to apodictic certainty of causal relation.

I'm not a criminologist and have no in-depth knowledge of Stop-and-Frisk policies, but it does seem established that the pattern of frisking in NY produced discrepancies along the color line which cannot be explained by beat cops' color blind street knowledge of which frisks would likely pan out. Until it's clear how design flaws might affect THAT outcome, then the policy is rightly discontinued, given the probability of greater harm than good done to law-abiding citizens and to the legitimacy of our legal system, which aspires to color blindness. "Messy," sure, but much less messy than bald assertion the policy is working as intended, based on one guy's guess or personal perception. That's often the choice. 

*Not that all social scientists agree with this. LOL. Seems like some contingent in every generation fails to relearn its disciplinary history.