i think parts of Anthropic genuinely believe this for non-instrumental reasons, so it’s weird that one of the first things they’re trying to make their machine Galatea do is go to war
besides some very recent work (eg Josh & Sol's linked Brookings piece) the debate over LLMs in political science has been bereft of recent history. we're working in the aftermath of a 30-yr war over methods that nearly crashed the discipline, after all :thread:
the optimistic read on the old wars is that disciplines absorb disruption, fight it out, find some pragmatic middle ground. the less optimistic read is that the "middle ground" was cosmetic and the same people kept winning. which pattern do you think is going to repeat here
but more importantly old wars looked like they were about how to do research. LLMs sharpen the contrast of whether whole categories of intellectual labor just get automated. neither were truly methods debates, they were/are both labor market problems wearing a methods costume
also speed. the paradigm wars took three decades. ChatGPT hit a million users in a week and capabilities change every few months. the discipline's machinery for sorting out methods disputes runs on tenure-cycle time, it physically cannot keep up with the Anthropic release schedule
of course, the paradigm wars were a family fight. nobody outside the discipline gave a shit. the LLM thing involves a trillion dollar tech sector that needs you to adopt their product. that is a fundamentally different dynamic than arguing about KKV at APSA
the empirical picture is more mixed than we'd all care to admit. GPT-4 beats crowdworkers at annotation tasks by 25%, but switching the model you use can wildly swing treatment effects. we own none of the tools. it's an enormous replication crisis happening in plain view.
and now LLMs land right in the middle of this mess. the questions look the same, though. deskilling fears? we had those. black box anxiety? we had that. gatekeeping through journals/hiring? had that too. most grad students worry about skill atrophy *while still training*
and the resolution of all this was mostly aesthetic. "nominal pluralization," new journals, new APSA sections, rhetoric about methodological diversity -- all fairly surface-level accommodation, same power structure underneath
how many people remember the Perestroika email? Skocpol, Tilly, 200+ tenured faculty signing a petition against APSR becoming a quant monoculture. area studies people watching formal modelers fail to predict the Soviet collapse and get rewarded anyway. all things that really happened
besides some very recent work (eg Josh & Sol's linked Brookings piece) the debate over LLMs in political science has been bereft of recent history. we're working in the aftermath of a 30-yr war over methods that nearly crashed the discipline, after all :thread:
maybe so! i've been meaning to see it since so much was filmed in my home base of Albuquerque.
I'll be damned if that isn't a harbinger of things to come
Definitely that and a disinterest in/fear of the pedagogy of teaching research methods, which an outsider with a bit of pathos might salve
