Great question! A common measure is not very "common" without evidence of comparable performance across relevant contextual factors. If we know *how* a measure works we can predict contextual factors that will impact its validity potential - so I think #understoodmeasures might be a better framing.
We also argue measurement requires a causal relationship from target attributes to measurement outcomes, making it a causal claim.
Upshot 1 even correlations between measured variables rely on causal inference.
Upshot 2 since good causal inference is hard, establishing realised validity is hard.
Let’s stop talking about “valid measures.”
No measurement procedure will *always* measure what it is supposed to measure.
Instead:
Consider Validity Potential when selecting a measure.
Check for Realised Validity when interpreting its output.
Validity Potential = how likely is it that this measurement procedure will measure what I want it to measure?
Realised Validity = does the specific output I collected from this measurement procedure measure what I am interpreting it as measuring?
Thrilled to share our new paper introducing “validity potential” versus “realised validity” as a key distinction for psychological measurement, with @davidmkaplanx.bsky.social, Alexander Gillett, @suttonprofessor.bsky.social, and @robert-m-ross.bsky.social www.sciencedirect.com/science/arti...
Thrilled to have been awarded an Australian Psychology Society Award for Excellent PhD Thesis in Psychology! psychology.org.au/about-us/wha...
Thanks to everyone who came to Sydney for #AIMOS2025! I can't wait to see you in Wellington, NZ next year!
Attended my PhD graduation ceremony yesterday. Gotta love the hat
A lot of psych is already conducted with online convenience samples & ppl are probably excited about silicon samples bc it would allow them to crank out more studies for even less 💸
How about we reconsider the idea that sciencey science involves collecting own data.
www.science.org/content/arti...
The Eyes Test knows where I live 👀
oh wow! lovely to see people engaging with my research 😃
www.bps.org.uk/research-dig...
My editorial on how journals can earn trust.
We often use journal names as proxies for quality. This is bad bc it’s not valid. But it could be. Editors could make journal name a valid signal. And we could place value on journals that show us how they do that.
journals.sagepub.com/doi/full/10....
Chuffed and excited to announce a new journal section in Cortex:
*Methods and Assumptions* is open for submissions starting today!
Read the opening editorial here (OA): www.sciencedirect.com/science/arti... 1/
Wendy Higgins (@wchiggins.bsky.social), who developed all these projects that interrogated the validity of the Reading the Mind in the Eyes test, has just been awarded her PhD and joined BlueSky! Follow her for more cutting-edge research on measurement and meta-science!
Btw, those leonine eyes in my mammal identification test belong to a Chow Chow (dog) – notice any parallels?
Stay tuned for the final paper from my thesis, which takes a step back to rethink validity itself.
It includes a series of papers critiquing the validity of the widely used Reading the Mind in the Eyes Test. Conclusion: The Eyes Test is unsubstantiated, and thus, uninterpretable as a measure of social cognitive ability www.sciencedirect.com/science/arti...
scholar.google.com.au/citations?us...
My thesis - Measurement in Crisis? A Critical Evaluation of Measurement Validation Practices in the Psychological Sciences - was awarded the Macquarie University Vice-Chancellor’s Commendation for Academic Excellence.
My PhD was conferred today 🎉 A huge thank you to my amazing supervisors @robert-m-ross.bsky.social @davidmkaplanx.bsky.social, @suttonprofessor.bsky.social, and Alexander Gillett and collaborators @dredeschrijver.bsky.social, @vsavalei.bsky.social, and Vince Polito
Introducing Dr Wendy Higgins and the Reading the Mammal in the Eyes Test.
