Dr. Jasmine B. MacDonald (00:00):
Hi there. I have an exciting announcement. This episode of Psych Attack is sponsored by Arcadia Beverages who are all about taking time out of the day for yourself. I'm really excited to team up with Arcadia Beverages because I love tea. I actually enjoy an Arcadia moment with a cup of their vanilla Chai every morning. Personally, I like it with soy milk and sugar, and this is usually what I'm drinking when I record Psych Attack. The cool thing is that Arcadia have put together a prize pack for one lucky Australian listener. To find out more about how to enter the draw to win the prize pack, please check out the Psych Attack social media accounts on either Twitter or Facebook. The handle is @psychattackcast. I hope you enjoy this episode of Psych Attack.

Dr. Jasmine B. MacDonald (00:49):
Hello and welcome to this episode of Psych Attack. I'm Dr. Jasmine B. MacDonald. Today, Dr. Nicole Sugden and I look into some practical considerations relating to psychometrics and psychological assessment tools. I hope you're going well and have settled in with a warm cup of tea.

Dr. Jasmine B. MacDonald (01:10):
Nicole. It is so great to have you here with me today.

Dr. Nicole Sugden (01:13):
Thanks for having me.

Dr. Jasmine B. MacDonald (01:15):
So, the last time that you and I caught up was right before you had a trip to the snow. How was that, and did you drink all the hot chocolate?

Dr. Nicole Sugden (01:25):
Oh, it was fantastic. Yeah, we had a family holiday, so got to watch my niece and nephew learn to ski and have many stacks in the process. um, drank lots of hot chocolate, which was amazing. And yeah, the snow was beautiful and the weather was perfect. Yeah. It was such a great holiday.

Dr. Jasmine B. MacDonald (01:41):
Oh, that's awesome. Yeah. I dunno that I should admit this publicly, but there is something kind of hilarious when people stack or fail. Right?

Dr. Nicole Sugden (01:49):
Yep. I spent three days just people watching.

Dr. Jasmine B. MacDonald (01:51):
That's fantastic.

Dr. Nicole Sugden (01:53):
Entertainment for days.

Dr. Jasmine B. MacDonald (01:55):
All right. So today we're gonna have a chat about your work in the area of psychometric evaluation of psychological assessment tools. But before we dive into that, I was wondering if you might introduce yourself and tell listeners about your training and how we got to this point.

Dr. Nicole Sugden (02:13):
Well, um, I'm Dr. Nicole Sugden. I am a lecturer at Charles Sturt University. I don't really know how I got to where I am right now. I've kind of just fallen into this. Um, I wasn't one of those people in high school that said "when I grew up, I wanna be a psychologist". It was not even on my radar. I think at that stage, I wanted to be a vet or a zoologist until I realized I was allergic to cats. So that kinda put a dampner on that one. Um, so yeah. Um, yeah, I think I just somehow fell into psychology after school and found out that I really enjoyed it. At first I thought I'd be a clinical psychologist, you know, your typical Freud sitting there listening to people in therapy, but I kind of soon realized that it wasn't for me at all that I wasn't interested in that, but there were these subjects that I really loved, like biopsychology, learning, cognition. And I had these amazing lecturers that just brought those to life. And I thought, "wow, this is something maybe I could do as a career". And I really like the research side of it as well. So putting those things together, I thought, "okay, maybe I can do something here with this". I ended up falling into the field of prospective memory, partly because of my grandmother. She now has dementia, but at the time, 15 years ago, she was complaining about her memory all the time and saying, "oh, Nicole, I wish, you know, with your psychology, you could do something about this". So that kind of got that little light bulb moment. I'm thinking, "okay, maybe I could do a PhD on memory". So yeah. Ended up doing a bit of Googling in all the databases and seeing what there was on memory out there. And yeah, I discovered this thing called prospective memory, which at the time was this little unknown part of the memory world. I think there were only maybe 400 articles at that time published, which in the scheme of things is tiny other bits of memory there's hundreds of thousands of articles around. So yeah, I discovered this little niche area of prospective memory and thought, yeah, this is really interesting. And I could work with this. My PhD supervisors were involved in clinical psychology. So they were involved in assessment and rehabilitation of people with brain injury and things like that. Um, so they said, "oh, how about we develop, you know, a measure of prospective memory that could be your PhD topic". And yeah, I've kind of just fallen into this world of looking at memory scales and how we investigate memory and how we can make those scales better. So yeah, kind of a weird journey into it, but I'm really glad that I got here.

Dr. Jasmine B. MacDonald (04:33):
So you kind of fell into this area, but what keeps you in this area now, as you say, you know, 15 years later?

Dr. Nicole Sugden (04:40):
I just discovered that I really love doing it. I think I'm a bit of a nerd. So I kind of like the statistics, most students that, you know, do psychology, like, "ah, statistics, that's the worst thing ever", but I kind of just love that you can put things into SPSS [Statistical Package for the Social Sciences] and do these analyses and it comes out with these beautiful numbers and patterns and things like that, which is really cool. And you're solving real world problems with these things. So people say, "wow, you've got a great memory. You know, you must be born with that". But I think it's, cuz I've learned all of these great memory strategies from my research that I'm able to apply them. And so when people are forgetting all the time, I'm like, "Hey, how about you try this?" And yeah, I think it's really cool that there's real world applications, you know, in everyday life to make things easier with all of these memory tricks that you learn.

Dr. Jasmine B. MacDonald (05:25):
I love that studying and researching psychology gives us life hacks. If we're insightful enough to think, "Hey, what can I take out of this for my own, my own benefit".

Dr. Nicole Sugden (05:36):
Yep.

Dr. Jasmine B. MacDonald (05:38):
And I kind of relate in terms of finding maths and statistics a little bit daunting until I started psychology. And then realizing that they're a tool. It's not that you have to enjoy necessarily the process of sitting down and working with numbers and analyzing data, but that kind of comes, uh, maybe it is just us nerds, but I think that kind of comes from knowing why you're doing it and that there are practical applications and implications like you've said. Yeah.

Dr. Nicole Sugden (06:06):
Yeah, that's it. Like I have this problem, you know, people with brain injury are having trouble with their memory, so I'm not just going, okay, I wanna run some stats here. I'm like, "okay, what's the best way that we can measure these memory problems so that we can develop the best therapies and treatments for these people?" So it's kind of, the stats is a means to an end. Um, yeah, it's not that...

Dr. Jasmine B. MacDonald (06:25):
Absolutely.

Dr. Nicole Sugden (06:25):
... I just sit there wanting to play with numbers all the time.

Dr. Jasmine B. MacDonald (06:28):
So I was thinking for the episode today, what could be good is to start with some background for those who are uninitiated in terms of psychological assessment and psychometrics, and then use a couple of your research works to unpack what that means in a practical sense and, and why having a better understanding of psychometrics and developing really sound psychological assessments is worthwhile.

Dr. Nicole Sugden (06:58):
Sounds good.

Dr. Jasmine B. MacDonald (06:59):
Lovely. So if you had to describe to someone who has no background in psychology, what a psychological assessment tool is broadly speaking, what would you say?

Dr. Nicole Sugden (07:09):
A psychologist will use the whole heap of tools. So they might interview a patient or a client. They might collect records. You know, it might be school records or medical records. It's all about gathering information. It could be as simple as a questionnaire. So you've probably been to a GP and been asked to fill out a little depression questionnaire. So how are you feeling today? That sort of thing. Um, or they could be more complex like an IQ test or a personality test. So it might be in these situations, do you like to do this or, true or false, you know, um, are you a person that enjoys art or are you adventurous? So those types of things are all different types of psychometric scales and they do vary in qualities. So there's, you know, really complex ones that have been developed by these huge publishing companies over, you know, 10 or 20 years and had a lot of rigorous testing to make sure that they're measures out there. And then at the bottom end of the scale list, the things you used to read in those Cosmopolitan magazines, like which ice cream flavor are you or which friends character are you , um, obviously those are pretty dodgy. Um, but yeah.

Dr. Jasmine B. MacDonald (08:11):
That is so funny that you bring that up because now I'm having flashbacks of being, you know, 12 or 13 and loving Girlfriend magazine. And I used to love going through those quizzes and maybe that's what influenced my psychology journey.

Dr. Nicole Sugden (08:24):
Who knows it might have done that to me as well.

Dr. Jasmine B. MacDonald (08:27):
That makes me happy.

Dr. Nicole Sugden (08:31):
Yeah. So I guess, um, yeah, these psychological tools are just one of the things that a psychologist will use to gather information and then they is that information to, I guess, make a psychological formulation. So, okay. Is this person feeling anxious? Are they more anxious than the average person or in terms of the memory area that I look at with someone with brain injury is their memory impairment worse than someone from the general population? And if so, how bad is their memory? And from that we can look at, okay, what parts of their memory aren't working well and, you know, do we maybe need to look at memory aids or do they need a carer to support them? Might they need NDIS [National Disability Insurance Scheme] funding to help them with memory aids or things like that, or it could be after a brain injury, does this person become eligible for some sort of compensation because they've had significant brain impairments? Yeah. Lots of applications.

Dr. Jasmine B. MacDonald (09:23):
Yeah, absolutely. When you say that some of these tools have been really well established...

Dr. Nicole Sugden (09:29):
mm-hmm

Dr. Jasmine B. MacDonald (09:30):
...it is always context specific, right? That...

Dr. Nicole Sugden (09:33):
Yeah.

Dr. Jasmine B. MacDonald (09:33):
...You know, they're developed in a context for a certain purpose and when we try to use them somewhere else or with a different kind of population, we can have some problems there.

Dr. Nicole Sugden (09:42):
Yep. And that's yeah what my research is very much looking at is people have developed this scale for one group of people, but does something that applies to a 30 year old male apply to a 90 year old female or someone with ADHD [attention deficit hyperactivity disorder] versus someone that doesn't can, we use these across different contexts or even culturally. So what's developed in Australia, is that actually a good idea to be using that in the U.S. or vice versa? So yeah, lots of issues that people that are interested in psychometrics can look at.

Dr. Jasmine B. MacDonald (10:10):
Lovely. So now we arrive at psychometrics. What do we mean when we say that?

Dr. Nicole Sugden (10:15):
I'm not very good at explaining this cuz when people ask me what I do, I think they just either doze off or um, look at me with blank stares. Um but yeah, I guess like a bit of analogy. So psychologists use all these tools. Um, so like a builder uses the hammer, um, I guess people that work in psychometrics, so the people that make the hammers and make sure that, you know, they use the right way. So you're not trying to screw in a nail or something with a hammer, like you're actually hammering a nail. So yeah, we're about developing these measures that psychologists use and then checking that they're consistent and reliable and measuring what they're meant to measure.

Dr. Jasmine B. MacDonald (10:49):
In a, a good value, psychological manuscript or paper, we'd expect to see some description of reliability and validity on all the tools used.

Dr. Nicole Sugden (10:58):
You would hope so in good quality journal articles, I guess you are expecting, you know, they're saying "we're using these measures that have been validated and are reliable to be used for the purpose that we're doing it." Otherwise the rest of what they've put in that article is going to be rubbish because if they're using dodgy tools, then the results are pretty questionable. So it's kind of important that they do report that. So we can check that and make sure it is appropriate that they're being used in the right way.

Dr. Jasmine B. MacDonald (11:22):
Yeah. Nice one I think now is as good a time as any, to jump into an example. So one of the pieces of research that you've published is around the test of premorbid functioning.

Dr. Nicole Sugden (11:35):
Okay. So the test of premorbid functioning is something that's often used in brain injury. So when someone has a brain injury, we wanna know how much impairment they've had. So how much damage was there. So ideally we want to have some sort of measure of before and after. So in terms of IQ, not everyone does an IQ test. So you might have done one of those dodgy ones online, but not a properly administered one by a psychologist. So it's pretty rare that people have this baseline data of what their IQ was before a brain injury. We can conduct an IQ test after their brain injury and go, "okay, yep, their IQ score is say a hundred now", but we don't know what to compare that to. These tests of premorbid functioning is there to estimate what their functioning would've been before their brain injury. So the way that we do this is through word reading tasks. We give people the Test of Premorbid Functioning, or the TOPF if you wanna abbreviate it. Um, cuz we love abbreviations...

Dr. Jasmine B. MacDonald (12:30):
I was just gonna say, we love that in psychology. Yes.

Dr. Nicole Sugden (12:33):
Yep. Um, so yeah, the TOPF is a list of 70 words and these words are irregularly spelled. So to give you an example, um, that's not on the word list cuz it is copyrighted, but the word echidna, so that's kind of not spelled in an easy way. So in Australia we should know it pretty well because we know echidnas and we recognise that word. But for someone that doesn't know that word, they'd probably pronounce it e-chid-na [as opposed to e-kid-na] or something like that.

Dr. Jasmine B. MacDonald (12:58):
Mm.

Dr. Nicole Sugden (12:58):
So we use this list of words that aren't spelled in a way that we would expect them to. So the idea is that before a brain injury, we would've learnt these words and should have that knowledge. Word knowledge is something that's preserved after brain injury. So that's one of the things that doesn't seem to disappear when we have an injury. So we can use that word knowledge to see how much damage there was. And we expect that people will struggle with some of these words. So yeah, the TOPF list is 70 words, starting with the easiest word at number one and the hardest word number 70. And I've had to Google the pronunciations of some of these words at the top. They are really hard. Um, um, some really unusual words, that I think "what is that"? Um, so what we do with that is we get the person's overall score out of 70 and we plug that into an equation, um, along with things like age and occupation and education, because those things are also predictors of IQ. So adding those things into the equation actually helps us predict more accurately what people's IQ was before their brain injury, which is pretty cool.

Dr. Jasmine B. MacDonald (14:00):
Yeah. Awesome.

Dr. Nicole Sugden (14:01):
And they're pretty accurate. Some of the studies when they develop these things found that about 80% were within 10 points of the real IQs. That's not too bad.

Dr. Jasmine B. MacDonald (14:10):
From a list of words. It's interesting.

Dr. Nicole Sugden (14:12):
Yeah.

Dr. Jasmine B. MacDonald (14:13):
All right. So I'm guessing then, one of the key issues comes in that if we have these scales, like the TOPF being developed in the U.S. and the kind of standardized norms where, you know, we might administer this test with an individual and compare their results to this standardized group of what we might expect the outcomes to be. But these norms aren't Australian based. What's the potential issue there?

Dr. Nicole Sugden (14:39):
Yeah. So the TOPF was developed in the U.S. quite a while ago as well, back in 2009. So it was standardized alongside the Wechsler Adult Intelligence Scale fourth edition. So that's getting a bit outdated now in 2009 and yeah, when they developed a version for the UK, they found that a lot of the words were out of order. So there were eight words that they had to change the order of, because there are kind of local language usage differences, like I mentioned with echidna. So it's a word that's really familiar to us in Australia, but use by word in America and they probably have no idea how to spell it or how to pronounce it properly.

Dr. Jasmine B. MacDonald (15:14):
Because they're in order of difficulty.

Dr. Nicole Sugden (15:17):
Yeah, that's right. Some words are harder for people in the U.S. compared to the UK and vice versa. Um, and the same would apply with Australia. So we don't really know Australians kind of use a mix of UK and American English. So some words might be really hard for us compared to the UK and U.S. and some might be really easy. So yeah, there's these potential issues with the difference across language use across those countries as to whether words are too easy or too hard. And then that has implications when it comes to estimating that premorbid IQ, because we could be underestimating people's IQ before and maybe not giving them the compensation they need or the support they need.

Dr. Jasmine B. MacDonald (15:55):
Yeah. I was thinking about your research and the experiences I've had in terms of word recognition and familiarity. I probably shouldn't disclose this, but sometimes when I've had a rough day and I just wanna chill out and relax my brain, I watch the U.S. Wheel of Fortune. .

Dr. Nicole Sugden (16:13):
Cool.

Dr. Jasmine B. MacDonald (16:15):
Sit back and have a cup of tea or wine and watch Wheel of Fortune. You know, you can go along pretty smoothly for most puzzles, but then a puzzle will come up with a word that you're not familiar with and it really throws you.

Dr. Nicole Sugden (16:27):
It does.

Dr. Jasmine B. MacDonald (16:27):
Yeah, I know this isn't exactly what you're talking about, but it's in that same ballpark of those differences of culture. So the one that got me the other day was leaky faucet. I was like, no, this is not a word that I use. So I, uh, struggled with that. And another was, um, dessert cart, cuz it's just not an Australian kind of phrase. So even state differences within Australia.

Dr. Nicole Sugden (16:49):
mm-hmm .

Dr. Jasmine B. MacDonald (16:50):
Castle [car-sil] in New South Wales being pronounced castle [cas-il] in Victoria.

Dr. Nicole Sugden (16:54):
Yeah.

Dr. Jasmine B. MacDonald (16:55):
Yeah. And that in itself kind of presents some challenges for this kind of psychometric validity.

Dr. Nicole Sugden (17:02):
You've actually just reminded me in the news this week, there was a thing about Pepper Pig in lockdown. So American children have been watching a lot of Pepper Pig and they're starting to use these UK words that are not American. So they're replacing vacation with holiday and things like that, which is really interesting. So you can kind of see how that translates into these tests as well.

Dr. Jasmine B. MacDonald (17:24):
Whoa, that's really interesting. And it leads me to something I wanted to ask you. In your method, you talk about the way that you score something correct or incorrect was the Macquarie dictionary pronunciation of a word.

Dr. Nicole Sugden (17:37):
Yep.

Dr. Jasmine B. MacDonald (17:37):
But it just made me wonder about people I've come across who have adapted their accent because of film and culture and these kind of differences that in itself has to make this challenging again.

Dr. Nicole Sugden (17:50):
Yeah, for sure. And I think that's the problem with these tests because you know, they are 2009 developed. So a lot of words, you know, things like iPad wasn't even around then. So a lot of the words we used now and not even in lexicons.

Dr. Jasmine B. MacDonald (18:03):
Interesting.

Dr. Nicole Sugden (18:03):
Yeah. Like I think it presents a big issue and kind of raises the question "Do we need to keep updating these a bit more regularly to be, I guess, more appropriate for different countries and different word usage over time?"

Dr. Jasmine B. MacDonald (18:16):
What did you find in your study? What were the kind of key takeaways?

Dr. Nicole Sugden (18:19):
It was actually a lot worse than what we thought. Um, but like I said, there were only, like the U.S. and UK had eight words that were, that they changed because they were kind of a long way out of order. Um, I think we found that there were 30 words different from the U.S. and 32 out of order, like quite a bit from, more than 10 places I think, from the UK and U.S. So that's a huge difference.

Dr. Jasmine B. MacDonald (18:42):
Interesting.

Dr. Nicole Sugden (18:42):
Yeah. So Australians obviously speak very differently to other countries and have different word knowledge. So that really suggested that yeah, a lot of these words aren't appropriate to be used in Australia and something else that they use with the TOPF is the discontinue rule. When you're doing a psychological test, it can take a long time. So you don't wanna overburden the person filling out the questionnaire or doing the IQ test. So if they get five wrong in a row, then the psychologist will, "okay, that's enough. Like obviously you're not gonna get any more right here because they're getting harder". But that's a bit of a problem because we found like 30 and 32 words different to the U.S. and UK norms. So if you apply that discontinue rule where people are stopping after five, those words are totally out of order and you could have someone that was getting questions right at the top correct, but missing a few in the middle. And we actually found that. So the person with the highest IQ actually got question 24 wrong. So that's a pretty easy item and someone could have been stopped way before they should have been. And then we also found that people had really low IQ were getting items 68 correct. So the third highest word people with, you know, IQ of like 88, so well below average, we're getting that right. So that's kind of screaming out that, okay, these words aren't right for use in Australia.

Dr. Jasmine B. MacDonald (20:00):
It sounds like a potential risk here is, I guess it's that false negative, false positive situation.

Dr. Nicole Sugden (20:06):
Mm-hmm .

Dr. Jasmine B. MacDonald (20:06):
If you have someone who has a, a high, high level of intelligence relative to other people and they've acquired some brain injury, but this test says that they're performing worse than they actually are, then the treatment and kind of rehabilitation plan for this person isn't gonna be appropriate.

Dr. Nicole Sugden (20:23):
Yeah. So that's a huge problem. And yeah. So we've kind of recommended that the discontinue rules probably a bad idea in Australia at the moment.

Dr. Jasmine B. MacDonald (20:30):
Yeah. Okay. Nice one. All righty. So the second example from your work was around different ways of measuring prospective memory and yeah, let's maybe start with a description of what prospective memory is.

Dr. Nicole Sugden (20:45):
Okay. Don't get me started cuz I love talking about this um, prospective memory is really cool. Um, it's something, no one really knows what it is, but it's something we use every day. So it's remembering to do things on time. So remembering to take your medication, remembering appointments, remembering to say happy birthday to someone, um, all of, of those things and it, yeah, pretty much remembering to do anything at the appropriate time. And interestingly, the studies show that this is the thing that people forget the most. So when people complain about their memory, it's usually prospective memory that they're complaining about.

Dr. Jasmine B. MacDonald (21:22):
Interesting.

Dr. Nicole Sugden (21:22):
So "oh, man. I meant to do that" or "whoops". Yeah. So they are prospective memory failures, which is pretty interesting.

Dr. Jasmine B. MacDonald (21:28):
All righty. So what kind of brain regions and cognitive functions do we use to deploy or to use prospective memory?

Dr. Nicole Sugden (21:37):
So lots of different parts of the brain. Um, the prefrontal cortex, which is related to our higher order processing. I like to call it like our little CEO of the brain. That's overlooking everything and managing everything.

Dr. Jasmine B. MacDonald (21:48):
Uhuh.

Dr. Nicole Sugden (21:48):
So it's really important. So when we set an intention, we have to, usually when we are doing these sorts of things, we're not all day going, "oh, I've got a doctor's appointment in four hours". That's not what we're always thinking about. We go off and do other things. So we might go check emails or go walk the dog or something like that. So we're not constantly thinking about it, but our prefrontal cortex is there in the background, actually searching the environment, going, "okay, now I've got that appointment later. So I just need to keep an eye on the clock, check in every few hours to make sure I don't miss that appointment". So it's there kind of just monitoring for those cues all the time. And then obviously we've got the hippocampus, which is kind of involved in consolidating those short-term memories in to long-term memory. So that's important for remembering the where, what, why and how so. Okay. Doctor's appointment at 2:00 PM today, once our prefrontal cortex says, "Hey, I've got something to remember here". The hippocampus jumps in and says, "yep, it's an appointment at 2:00 PM. Let's go". So yeah, they're probably the main two areas. So things like working memory are really important, so that multitasking sort of aspect, executive functioning. So being able to switch between different tasks when we know it's the right time to fill that intention. Yeah. They're probably the main cognitive processes.

Dr. Jasmine B. MacDonald (22:57):
My exposure to memory research has been around teaching. I've not researched this area myself. And sometimes when we teach psychology, it can kind of just be like it's it's own thing in a vacuum of here are the structures and what the processes are and how they're different from each other. But, uh, I guess I hadn't sat back and thought about how important prospective memory is as maybe an indicator of a range of clinical groups. And you listed quite a few in your paper...

Dr. Nicole Sugden (23:24):
mm-hmm .

Dr. Jasmine B. MacDonald (23:24):
... so Parkinson's disease, schizophrenia, multiple sclerosis, HIV, traumatic brain injury, acquired brain injuries like stroke, dementia, mild cognitive impairment. And then also this connection with alcohol and other drug use. This...

Dr. Nicole Sugden (23:40):
Yeah.

Dr. Jasmine B. MacDonald (23:40):
This is a fascinating topic to me because it clearly has that importance clinically, but also a day to day relevance for people needing to remember, to put their bins out, to take their medication to facilitate their relationships by calling someone back when they said they would. Yeah, this is a, a really interesting kind of broad reaching function that we use.

Dr. Nicole Sugden (24:00):
It is. We kind of take it for granted, but we're really bad at remembering. I think we just lead these busy lives and yeah. We tend to not remember very well. So I'm sure everyone listening at some point will go, "oh whoops. I totally forgot to do that today". So um, you kinda realize, yeah, it's kinda important.

Dr. Jasmine B. MacDonald (24:17):
Totally. Your work has looked at different ways of measuring prospective memory. You had three types, it was self report, informant report and performance based. What are those three things? Like, how are they different from each other?

Dr. Nicole Sugden (24:31):
So there's lots of ways, this is what I discovered in my PhD is, we can measure prospective memory in lots of different ways. So self report is quite popular. So giving some of the questionnaire and saying, "okay, how often do you forget to put the bins out? How often do you forget appointments? How big of a problem is your memory? Are you concerned that you forget things all the time?" So that's one way of doing it. Just asking the person directly, "what are you forgetting?"

Dr. Jasmine B. MacDonald (24:54):
So why is that popular, that method?

Dr. Nicole Sugden (24:57):
It's quick, it's cheap, it's easy. So most of these are freely available online. So you can kind of get one of those out in 10 minutes and get the information you need and I like them because they measure everyday activities. So they're covering things that, you know, are commonly forgotten in everyday life. So I think they catch the real aspect of prospective memory. So they might even asked what memory aid you use and are there personality things or motivational things that could be influencing your memory as well?

Dr. Jasmine B. MacDonald (25:26):
Yeah cool.

Dr. Nicole Sugden (25:27):
I think they've got a few advantages. A lot of people don't like them though. People say that questionnaires are prone to all these different biases. So people might present them as not having as bad a memory as what they actually have or yeah. Faking good or faking bad. A big problem is people might forget that they forget. So you ask somewhere with dementia. So my grandmother, for instance, the other day it was her birthday and she's like, "who's birthday, is it today?" "Yours nan" She's like, "oh, did I forget that?" "Yes, nan, you did" so yeah. Um, yeah, forgetting that you've forgotten is kind of an issue with these as well. And that's probably where the informant reports come in, getting someone else to complete the same questionnaire. But on behalf, I guess I could say, you know, I'm filling a out one on behalf of this person with a brain injury because I know them well. So how often does that person forget? Oh yeah. I noticed they do that all the time. So yeah, tick that one.

Dr. Jasmine B. MacDonald (26:16):
I got a concussion playing basketball a couple years ago. We had this lunchtime tournament when I lived in Sydney and we'd just play against other corporate teams. Anyway, I kind of turned around and he was running towards me and headbutted me on the jaw and I passed out like for a split second hit the ground.

Dr. Nicole Sugden (26:35):
mm-hmm .

Dr. Jasmine B. MacDonald (26:35):
And anyway, ended up going to the doctor and called my husband to pick me up. And in the meantime I was sitting in the office at work with a friend, shout out to Claudia who took very good care of me, but we were having this conversation and I felt, I felt pretty good, but I had a complete lack of insight of how the concussion had impacted my memory. So I, I think we're having this really normal conversation. And Tim came into the office to pick me up and I'm all smiley and, and greeting him. And Claudia says, like, gives him this kind of handover of the patient. She's like, "okay, so she's, she's happy, but actually she's forgetting things and she's having thought blocking" and all this stuff. .

Dr. Nicole Sugden (27:16):
Oh gosh.

Dr. Jasmine B. MacDonald (27:16):
And I just remember after that experience going, "wow sometimes we, we are really not aware of the things that we're not remembering". That was a really profound experience for me. And an example of informant report.

Dr. Nicole Sugden (27:30):
Exactly. So yeah. Informant reports are useful.

Dr. Jasmine B. MacDonald (27:35):
So how are those two different from a performance based measure of prospective memory?

Dr. Nicole Sugden (27:41):
Yeah. So I guess these ones are, I guess, indirect measures because they're asking you and trusting you to report accurately on your memory activities. Whereas the performance based task is getting you to do a task and seeing if you remember. So I guess that's the most direct way to measure, you know, is someone forgetting, well, let's test it on a memory task, common ones that are used in, I guess, clinical settings or things like when you get to the end of this booklet, sign your name at the top of the page, or remind the researcher in 15 minutes that they've gotta make a phone call or the hidden belongings a very common one. So they might take a watch or a personal belonging and say at the end of our session, you need to remind me that I need to give this back to you. And then we see if that person remembers, kinda fun.

Dr. Jasmine B. MacDonald (28:27):
It is fun. It sounds like a way to score a bunch of items from people who have poor memory.

Dr. Nicole Sugden (28:32):
yeah. I dunno if the therapist actually gives it back at the end or not, but, um, I guess it's more of an incentive to remember.

Dr. Jasmine B. MacDonald (28:40):
So we would expect then if they're all measuring prospective memory that they should, you know, scores on these things should correlate, as, self report should go up at the same time as a, the performance based measure. What did you actually find?

Dr. Nicole Sugden (28:56):
Not the case no, the correlations were terrible. They were really low. So yeah, probably correlations of like 0.3, which is pretty dodgy at the most. So basically it was saying that there's not much relationship at all, between what someone says on a questionnaire and how they perform on these particular tasks, which is kind of alarming. And this is probably why a lot of researchers say "we shouldn't do self report. They're not valid. They're not measuring real performance", but I kind of have the argument, um, that maybe the self reports are still okay, but they're measuring a different thing. So these performance tasks. If you look at them, they are only measuring one occasion. So usually it's just one thing, remind me, in 15 minutes to do this, it's usually artificial. So a lot of the tests that they used in the studies that are reviewed were computer based tasks. So words would pop up on a screen. And it'd say, "when you see the word tiger hit the space bar", that's not a task that you would do in everyday life. Is that really measuring what a self report questionnaire is measuring? I don't know. So yeah, I think there's a few issues with the actual performance based scale, not just a self report, but even when they use more realistic things like virtual reality, they still found that the correlations were pretty low. My conclusion was kind of that self-report is probably measuring these performance things, but in a much broader sense. So looking at how people use memory aids, things like task importance, things like that.

Dr. Jasmine B. MacDonald (30:19):
Yeah. Interesting. I guess the reason for having an informant report is the potential memory issues in the person we're working with. So where did that one come in in comparison to the other two?

Dr. Nicole Sugden (30:30):
Probably a little bit better, but still not ideal. Sometimes the informants weren't overly accurate or maybe it was the self report that wasn't accurate. And the informant was, we don't really know which was the correct per person, but we found that in healthy people, um, the informants weren't very good. Um, and this is probably because if you are a healthy person wandering around in your daily life, an informant's not like hovering over you going, "oh, did they take their tablets today?" or anything like that. So they're probably not paying attention to the memory failures, but someone's actually having, um, whereas if you're looking at someone with dementia or brain injury and you've got nursing staff or a carer or a spouse looking after them, you're monitoring all those things and really paying attention to those memory failures.

Dr. Jasmine B. MacDonald (31:13):
Mm-hmm .

Dr. Nicole Sugden (31:13):
So, yeah, I think those informants were tending to overestimate the problems, whereas informants of healthy people were underestimating the problems.

Dr. Jasmine B. MacDonald (31:21):
Interesting.

Dr. Nicole Sugden (31:22):
And context is probably important too. So if your informant is someone that you work with, they might be very good at knowing how your memory works at work. But do they know if you remember to do the dishes, they probably don't have that info report accurately.

Dr. Jasmine B. MacDonald (31:35):
Right. I think that makes a lot of sense that in different areas of our lives, we are going to be more diligent or kind of, I don't know, more intense about remembering certain things...

Dr. Nicole Sugden (31:48):
mm-hmm .

Dr. Jasmine B. MacDonald (31:48):
... like at work compared to at home. Yeah. It's probably like the foundation of a lot of relationship disputes.

Dr. Nicole Sugden (31:56):
Mm-hmm yeah. I think the implications are bad. Yeah. If you have a prospective memory failure, it's, you know, "do you not care enough about me to remember my birthday?

Dr. Jasmine B. MacDonald (32:06):
That's what Facebook is for.

Dr. Nicole Sugden (32:08):
Exactly. What did we do before Facebook?

Dr. Jasmine B. MacDonald (32:11):
Hopefully not for your, um, you know, your spouse, but um, for everybody else, that's what Facebook is for.

Dr. Nicole Sugden (32:17):
Mm-hmm Yep.

Dr. Jasmine B. MacDonald (32:19):
One of the things you talked about in your paper was the self-report probably incorporates aspects of mood and personality. And I thought that was really interesting.

Dr. Nicole Sugden (32:29):
That's probably an area that I am looking into a little bit is things like personality and mood. So they've found that personality traits like neuroticism. So this is where people are highly anxious. They worry, they're always ruminating on things like kind of worrying that something bad will happen. Um, so these people tend to report high levels of self-report memory problems. It's not necessarily that they do have more memory problems, but they are reporting more. Maybe they're more concerned about their memory, cuz they're concerned about everything in general, the research found that on self report, neurotic people tended to have higher scores, but not so much on the performance based tasks. So it didn't seem to be translating into those real life tasks, which was interesting. People that are conscientious tend to do very well. This actually translated across to performance and self report. So people that are conscientiousness seem to monitor more. So these are the people that have their to-do list and they're checking the clock. They're not, you know, wanting to forget to do things on time. So if you need to remember something, make sure you get a conscientious person to help you remember um, would be my advice. Things like stress and depression can affect prospective memory as well. So when we're stressed, we're, you know, usually ruminating on something that stressing us out and we don't pay attention to things and we tend to forget. So when we're distracted and multitasking, this is when yeah, we're gonna report more memory problems because we're just not focused on the task.

Dr. Jasmine B. MacDonald (33:52):
Right. If we have, as researchers or clinicians, the ability to draw on these various kinds of measures of prospective memory,

Dr. Nicole Sugden (34:01):
mm-hmm .

Dr. Jasmine B. MacDonald (34:02):
... how do we choose between them?

Dr. Nicole Sugden (34:03):
Well, I guess it's like any tool you wanna use it for the right purpose. So you wanna use a hammer to hammer in a nail. You wanna use a screwdriver to screw in a screw. So it's about kind of knowing what you wanna measure and then using the right tools. Um, my approach would be to actually use a combination so that you're getting a little bit of both because I think they are measuring different things. So if you are interested in the everyday life aspects, so what memory age is someone using? How is this actually affecting their daily life? I think that's where a self report is really important. If there are concerns, but there is maybe clinically significant impairment, then I'd be looking at informant report as well to get that kind of corroborative information. And then yeah, the performance is also good to actually get that one off measure of that person at that time. So are they actually performing badly? And yeah, the more information a psychologist can have to make a decision, I think the better. So why not use all of them?

Dr. Jasmine B. MacDonald (34:55):
Yeah. Makes a lot of sense. All right. Let's step out of the clinical realm. Day to day, what are the tips that listeners can observe to improve their prospective memory?

Dr. Nicole Sugden (35:07):
The thing that people don't do, and this is why people fail the most is they don't pay attention. So you just assume that you've got this intention, this thing to do. And you're going to remember, and I know I've done this at least three times this week. I'm like, yep. I need to send out a zoom invite to people that I'm meeting with later in the week, woke up this morning and went, "oh, whoop, I forgot to send that one". Um, so I, you know, I was distracted. I was doing lots of emails. I'm like, "yeah, I'll remember to do that later". And because I didn't pay attention and I left it till later, I totally forgot about it. So I think paying attention and actually being mindful when you set those intentions is really important. So if you think of something you need to do write it down straight away so that you are encoding it properly and will remember to do it. This is probably why people lose their phone. And "where did I leave my keys?" I wasn't paying attention when I walked in the door. So yeah, that's one big one. Yeah trying to visualize yourself or being really clear about the intention. So when I go to the supermarket, I'm going to picture myself in the milk aisle, grabbing the milk. And you're more likely to remember to do that. So you don't have that horrible experience, you go home and go, "oh man, I've left the milk". uh, so yeah, that's another big one. Yeah. I think if you are distracted and busy, don't assume that because it's important you will remember. Um, I have recently done a bit of research on importance and when things are important, we tend to use more memory aids, which helps us remember. But often we think, "oh no, it's important enough. So I don't need a memory aid" and that's when we fail. So if you think it's important, use a memory aid and you're more likely to be successful, write it down, put it in your phone and make sure your phone is charged and actually going to remind you at the right time. Um, cause yeah, we do rely on our smart technology to do all of this stuff, which is great. But sometimes if that fails, then we're in big troubles.

Dr. Jasmine B. MacDonald (36:47):
Oh absolutely. I depend on post-its mostly...

Dr. Nicole Sugden (36:51):
mm-hmm .

Dr. Jasmine B. MacDonald (36:51):
...and the more I am working across different tasks, I have post-its and piles of paper just everywhere.

Dr. Nicole Sugden (36:58):
Mm-hmm .

Dr. Jasmine B. MacDonald (36:59):
Which is okay if I'm in my office, but it's not okay if I need to remember when I'm outta the house or something.

Dr. Nicole Sugden (37:05):
Mm-hmm .

Dr. Jasmine B. MacDonald (37:05):
So yeah. Thinking about what's gonna be a useful aid at what time and in what location.

Dr. Nicole Sugden (37:11):
Yeah. And that reminder's gotta pop up at the right time, at the right place. So there's no point having a diary and writing it down if you leave your diary home. So yeah. You've gotta have a strategy to actually get that at the right time.

Dr. Jasmine B. MacDonald (37:22):
Right. And smartphones will often have that feature of remind me when I get here, if you program certain locations like work or whatever, which is now just making me realize they must have prospective memory psychological researchers on their team to be adding things like that.

Dr. Nicole Sugden (37:36):
Probably there's actually some really cool apps coming out now looking at prospective memory. Um, and they've been trialing them in people with brain injury. So they've location based reminders and yeah. Having quite a bit of success with them, which is cool.

Dr. Jasmine B. MacDonald (37:49):
Yeah. Awesome. The thing that we've looked at today is the psychometric aspects of psychological assessment tools and your kind of interest in and niche area is prospective memory and, and neurological assessments really, but you're quite, um, diverse. And so I wanted to check in and say, what else are you working on? Or is there anything else you would be wanting to make listeners aware of so they can look out for it?

Dr. Nicole Sugden (38:13):
So, yeah, I've got a few things that I've been working on. One is related to that TOPF research. So that test of premorbid functioning. So we're actually looking at the Flynn effect.

Dr. Jasmine B. MacDonald (38:22):
Interesting. Earlier on when you mentioned that some of these norms are, you know, from 2009, I was wondering, "is that long enough for the Flynn effect to have been impactful on this?"

Dr. Nicole Sugden (38:34):
Mm-hmm .

Dr. Jasmine B. MacDonald (38:34):
So yeah, please tell us about it.

Dr. Nicole Sugden (38:36):
So the Flynn effect is this finding that every decade our IQ increases by about three points. So we're actually getting smarter over time. If you completed an IQ test back in 1970, you would've got a way lower score than what someone would get now. So that raised the question with these IQ tests. We do need to keep updating them because maybe we are smarter. And these equations that we're using to predict IQ aren't as relevant as they used to be. So we've actually looked at, so the TOPF, which was 2009, but also word reading tests that were developed with earlier versions of the Wechsler intelligence scale. So there was the national adult reading test, which I think was developed in, I'm gonna say 1981 or 1982. And the Wechsler test of adult reading, which I think was in the nineties as well. Some of those are still in use, but the question is, should you be using these because of this Flynn effect? So how far away are these from, you know, if you did the proper TOPF now or used norms from now, would they be different and yep we're finding that it probably does have a little bit of an effect, although it does look like the Flynn effect could be leveling out a little bit, which is interesting as well.

Dr. Jasmine B. MacDonald (39:44):
Oh really? Okay.

Dr. Nicole Sugden (39:45):
So we have been getting smarter, but maybe we're slowing down

Dr. Jasmine B. MacDonald (39:49):
Cause we put everything in our phones to remember.

Dr. Nicole Sugden (39:51):
Probably, that's probably it, you know.

Dr. Jasmine B. MacDonald (39:56):
That sounds awesome. I'm really keen to read that.

Dr. Nicole Sugden (39:59):
Yeah.

Dr. Jasmine B. MacDonald (40:00):
Anything else?

Dr. Nicole Sugden (40:01):
Also working on another psychometric paper, the hospital anxiety and depression scale. So this is a 14 item test of depression and anxiety. Um, and that was developed sometime in the eighties and yeah, when they developed that one, they had, you know, scoring, so seven items are depression, seven are anxiety. Uh, what we are finding is that, we've done these things called factor analysis where you kind of cluster items that are measuring the same thing. And some of them are not measuring anxiety and depression in the way that they predicted. So we're finding this third factor, which is a lot of the anxiety items are actually measuring this kind of restlessness or psychomotor agitation kind of factor. So yeah, kind of at the moment, I'm still working on that and running some analyses, but yeah, it looks like, you know, maybe some of those anxiety items should not be going on on anxiety scale for the clinician to add up and say, yep, those seven items, measure anxiety, maybe three measure anxiety but the other four are more psychomotor. So yeah, watch this space on that one.

Dr. Jasmine B. MacDonald (41:03):
Yeah, cool. When these things come out or you've got references for them, let me know. I'll pop them in the show notes so that listeners can...

Dr. Nicole Sugden (41:09):
That'd be great.

Dr. Jasmine B. MacDonald (41:09):
...keep up to date. Amazing.

Dr. Nicole Sugden (41:11):
Yeah. Great.

Dr. Jasmine B. MacDonald (41:11):
So in the meantime, or say they look in the show notes and they're really interested in what you're doing...

Dr. Nicole Sugden (41:16):
Mm-hmm .

Dr. Jasmine B. MacDonald (41:16):
... where can they find more info information about you and the work that you do?

Dr. Nicole Sugden (41:20):
I have just joined Twitter. Can't say I'm great at it yet, but I am on there. um...

Dr. Jasmine B. MacDonald (41:24):
Welcome.

Dr. Nicole Sugden (41:26):
Yep. So you got me into it. Um, yeah. So @sugdennicole. I'll put up any links to any articles that I have published on there or anything exciting happening in my research. And I'm also in Research Gate too. So that's a really good way to see who I'm collaborating with and what research is coming out. So I do keep those quite updated.

Dr. Jasmine B. MacDonald (41:46):
Yeah. That's awesome. The reason I like the combination of the two is Research Gate's really good to list the things, you know, your, your publications and if possible, full versions of the articles, but we can't always do that, but Twitter is so cool to interact with other people and whether they're academics or psychologists or not. So yeah, if you're listening and you wanna reach out, we love it. .

Dr. Nicole Sugden (42:09):
Yeah, we do. Even if you like it and to say, thank you, we had one on an article we published together and it was so nice just to have someone read the article that we'd spent, you know, years working on. And they've just said, "thank you, we enjoyed that". Which is really nice.

Dr. Jasmine B. MacDonald (42:21):
Yeah. That connection, I think is always important, but especially right now, as we're all, because of various reasons isolated or in lockdowns and things, so human connection. Very good. We do like.

Dr. Nicole Sugden (42:33):
And I also find if, you know, someone follows you, then you kind of look them up as well and go, oh cool. I didn't know them. They're working in that area too. I'm gonna check out their research. So yeah. You kind of discover a lot more research as well, which I like.

Dr. Jasmine B. MacDonald (42:46):
Yeah, absolutely. And the kind of informal conversation about it, which we're trying to do here.

Dr. Nicole Sugden (42:51):
Yeah.

Dr. Jasmine B. MacDonald (42:51):
But it's just the two of us. So yeah, if you're listen and you, you've enjoyed the episode or you've got comments or questions, um, let us know.

Dr. Nicole Sugden (42:58):
Yeah.

Dr. Jasmine B. MacDonald (42:59):
Lovely. What about when you are not researching Nicole or when you're not teaching biopsychology or other things, what do you do with yourself?

Dr. Nicole Sugden (43:08):
Uh, like we talked about earlier, I love to travel. So, uh, I used to be overseas travel, but now it's obviously a lot more closer to the home. Yeah. A bit of camping. Um, I've got four dogs, so I love to get out with them and take them for walks and just spend time with them. Yeah. They take up a lot of my time, which is lovely. Yeah. Just spending time with friends going out for coffee. Yeah. Reading. I am in a book club, so we tend to catch up every month and talk about books for about five minutes while we catch up for the rest of the hour or two and yeah. It's good. Fun.

Dr. Jasmine B. MacDonald (43:40):
It's reminding me of, um, study dates when I was in undergrad. .

Dr. Nicole Sugden (43:43):
mm-hmm Yep.

Dr. Jasmine B. MacDonald (43:46):
Um, do you take your dogs camping?

Dr. Nicole Sugden (43:48):
I do. They love it. Yeah. I've got a golden retriever named Lily and she is obsessed with the water. So yeah, they just go a crazy, usually go camping for a week. And you know, in the first hour they run themselves completely ragged thinking that they're only gonna be there for an hour and yeah, by day seven, they're completely wrecked because they've just been having so much fun.

Dr. Jasmine B. MacDonald (44:09):
My dog does this thing where, his name is Taz, and he's a staffy...

Dr. Nicole Sugden (44:12):
Mm-hmm .

Dr. Jasmine B. MacDonald (44:12):
...he does this thing where...

Dr. Nicole Sugden (44:12):
Yep, I've got one of them too.

Dr. Jasmine B. MacDonald (44:12):
Aw, beautiful dogs. As soon as we get to a camping spot where we're gonna set up for however many days or a week or something, he'll just sit by the car, like, "all right, time to go". He'll do that for the first kind of 24 hours. Like, "all right, I'm done. This is what a trip is". And then he totally grooves into it and, and explores and loves it after that. But I always love how he's trying to train us, like "okay time to go home now". .

Dr. Nicole Sugden (44:39):
Nup, he's not the boss. .

Dr. Jasmine B. MacDonald (44:43):
Increasingly thinks that he is.

Dr. Nicole Sugden (44:44):
Yes, mine are like that too.

Dr. Jasmine B. MacDonald (44:47):
Nicole, I have had an absolute blast catching up with you today. Thank you so much.

Dr. Nicole Sugden (44:52):
This has been fun. Thank you.

Dr. Jasmine B. MacDonald (44:57):
For those of you at home, that's all for today. Show notes for the episode can be found at www.psychattack.com. If you've enjoyed listening to Psych Attack, please rate it on your favorite podcast platform and share this episode to help other people find the show. If you have questions or feedback, you can reach out on Twitter: @psychattackcast. Thanks for listening. And we'll catch up with you again next time.