Our friends over at The National Psychologist alerted us to a change in eBay’s policy about selling “protected” psychological tests — they now allow the sale (and resale) of such tests, whereas before this policy change they did not.
Their reasoning? Test publishers have not shown any basis, other than pure speculation, that making such tests available for sale could cause the public harm.
Many psychological tests are “protected” or “secured,” meaning they are originally sold only to certain qualified professionals (e.g., licensed professionals or researchers), with proof of their qualification. This is done to protect the integrity of the test. The more people know about a specific test (say, one of the IQ tests commonly used, or the MMPI-2), so the belief goes, the more they will be able to manipulate the test’s results (e.g., artificially inflating one’s IQ or making one’s personality look more “normal”). We see this in the test-retest psychometrics of a test. Take the same test enough times and you’ll begin to learn how to manipulate it (if that’s your intention).
Some psychological testing could lose its validity and usefulness if the results of certain tests become widely or easily manipulated.
Psychology is full of these odd kinds of paradoxes that plague the profession. Psychology generally prides itself on its research heritage and its emphasis on empirical data. Long before physicians and hospitals were talking about “evidence-based medicine,” psychology was founded in looking for rational, scientific explanations for human behavior (and subsequently, treatments for irrational or disordered behavior).
For instance, on one hand psychology prides itself on professionals attaining the doctoral degree, but there’s little empirical support that this higher degree (over something like a Master’s) results in better client outcomes. In other words, research shows that generally the type of degree (and in some cases, even the level of experience) makes little difference in whether people will get better in psychotherapy or not. Yet the psychology profession continues to hold out the doctoral degree as the necessary level of education in order to counsel people, even though the evidence clearly shows it is unnecessary for good client outcomes.
So eBay is asking a legitimate question — show us the research that suggests making psychological testing available to anyone actually causes harm. Otherwise, the same protections that apply to any other copyrighted material that legitimately exchanges hands applies. Because eBay isn’t in the business of being concerned about psychological test validity — it’s in the business of giving people a free marketplace to exchange goods.
Naturally, such data, in empirical form anyway, doesn’t exist. So what do you find on Page 5 of The National Psychologist‘s Jan./Feb. 2008 issue? A huge full-page ad from The Psychological Corporation (one of the largest psychological test publishers) begging clinicians to send them examples (case studies?) of where such public availability of a test resulted in a client’s harm. Not exactly the typical way one looks for harm and a threat to public safety, but hey, at least they’re doing their best to protect their interests.
And that’s the point. Make no mistake about it, this is about test publishers’ self interests (and revenues), not patient safety. eBay is perfectly within its rights to say, “Look, this isn’t our job to police your copyrighted material legally purchased and now being resold. There are hundreds of thousands of these things in distribution and we’re not the ‘test police.'” The more professionals who buy used tests (rather than shiny new ones), the less money these publishers make.
So while I’m all for the continuing protection of psychological testing, I’m against the restriction of free-trade amongst individuals under the guise of “protecting patients from harm.” It’s a red herring and test publishers should be ashamed of themselves for using it to suggest such sales are detrimental to the general public’s safety.
31 comments
I’ve been seeing some psychological tests for sale on eBay in the past (I’m always on the lookout for a used WAIS-III) so I’m not so sure this policy change make much of a difference. The “harm” argument may be the wrong one to use to discourage the practice of selling tests. Speaking as a forensic psychologist, there is a greater danger in making tests available so that people can learn to “fake” standard tests. It’s already an issue with tests of malingering and psychopathy and may become one with tests of personality in general.
I have to admit that trying to get eBay to stop all sales of tests does seem like protecting the interests of the testing company as opposed to say a retiring professional selling materials to an early career Psychologist for example. I think it would be best for the person listing the item though to be sure the individual is qualified. anyway, as a totally random aside- like my new domain? 😉 http://www.wais-v.com
Half glass full or empty? Recently a counselor at http://www.Goodtherapy.org New York’s blog stated “a person who seeks testing is most likely interested in his or her own health and would be seeking such tools to support his or her therapy needs going forward.” If someone is seeking to subvert the process, limited the sale of such tests will not keep the individual from achieving his or her pursuit. Test publishers merely have their own “selfish” interest in mind.
One situation in which this could result in harm is when psychological screening tests are used to screen against unsuitable applicants for such jobs as being a prison guard, being in the army, and so on. If an applicant learns how to score the test such that the questions are transparent to them then the test uses its validity with respect to screening out unsuitable applicants, yes. (We might have issues with respect to how predictive the tests actually are – but I think that you grant the tests some validity.)
Another situation in which this could result in harm is when psychological screening tests are used to provide evidence of diagnosis in court (e.g., tests for dissociation, psychosis, and so on). If people learned how to answer the questions that are not transparent (maniplate the liar scales and so on) then the consequences could be harmful, yes. They could be harmful in the sense that people who have studied them may feel they are relatively immune to the legal consequences of their actions.
Of course I understand that there is a degree of professional pride in this, as well. Psychological tests are guarded by psychologists in much the way that prescribing rights are guarded by medical doctors.
With respect to this:
> on one hand psychology prides itself on professionals attaining the doctoral degree, but there’s little empirical support that this higher degree (over something like a Master’s) results in better client outcomes.
The problem is that there really is little evidence that something like a Master’s results in better client outcomes to a 6 week course in counseling at ones local community college.
I do hear what you are saying with respect to the Doctor of Psychology / Doctor of Philosophy in Psychology issue (the political aspects, I mean)… But I’m not sure this is the way to set that right…
This was useful information. As an academic librarian I’m always being asked by graduate students how they can find a copy of a test cited in their literature searches. It’s frustrating to continually have to tell them that such tests generally are not available.
Er… Actually, in universities that have a clinical psychology department I think that you will find that the tests ARE freely available to people who want / need them for research – in the psychology department psychological testing library.
I’ve heard the psychological testing librarian say that she was forever getting requests from people who were going to have to do psychological screening whether they could borrow the tests. You know, prison guards and the like. Maybe she should just refer them on to Amazon, too?
Something that is restricted to university students or faculty of a specific department hardly makes them “freely available” to the general public, no? I mean, that’s always been the case, but most people are not students or faculty of universities. 😉
I did do a wee chuckle at this…!
I don’t know if there is any ‘evidence’ that tests should be restricted.
As one commentator above says, it is how results from testing are used (to discriminate, or label for example) not that test takers will ‘fake’ or otherwise manipulate the tests. There are huge numbers of books and websites that purport to teach people how to manipulate tests to ‘get the best results’…
It’s not the sale of psychological tests that worries me, my concern is about standards for *interpreting* test results. But the problem is that any result out of context can be misused.
I have no problem with people being able to purchase psychological tests – but I do have a problem with people misusing and misinterpreting the results. I don’t think restricting sales of tests is going to stop *that* problem!
Since psychology depends on the way that we perceive our lives, our surroundings and our emotions there are many things that need to be taken into account when you are studying people. So when you elect to study psychology you should be aware that it contains many facets of humanity itself. Therefore never rush to make assumptions and judgments – for just as you are studying the way other people react in society so too can some one else be studying you.
My understanding is that people are attempting to make it the case that psychological tests are NOT freely available to the public. I’m generally supportive of that goal.
What I was trying to point out was that one CAN access the tests for research purposes. If one is a graduate student and one requires the tests for ones research then the process is extremely straightforward (one simply borrows them from the psychological testing library). Graduate students in the psychology department should know this, so I’m very surprised that they would approach the universities general library in search of them, and I’m very surprised that the universities general librarians wouldn’t know to tell people to go to the psychological testing library.
With respect to undergraduates or non-university people (e.g., members of the general public) who would like to access the tests for research purposes – the process is still fairly straightforward. In my university, for example, the prospective borrower would meet with a person and say why they wanted the tests. The tests were not freely available to people who wanted to:
– Study the intelligence tests in order to boost their score
– Borrow the intelligence tests because they thought it would be fun to intelligence test their friends during a cocktail party they were planning
– Learn how to answer the test such that one would obtain a post in the prison system / army / and so on
– Diagnose themselves
Because the thought is that those later things ARE harmful. Harmful to the validity of the tests, yes. And also harmful to society when people who the tests are designed to rule out of taking certain positions of power get positions of power by manipulating their test result.
The university took an extremely liberal view of research (one certainly didn’t need to be in a formal research environment for ones research to be considered legitimate by them). But they did take an extremely hard line on NOT lending to people who were attempting to misuse them in the above mentioned ways. I’m generally supportive of that.
Good points…
But none of those examples could cause actual harm to an *individual*. Because eBay isn’t looking for hypothetical examples of possible “harm to society,” but to individuals.
After all, IQ is a theoretical construct. How can this theoretical construct’s score cause actual harm to a person?
Lets imagine that well intentioned (but severely misguided) mummy and daddy are concerned that little tommy might be developmentally disabled. mummy and daddy decide to obtain an intelligence test in order to establish this for themselves, rather than taking little tommy along to get tested by a qualified professional.
the result that mummy and daddy gets doesn’t mean a great deal. how come? because mummy and daddy haven’t received the (considerable) training that is required in order to properly administer and interpret the results of the test. have you seen the size of the manuals that come with the MMPI (for example)? also… it isn’t just about VERBALLY learning how to administer the test appropriately, it is an ABILITY that needs to be practiced over and over until one acquires some mastery of it (like hitting a golf or tennis ball properly). Or… delivering CBT.
the result of this is that mummy and daddy will obtain a score that is likely very different from the score that a trained professional would have obtained out of little tommy. unfortunately, the tests results aren’t at all valid when mummy and daddy do their own idiosyncratic version of testing. the test results are only have the validity they do when they are conducted in the standardized way. if mummy and daddy were to find that they should be worried about little tommy and they take him along to get tested properly the professional probably won’t be able to get a valid result – because mummy and daddy have familiarised tommy with the test already.
and if tommy doesn’t get a valid result he might not get the assistance he really does require (e.g., if he does better in the test under standardised conditions in virtue of his practice run with mummy and daddy).
same goes for self diagnosis attempts (e.g., a person obtaining a copy of the MMPI or DES or whatever and learning the scoring enough to score themselves). by learning how to score the test the test is now unusable on them (the results won’t be valid).
i always tell clinicians who suggest testing me that i know too much about scoring for my results to be valid. they will only test me if it is required by the health service.
so… while ‘IQ’ is a theoretical construct artificial inflation of ones IQ score can be harmful to persons when it results in a person not getting the treatment that they actually need…
Similarly, a person with sadistic tendencies who manipulates the screening test such that they get a job in the prison is likely to harm the people they are supposed to be looking after…
And a person who attempts a little ‘self diagnosis’ f’s things up such that it makes it much much harder for others to diagnose them, too.
Great hypotheticals, again, but no actual evidence that any of this has ever happened or would likely happen in the future if such tests were widely available.
It’s also a great argument against any type of self-assessment in general, or actually learning anything about a disorder, because one might “self diagnose” as having (or worse, not having) a specific concern, and then not seek out treatment for it.
But I’d rather err on the side of self-knowledge, in general, and the freedom of markets unless evidence of harm is apparent or proved. In the case of psychological assessment, it’s all been hypothetical harm, without a single case study ever published suggesting otherwise. And that’s the problem — without the data, it would be ridiculous to suggest there’s a problem. Only real, live data can tell you that…
As in most complex decisions there is unlikely to be hard data to validate a decision, but it doesn’t stop us from making those decisions. Another approach is balancing interests. I believe that is in the public interest generally to restrict access to psychological test materials, not because they are copyrighted which is another argument, but because dissemination of these tests means that the hypothesis of a naive test taker can become untenable. The tests, when used properly, can and do contribute to accomplishing ends such facilitating treatment of the mental illness, accurately and objectively assessing individuals for jobs and contributing to fair and just legal outcomes for example. The degree to which tests contribute to such ends is debatable of course, but it is reasonable to assume at least some positive benefits. The risk that any particular test taker’s performance in the test is spoiled as it were by such dissemination is likely quite low, however it is a risk and there is evidence that this takes place in some important contexts such as forensic evaluations. These concerns must be balanced against the broad concerns about restricting a free market for sales. (The cynical argument that restricting such sales only benefits the greedy psych testing industry is a side issue and I have bones to pick about aspects of that myself). While restrictions on selling these items on the free market clearly interferes with free trade, what is the harm done there? Harm to the idea of completely free trade of all goods at all times to all people? We do not have that now. I would argue that we accept restriction of trade in a wide range of situations where there is demonstrated harm to other social interests, but also where the harm is hypothetical or even theoretical or where it is a widely accepted social norm that it is likely to be harmful. Restricting trade in this narrow situation seems unlikely to do serious damage to a large class of people or the general idea that trade should be largely unfettered and governed by market forces. Finally, the are numerous ways to sell the tests on the ‘free market’ by psychologists or other qualified persons who will respect the professional interest in taking care that the tests be sold only to those who will abide by at least some shared interest in test security.
I guess my point is that, as psychologists, we seek to make decisions and base our treatments on empirical research. We’re scientist-practitioners. We get the data, then we act on that data.
Too many times, however, psychologists flip the script and do things one way for a long time, and then seek data to support their way of doing things. This type of self-selection bias is not typically a good way to do research, or to be scientific.
It is incumbent upon psychologists to first prove that having such tests available to the public is somehow harmful. That would be the ordinary expectation, especially as profession based in science. We would scoff if someone published a book and said, “The contents of this book would be harmful if you were allowed to read it. Therefore you have to belong to the Secret Society Club in order to read it.” I know some organized religions that do this sort of thing… And I don’t think psychology wants to take its cues from organized religion.
Since psychological tests haven’t been publicly available I’m supposing it would be hard to find evidence of the sort amazon seems to be requiring. I’m not quite sure why amazon is setting the bar on ‘evidence’ in quite the way it is, however. We don’t have evidence that if the speed limit was precisely twice what it is now there would be more accidents (nobody has done a study where they manipulated the speed limit in such a way) and yet since there is GOOD REASON to believe it would be harmful it really is unclear why we would require such evidence in order to reasonably conclude that raising the speed limit in such a way would be a bad idea.
There is evidence that the tests lose their validity the more people take them. This is precisely why psychologists safeguard the tests – so they don’t become common knowledge, and so that they retain whatever validity they have. A person might not like the fact that the tests do have some validity and they might support us altering our social practices such that the tests lose the validity that they do have, but I guess I’m unclear on why a person would attempt to undermine the tests *in this way*.
There are indeed problems with self-assessment. That being said the majority of self-assessment rating scales that are available online are either 1) Not valid measures of anything much or are 2) Aimed at being over-inclusive such that MORE people seek out professional help rather than LESS. As for the self-diagnostics, I think they tend to harm more than help, but do what you will with most of them since they aren’t valid measures of anything much at any rate.
Show me the proof (of the sort that amazon requires) that yelling ‘bomb’ on a plane or ‘fire’ in a movie theatre is harmful. We do rely on psychological testing for determining who is and who is not likely to profit from certain kinds of treatment. We do rely on psychological testing for screening out potentially grossly unsuitable job applicants. One might be concerned about the degree of validity that the tests currently have (I really do think that is a legitimate concern) but I really can’t at all see what good will come of psychological tests (proper ones) being made available to people who don’t know how to score them (probably aren’t interested in studying them for several months in order to learn how to score them) and don’t know how to administer them (and can’t simply ‘teach themself’ that).
Whereas we know already that such a move will render the tests less valid (hence less useful for the things they are currently used for).
I wouldn’t be surprised if the American Psychological Association developed some professional guidelines on who one was and was not allowed to sell the tests to. Basically prohibiting psychologists from selling them onto people who are not authorised to administer / score them.
What about making the current version of the Graduate Record Exam or University Entrance Exams, or Medical Exams available to all over Amazon? Why wouldn’t we want that to happen? (People would qualify for things they shouldn’t, and would be exluded from things they shouldn’t). Do we really need to do a study to see the harm that is likely to result from this?
Virtually every example you gave actually does have evidence to support the laws we have in the U.S. today. There are a great many highway traffic safety studies that show the faster you go, the more dangerous (e.g., life-threatening) an accident will be. The shouting of “fire” in a crowded is the hypothetical basis for the Supreme Court’s “clear and present danger” doctrine. Psychological tests hardly reach the level of a “clear and present danger” if misused.
It completely depends on the test as to how “invalid” retesting is. In most cases, it doesn’t make the subsequent tests lest “valid,” it’s just something the test report needs to take into account. There are actual statistical corrections for some tests when retaking the test, so such scores can be statistically corrected.
Psychologists aren’t the only ones who give psychological assessments, surprisingly. And only about half the psychologists in the U.S. belong to the APA, meaning that whatever they do doesn’t mean a thing to 50% of psychologists.
As for the GRE, SAT and the like, you have a point… But you’re mistaken if you think they are not already readily available in the form of test prep courses (that simply take prior years’ test questions and given them as “sample questions”). Why do you think there’s such a prevalence and popularity of such prep courses? If taking and retaking a very similar test for the SATs or GREs was “harmful,” then such courses would be outlawed as well, since they have a proven track record of helping a person score better on their respective tests.
I’m not for the widespread use of complex psychological testing by ordinary folks (although the MBTI is taken all the time by ordinary folks and it doesn’t seem to invalidate it as a measure — in fact, most people who take it really gain something beneficial from the experience), but I am against the infringement of free commerce without cause. And the cause cited by the test publishers simply has no basis.
Yeah.
If I was a member of the APA I would tell them not to buy into the ‘find a particular case’ challenge. That is likely to lead to the ‘yes it is a good case’, ‘no it isn’t’ move.
Instead… They would be better off saying that they want their ‘tests’ to be granted the same status as other ‘tests’ (like those that appear in teachers editions – as they stated and also) like the GRE, GMAT, etc etc etc. For… Similar reasons (tests are used as diagnostics for entry into various things and their validity needs to be protected).
Sorry – we cross posted.
> There are a great many highway traffic safety studies that show the faster you go, the more dangerous (e.g., life-threatening) an accident will be.
What I was (attempting and probably failing) to get at was the difference between an observation (one kind of evidence) and an inference from an observation (another kind of evidence) – though something is more or less obervational / inferential rather than it being an all or nothing matter.
The notion was that nobody (to the best of my knowledge) has run the PARTICULAR experiment that would show whether PRECISELY doubling the speed limit would result in more crashes. Of course we do indeed have other experiments (that don’t test precisely that) that are related enough for us to REASONABLY INFER that precisely doubling the speed limit is likely to harm.
> The shouting of “fire†in a crowded is the hypothetical basis for the Supreme Court’s “clear and present danger†doctrine.
I’m not sure what the doctrine is… Is it that shouting “fire” inappropriately constitutes a clear and present danger? If so… then do you think you could find a clear cut case where someones yelling that constituted one, or is it more that ‘obviousness’ is appealed to?
> There are actual statistical corrections for some tests when retaking the test, so such scores can be statistically corrected.
Okay. I didn’t know that. I’m not sure that there are statistical corrections for someone who has learned how to score the test, though.
With respect to the GRE etc I know there are PAST YEARS versions available online. What there aren’t, though, is the CURRENT YEARS versions. The APA could conceed that amazon can sell PAST YEARS versions of psychological tests (that aren’t currently used) while requesting that they not sell CURRENT YEARS versions.
> If taking and retaking a very similar test for the SATs or GREs was “harmful,†then such courses would be outlawed as well, since they have a proven track record of helping a person score better on their respective tests.
It has become a fitness trap. In order to get accepted you fairly much NEED to study the tests. If you don’t study the tests then you will be putting yourself at a severe disadvantage, because most other people will have. That serves its own function: You can rely on people who score well / who get a place having mastered those cognitive abilities. For example… In order to get a place in a US PhD program I would have had to be competent in mathematics. Even if I wanted to study… English literature.
MBTI?
With psychological testing being increasingly used as part of civil and criminal liability cases, I would think that the potential for harm if the tests were freely available to the public was clear. Malingering is a significant issue in this field, and if people are able to see how tests are scored they will be able to more effectively malinger. This will not directly harm the person who obtained the test, but it could significantly harm the defendent. It’s not realistic to expect clinicians to come forward about this type of abuse – a good malingerer will not be detectable by definition.
Also, there is the problem of bogus professionals. As it stands, the high cost of the tests coupled with the requirement to prove professional qualifications puts some limits on who can legitimately access the tests (they’re often bought by organisations, not individuals). People without appropriate qualifications may pose as professionals if they have access to all the same tools as professionals use, with the potential for harm being clear. People who use the tests need to do so within the framework of statutory responsibilities and appropriate supervision – these checks are bypassed by selling direct to the public.
I think there are valid arguments in regards to protecting the public, however shutting down all testing sales on eBay goes above and beyond what would likely be necessary to protect the public- assuming there is some mechanism in place to allow for the selling individual to refuse the winning bid of a non-qualified person (license searches are easier and easier with growing technology on the web). If the testing companies wanted to be serious about protecting others and not just shutting down a used testing “aftermarket” that might cut in to their bottom line, perhaps they could consider creating a “safe” used testing exchange on their website where one professional could unload the used test they no longer need to another qualified individual- kind of like sports teams now due for season ticket holders selling tickets they won’t use, etc. They can charge a minimal fee competitive with eBay (though sadly this might prevent the sale of a new testing kit from their site). I suppose perhaps I am too trusting that all eBayyers with access to psychological testing are subject to ethical codes and/or are going to try and protect the test, etc. While I’ve never seen any instance where this didn’t happen (doesn’t mean much)- I’m not sure how many are known in general and thus the press release begging for people to come up with examples 😉
Amusingly, I just went to eBay to do a little searching as a result of the press releases, article, etc: and this one is just sick 😉 who puts Rorschach card images on coasters?
http://cgi.ebay.com/4-Rorschach-ink-blot-COASTERS-counselling-psychology_W0QQitemZ110218239833QQihZ001QQcategoryZ13907QQssPageNameZWDVWQQrdZ1QQcmdZViewItem
A highly recommended site is elfingo.com for online auctions. They are the new ebay. Many smaller sites like this offer buyers far better deals than ebay ever could. Buyser also save a ton because this site charges little or nothing depending on the day. One more reaso I like elfingo.com is because they don’t take a part of the sale at all. No commissions or final value fees. A+++ http://www.elfingo.com
I think the argument on the part of the test publishers is about protecting their monopoly. There should be a way for psychologists to legitimately sell and purchase used testing materials. It’s wasteful to have 1000’s of unused test kits lying in closets because clinicians purchased kits and gave up assessment work. Assessment materials are expensive and it is costly for new clinicians beginning their practices to build up batteries that might cost 1000’s of dollars new. The argument about test security is specious. The publishers currently maintain a very low standard for securing and checking qualifications before selling their materials–virtually anyone could fake credentials now if they really were all that intent on it. Opening up a market on eBay is not going to change this. The APA ethical standards for psychologists should be enough to safeguard test security. Checking credentials before reselling should be the ethical responsibility of the reseller.
I am a school psychologist, Marriage Family Therapist, being asked to do independent contract testing. I would love to purchase used test kits vs. spending several thousands of dollars if it is at all possible.
Please let me know if/from where I might purchase used but current versions of test kits. I would be so grateful ~ Thank-you for your time.
Aside from persons in the psychology community (most of whom would be “qualified”), I don’t believe there’d be much demand for a used test kit (a WAIS-III, for example), costing $300-$600, on Ebay or anywhere else. The price is a substantial barrier that can’t be overcome unless someone were willing to practice clinical psychology without a license. If the test publishers take out full-page ads concerning this, they’re wasting thier money and drawing attention to something that should probably be ignored (at least until we see widespread quack psychology testing rings somewhere).
I’m an early-career psychologist, and have been looking for a website that sells used test kits. Since I’m early career, I just can’t afford brand new kits. If anyone knows of a place that sells used test kits, *please* drop me a line at [email protected].
Thanks!
http://myworld.ebay.com/valence17/
FYI- i sell used psych testing items at the address above. I hopefully will be setting up my own site in 2-3 months.
Peace.
This is a good article on psychology, i impressed with this.
thank you
I appreciate all the dialogue about this issue! I have some tests to sell and am scared to do it on ebay because of the ethics code that states that we protect test security. Is there someone qualified to used and administer the WMS-IV and WAIS-IV that would be interested in purchasing them from me? I am leaving private practice. Email me at [email protected].