Does Adaptive Exam Devalue the CISSP?

[gphalpin]

Hi Everyone, 

This may have been covered a few months ago when the news was first announced, but I just recently learned the CISSP exam became adaptive. Colleagues asked me about my exam experience. That was three years ago when the exam was up to 6 hours long and 250 questions. So I showed them the web site to go over the domains, exam info, etc. 

I was very surprised to see the exam (English language) is now 100-150 questions. While the material is still demanding, I think the CISSP had a strong reputation as the premier information security certification because it was so rigorous with 250 questions. It was a long, tough exam. And people respected (sometimes grudgingly) those who passed. 

At 100-150 questions, does this devalue the CISSP? The Security+ is 90 questions. People used to believe the CISSP was several notches above Security+. Now people might think the CISSP is just one notch above or lump them together.  

I'm not trying to take anything away from those who passed the adaptive exam. I'm concerned about the long term implications this has on the value of the CISSP certification in the eyes of IT security professionals.  

Thanks,

Greg

[denbesten]

---

@Beadswrote:

your official membership number ... compile a pretty rational idea as to if people are passing more frequently than in the past.

The member number is issued when one signs up for the web site and represents everyone who has one or more (ISC)² certificates, has ever scheduled an exam, or even might just be considering it.

You can use the member number to estimate the age of somebody's oldest cert and you can use it to gauge the overall level of interest in (ISC)² certification, but it does not help estimate how frequently people are passing the exam. You would be better off tracking the published member counts to estimate that.

[Lamont29]

The

---

@gphalpinwrote:

Hi Everyone, 

This may have been covered a few months ago when the news was first announced, but I just recently learned the CISSP exam became adaptive. Colleagues asked me about my exam experience. That was three years ago when the exam was up to 6 hours long and 250 questions. So I showed them the web site to go over the domains, exam info, etc. 

---

The biggest gripe I see online is with people who say that they studied for the 250 question linear exam but took the CAT exam instead and failed. You got 6 hours on the old format and 3 hours on the CAT format. On the old format, you can go back and check or correct something, on the CAT format, it's a 'one and done' deal. I guess because I know how CAT exams are statistically measured, I don't really get caught up in the 'watered down' theory. 

ISC2's mission is to maintain the integrity of the testing for CISSPs. If the CAT format made things 'easier' then I know that I would have never bothered to sit for the exam. The critical thinker may interject and ask critical questions such as: 'Why is it easier?' 'Has the person who made such a judgement taken both versions?' What's the failure rate between the CAT format and the linear exam?

Based on anecdotal evidence that I've encountered, there are even MORE disgruntled test failures now than when the linear-only format was in place.Just my thoughts.

[Lamont29]

---

@MiniMewrote:

Having said that, maybe it's a lot easier for some folks to take CAT exams... for me, linear tests are very predictable and a professional test taker can easily increase his/her chance of acing the test by preparing for the exam itself as well as the content of the exam.

Thanks,

JohnZ

---

JohnZ,

You have taken the words right out of my mouth! I try explaining this concept to people who are not professional students like we are and the message never seems to get over. Exhibit A: I have NEVER taken a linear test and failed - NEVER! On a linear exam, you can always brain dump. You can go back and recheck a question and change an answer. You can always get cues from other people who shared their experience with the linear test, and all of this aggregate information allows a person who is a professional test-taker to have a much easier time with linear exams - they can be 500-question tests and it wouldn't make a difference.

CAT examination all but nullifies the ability of brain-dump test taker success.

[Lamont29]

---

@denbestenwrote:

---

The member number is issued when one signs up for the web site and represents everyone who has one or more (ISC)² certificates, has ever scheduled an exam, or even might just be considering it.

You can use the member number to estimate the age of somebody's oldest cert and you can use it to gauge the overall level of interest in (ISC)² certification, but it does not help estimate how frequently people are passing the exam. You would be better off tracking the published member counts to estimate that.

---

Great observation and response...

[meincke]

CAT in no way nullifies brain dumping. CAT still draws from a pool of questions. If somebody gets their hands on the test bank then they can just as easily memorize the questions as they would with a traditional test. Only exam security and frequent rotation of questions can combat dumping.

[meincke]

Here's the thing I think proponents of CAT are missing; the 6-hour exam was a right of passage. The length of the exam was something to be respected. I could tell someone who didn't know anything about IT or security that I had to sit for a 6-hour test with only 90 seconds to spare between questions and they would genuinely be impressed. 

It's not nearly as impressive to say that your exam ended early because an algorithm determined you'd pass a longer test.

[Lamont29]

---

@meinckewrote:

CAT in no way nullifies brain dumping. CAT still draws from a pool of questions. If somebody gets their hands on the test bank then they can just as easily memorize the questions as they would with a traditional test. Only exam security and frequent rotation of questions can combat dumping.

---

I will add that I am in no way sure that the linear test was favorable to brain dumps because I never taken the linear exam. However, ALL of the people that I know who took the linear exam suggested to me that the test banks and brain dumps were viable resources for passing the linear exam when they took it. Maybe my experience might have been different. I do know that I have NEVER failed a linear exam (I am at 100%) so long as I studied similar questions from test banks - akin to brain dumps.I have on the other hand experienced difficulty on CAT exams, as they are in my opinion tougher than linear exams. You will NEVER find similar questions from a test engine or other resource on the CISSP CAT exam (based on my experience). Nearly every test question was based on my experience, which if that makes it easy then, I guess it was easy for me. I found the test to be uneventful. I have other colleagues who have taken this test several times without success - both CAT and linear format. Furthermore, ALL of the referenced individuals have earned advanced degrees.

Lastly, I don't know of anyone who passed both versions of the exam. In this case, this thread is for bravado and very opinion based.

[meincke]

---

---

@Lamont29wrote:

---

---

I will add that I am in no way sure that the linear test was favorable to brain dumps because I never taken the linear exam. However, ALL of the people that I know who took the linear exam suggested to me that the test banks and brain dumps were viable resources for passing the linear exam when they took it. Maybe my experience might have been different. I do know that I have NEVER failed a linear exam (I am at 100%) so long as I studied similar questions from test banks - akin to brain dumps.I have on the other hand experienced difficulty on CAT exams, as they are in my opinion tougher than linear exams. You will NEVER find similar questions from a test engine or other resource on the CISSP CAT exam (based on my experience). Nearly every test question was based on my experience, which if that makes it easy then, I guess it was easy for me. I found the test to be uneventful. I have other colleagues who have taken this test several times without success - both CAT and linear format. Furthermore, ALL of the referenced individuals have earned advanced degrees.

Lastly, I don't know of anyone who passed both versions of the exam. In this case, this thread is for bravado and very opinion based.

---

---

Completly anecdotal but I had the opposite response. People I've known who are open about dumping have failed the CISSP after trying to use a dump as their only study method.

A CAT isn't going to change this, the software isn't generating questions on the spot, there's still a pool of predetermined questions. The only difference is that the questions are ranked from easy to hard and you might not need to answer every question once you hit that magic number for a given category.     

[Lamont29]

---

@meinckewrote:

---

---

---

---

Completly anecdotal but I had the opposite response. People I've known who are open about dumping have failed the CISSP after trying to use a dump as their only study method.

A CAT isn't going to change this, the software isn't generating questions on the spot, there's still a pool of predetermined questions. The only difference is that the questions are ranked from easy to hard and you might not need to answer every question once you hit that magic number for a given category.     

---

I completely agree. Your anecdotes are just as plausible as the one I gave. Now, on the other hand (I was just thinking of this experience) when I was the deputy in charge of battalion operations, the operations officer had everyone working long hours. I mean 16-18 hours on most days. That person had to go on mission which left me in charge. I told my executive officer that I was going to add efficiencies to our processes and get everyone home at a decent hour. To make a long story short (see the pun?), I improved our battalion's overall performance and got everyone (including the commander) home in time for them to have dinner or a life with their families. So longer doesn't equal to smarter or better performance. My processes were adopted even after the senior operations officer came back. He may never tell me - but yes, I think he was happy as well.

I tend to lose interest after about an hour and a half of reading. Now, give me a couple of 200 mg caffeine pills and I will jump all day and into the night. So anyone sitting for three hours could have just as well sat for 6 hours. I did not use all 3 hours - and I would avoid that at all costs knowing that my best performance is going to be around 1 1/2 to 2 hours. But if I had to, there's would have been no cheating in me taking the caffeine pills if the exam would have stayed linear.

[denbesten]

@meinckewrote:

CAT in no way nullifies brain dumping. CAT still draws from a pool of questions. If somebody gets their hands on the test bank then they can just as easily memorize the questions as they would with a traditional test. Only exam security and frequent rotation of questions can combat dumping.

---

No question that brain dumping remains possible, but it is also less fruitful for one simple reason. Now they will only get 100 to 150 questions from the bank, whereas they linear test guaranteed them 250 questions.  So, the question bank is a bit more protected than it was.

There are a few more techniques that can be used to defend an exam, as this article describes.