Does Adaptive Exam Devalue the CISSP?

[gphalpin]

Hi Everyone, 

This may have been covered a few months ago when the news was first announced, but I just recently learned the CISSP exam became adaptive. Colleagues asked me about my exam experience. That was three years ago when the exam was up to 6 hours long and 250 questions. So I showed them the web site to go over the domains, exam info, etc. 

I was very surprised to see the exam (English language) is now 100-150 questions. While the material is still demanding, I think the CISSP had a strong reputation as the premier information security certification because it was so rigorous with 250 questions. It was a long, tough exam. And people respected (sometimes grudgingly) those who passed. 

At 100-150 questions, does this devalue the CISSP? The Security+ is 90 questions. People used to believe the CISSP was several notches above Security+. Now people might think the CISSP is just one notch above or lump them together.  

I'm not trying to take anything away from those who passed the adaptive exam. I'm concerned about the long term implications this has on the value of the CISSP certification in the eyes of IT security professionals.  

Thanks,

Greg

[Jaswhee]

Greg,

I would invite you to read up on the CAT (Computer Adaptive Testing) . Basically the CAT in most cases is able to determine within 95% statistical confidence that you would be a pass or fail even if you had 250 questions. I do not believe that more candidates are passing as a result of the CAT. I personally know quite a few people who have failed the CAT.

Jason

https://www.isc2.org/Certifications/CISSP/CISSP-CAT

[perryjurancich]

I would suggest that if the fail rates go down, that indicates 1. the CAT algorithms are not doing a good job, or 2. the test is effectively easier thereby reducing (IMHO) the value to those who have passed.  I can tell you from several different types and levels of tests (CISSP, CCSP, GSEC, PMP, BlackBelt, etc) CISSP used to be a good challenge, hope it stays that way. 

Just my $.02

[CEMyers]

We should all bear in mind that quality is always a better yardstick than quantity. 

Were the exam to be a straight set of questions there might be stronger reason for doubt, but the adaptive nature of this framework is there specifically to address the quality question and to maintain the current high standard of the qualification. 

Nothing worthwhile is ever easy, remember that, when comparing with other qualifications. Also remember the qualification is not just about passing the exam but also about having the (appropriately recognised) experience to support the knowledge. Knowledge supported by experience for me is always the best form of assessment.  Remember the human race moves forward through learning from its failures as much as from its successes.

[MiniMe]

I am willing to bet a person, who thinks CAT makes CISSP exam easier, has never taken a CAT exam in their life.

I have recently taken the CISSP in CAT format. Having taken GRE when I applied for grad studies, I was familiar with the CAT concept.

If I had an option, I'd have taken the 6 hour linear exam where I could go over "A" number of very easy, "B" number of easy, "C" number of hard, and "D" number of very hard questions.

1. If you're not sure about an answer, you can skip the question... you do not have that luxury with CAT format.You have to answer the questions in the order presented and your answers are final.

2. More than often, in an exam with hundreds of questions, you end up inferring the answer of some previous questions just by reading the proceeding questions. Well, once you answer a question, you cannot go back in CAT format.

3. Going through a hard question may discourage you but all you need to do is reading the next question to gain your courage back. Well, in CAT format, if you answer a hard question correctly, you get a "harder" question. And if you get an easy question after a hard one in CAT, you know that you screwed up the previous one. 

4. You might be really good at time management and you can easily come up with a plan when you're taking a linear exam. Guess what, you have the same amount of time whether you're done at question 100 or need to answer 50 more in CAT format. 

Having said that, maybe it's a lot easier for some folks to take CAT exams... for me, linear tests are very predictable and a professional test taker can easily increase his/her chance of acing the test by preparing for the exam itself as well as the content of the exam.

Thanks,

JohnZ

[gphalpin]

Thanks for the responses, everyone. 

The question is not whether the exam is easier or harder. The question is whether the CISSP will have the same perceived value to other IT pros, IT security pros, and hiring managers now that it's a shorter exam?

Anyone who has a CISSP will know that the exam is still very hard. Years of experience and knowledge are necessary. It also takes many months to study for the exam. You have to be endorsed, etc. 

But people who are not yet familiar or as familiar with the CISSP may wonder why it stands out since it's not much longer than other exams. In fact, one of my colleagues who expressed interest it in said it looks a little harder than Security+ but not by much. I don't want the CISSP lumped in with Security+.

Again, I'm concerned the CISSP may just be another cert in five years, not the premier IT security certification.

[mgoblue93]

No.  What devalues the CISSP is the people who have it, take a title of cybersecurity engineer, and then do not do any engineering work.

My personal opinion is adaptive exams are harder.  Someone sitting for that test pretty much has to be mistake free.  In the previous test, one could miss a question and recover.  In adaptive exams, you miss a question and you're going to see more of the same material.  2 thumbs up to people who pass (any) adaptive exams!

In general, I don't think comparing old version to the new version's question count or their duration and somehow equating that to a degree of difficulty is applicable or productive to the community. 

I have witnessed in the past, people throw shade at others because their CISSP number is too high -- the assumption is they got into the game late or just because of the gov't and therefore aren't a real CISSP.  To me, I interpret some of this old test/new test thing as a continuation of the "your number is too high" silliness.  Adaptive tests are no joke.

[CISOScott]

I don't feel like it devalues it. Adaptive tests are supposed to ask progressively harder questions, so if you know about encryption and it's correct application to a certain situation, whether you get asked 3 questions or 30 doesn't necessarily make you more knowledgeable in that situation.

[Beads]

ISC(2) doesn't publish the pass rate so their is no way of understanding if the test is easier, harder or about the same. We simply don't have enough information to make that claim one way or another. I will direct our readership to the one clue we do have access - exam numbers.

This will be a bit controversial so if your easily offended please pass on reading below this line.

Consider the year and month you received your official membership number and the number of exams taken by all candidates before. This will become your baseline. Now advance a few years and think about those resumes with membership numbers from more recent members. Those numbers go up not down. If you have a really low number say in the 50,000s from 200x and compare to a very recent member with a membership over three-quarters of a million. Well, you can compile a pretty rational idea as to if people are passing more frequently than in the past.

Its just maths and some observation skill but I have confidence you can make an argument either way.

We have over 125,000 members and can remember when we hit 50,000. The later being a huge relief and celebratory moment for the organization.

[meincke]

I wonder about that too.

The exam was difficult for me. Not due to the difficulty of the questions, I was well prepared by the time I took the exam, I answered the vast majority of them with a high degree of confidence. I didn't rush, I read every question carefully, weighed each answer and came to a conclusion I was happy with. After answering each one at a time I had 11 minutes left on the clock. I didn't go back to review my answers, I was too tired, and I was reasonably sure that a review wouldn't turn up any better answers. 

I felt like a ran a mental marathon, I certainly had the knowledge to pass but there was no getting around 6 hours of testing. That was the hardest part. I feel like a CAT test would've been a breeze for me or anyone else who was well prepared.