CISSP Dilution

[SunnyDee]

Dear ISC2,

I hope you're well aware of the pass rate from the new CAT format. It has sky-rocketed. Comparing between the reddit/facebook/techexamforums, it's clearly apparent that more people are passing, daily, with minimal effort. What used to be, 3 months of hard work studying the entire CBK from head to toe along with thorough, comprehensive practice testing to attain this elite certification, others have been spending just a few weeks to pass the CISSP exam. I studied for the CISSP religiously, every day for 3 months to pass this exam. I worked hard for this so it pains to see others, with minimal effort, pass the same exam, at a higher frequency since the format changed to CAT.

And to no surprise, the word is getting out, everyone who has been studying for this is being told to go ahead and jump on it without wasting more time or before ISC2 realizes that they may just have made this exam significantly easier. The ease of attaining this cert now isn't fair to all those who studied, laboriously to attain it.

Sincerely,

Pre-CAT,Certified Information Systems Security Professional

[Kaity]

Hi @clementdupuis

Thanks for joining the conversation and sorry for the delay! As stated earlier in this thread, the change in format to the CISSP exam (from linear to CAT) has had no impact on the pass rate. However, it is (ISC)² policy to not publicly disclose exact pass rates.

(ISC)²’s transition of CISSP to CAT is an important investment in the future of its certification program. The implementation of CAT strengthens our commitment to meet the critical demand for cybersecurity professionals worldwide by providing a fair, valid, reliable, and efficient exam administration process. The CAT exam follows the same exam blueprint and contains the same percentage of items in each Domain on the test as in the linear exam.

Because the exam items are targeted to the ability of a test taker in each Domain, the information obtained from those items are much more precise and are used while making the pass/fail decision. CAT provides numerous benefits to candidates including:

• A more precise and efficient evaluation of a candidate’s competency
• More opportunities for examination administration
• Shorter test administration sessions
• Enhanced exam security.

As for your questions related to the CAT format, I’ve checked with our Exam team for more information. While it may seem counter-intuitive, it is very common for CAT exams to be shortened by as much as 50% (from a fixed, linear exam) while providing an even greater level of precision in measure a candidate’s competency. CAT has been around for decades, and science and data support it as a more precise and reliable exam format.

I hope this helps! 

[DAlexander]

Ms. Eagle,

Just my two cents on the matter:

The message has gotten out about the new CAT format through various social media sites, word of mouth, and other unofficial means.  Unfortunately, the most common message I've seen is that the exam is much shorter and easier - so now is the time to challenge the exam before (ISC)^2 catches on and reverses course.  I cannot personally attest to the difficulty level as I earned the CISSP on the old format but it's important to acknowledge that, many times, perception is reality. 

Very generally speaking, people do not do their own research to build an educated opinion on most matters.  Whichever 30-second, talking-head, sound byte is last heard is what is accepted as truth and major media outlets have mastered the ability to sway public opinion in this fashion.  I fear that employers will hear about the supposed easing of the CISSP and configure their HR filters to look for other "less attainable" sets of letters to in order to separate the wheat from the chaff.

Now, I am by no means implicating that a certification makes the professional but it is a major factor in what gets us past the hiring algorithms and a foot in the door for an interview.  I would strongly advocate that (ISC)^2 change its policy and publish their pass/fail rates.  In my opinion, this is the only way to prove to employers that the CISSP is truly the gold standard that it has been thought of for so long.  This is what credible higher learning organizations do.  For instance, you can find the acceptance and graduation rates of any quality university because it is important to distinguish their graduates from those that may have simply completed a curriculum from a degree-mill somewhere.

Another thing that may help your cause would be to cite some of the research that supports your statement, "science and data support it as a more precise and reliable exam format." 

Thank you for all you do!

[JoePete]

---

@DAlexander wrote:

 I would strongly advocate that (ISC)^2 change its policy and publish their pass/fail rates.  In my opinion, this is the only way to prove to employers that the CISSP is truly the gold standard that it has been thought of for so long.  This is what credible higher learning organizations do.  For instance, you can find the acceptance and graduation rates of any quality university because it is important to distinguish their graduates from those that may have simply completed a curriculum from a degree-mill somewhere.

---

But pass/fail and acceptance rates can be misleading. For example, the first generation or two of CISSPs probably had a high pass rate because the only people who knew about the CISSP were people established in the industry. Today, I suspect the rate may appear low because you have a lot of folks simply trying to chase a credential and salary. Just to wax curmudgeonly here - we have raised a generation (going on two now) of kids whose education has been geared toward standardized testing. They are probably disappointed - outright aghast - to find questions on the CISSP that weren't specifically outlined or asked in their study guides etc. The CISSP exam is supposed to measure an ability to apply a comprehensive body of knowledge AND experience, which means there should be questions and even topics not always seen in a prep guide etc. It should test that age-old ability to "know it out" and not just regurgitate content.

It's on that note that I share your concern about how the CAT impacts the CISSP because I see the CAT as bowing to an idol of regurgitation rather than mental acuity and stamina. Efficiency and security often travel in opposite directions. Do we really want to suggest that if you can "prove" knowledge in 10 questions - rather than 15 - those other 5 aren't necessary? Shortcuts are the anathema to security. 

From what I have read, a CAT format does not work in every environment. The CISSP is a comprehensive credential. For example, I believe the DCO (detailed content outline) for the CISSP (can't find a link to it any more) has at least 250 topics. If we are trying to ascertain whether someone has a mastery of the CISSP CBK, you'd think a standard exam would need a commensurate number of questions at least. However, with the CAT being 150 questions and people being able to pass by answering as few as 100, those figures seem incongruous with the CBK. There seems to be a left and right hand issue here (and maybe a foot thrown in there too). This stuff over here, doesn't fit with what is being done over there. To put this in true tech speak: The (ISC)2's DCO of the CBK for the CISSP doesn't fit with a CAT, OK? IMHO.

All that said, the exam really is an entrance, not an achievement, test. It is one of several criteria that gets you into a cohort of CISSPs, who then, ideally, continue and progress in their education. Like you, I worry though that the CISSP brand is not carrying the weight it should. Years, ago I would explain a CISSP as "We're the ones TJX should have listened to." Then I substituted Target, later the DNC, then Equifax, and tomorrow it will be someone else.

[M_Thomas]

Maybe it's just because more people can set aside the time for a shorter test.

What was essentially a full day requirement becoming a half day; that almost certainly cuts across an availability threshold of some kind.

[pkrainman]

I was thinking the same thing (dilution) when I first saw the format change.  I hadn't heard about the increased pass-rates until reading your post.  I certainly share your concern, though.  In addition to my 20+ years of industry experience, I, too, studied daily for over 3 months.  Online bootcamps, study guides, practice exams, etc.

I'm no fan of 6-hour exams, but I busted hump to pass mine.  I can understand and appreciate the adaptive exam approach, but the first iteration should have pared down from 250 questions to 200-250, IMHO.  With the breadth of scope of the CISSP certification, and demonstration of knowledge and understanding that the former exam format afforded, I question how anyone could demonstrate such in as few as 100 questions!

[James]

While I fear that the industry might get saturated with CISSP's, we also need to realize that studies are showing we are going to have a shortage of InfoSec professionals by 2020. While we think there are too many CISSP's, we honestly don't have enough to support the upcoming work load needed within the industry. If ISC2 is stating the pass rate is the same, then I'm good with that. If I have any bias or discourse, is that folks will miss the wonderful opportunity of sweating through a 6 hour exam on paper and pencil. Times change, so we can adjust to it or move on.

[HardCorps88]

I took it just last week.  Did not pass...I studied for over a couple of months and really hard last two weeks.  I found the questions OK but the answers were not on par to what I had studied.  I found the test was hard due to that fact alone.  Following is what I have a beef about in addition to POORLY worded questions on a few.  The punctuation was terrible on some of these questions (are these not screened and reviewed?) 

The questions answers should not be  a "KEY WORD" test but more of what is your knowledge test and not make a person have to reread a question to lock into a particular word sandwiched between non important phrasing.  That's not testing my knowledge thats testing if I can take a test well.  My main concern is that the test is not easier but been made harder with the answers alone.  Make sense?

[Early_Adopter]

Firstly commiserations, you should probably get back up on the horse as soon as you feel ready, and you'll have a good story to tell after you do pass.

I don't think that your concerns are much from folks before the switch over to computer-assisted testing. Were you able to flag the questions you felt bad grammar to ISC2 in the exam?

My personal opinion having sat CISSP twice and passed it twice (paper and non-CAT CBT), and the exam questions are designed for close parsing, comprehension application of knowledge more than they were for the recall of facts. Native English speaker? Big advantage. Read a lot(fewer people do these days)? Big advantage. Deal with wooly concepts, questions and security negotiations? Big advantage. Generational linguistic drift may also play a part here. CISSP exam writers are probably, in the main, crusty old kippers like myself so we might need help with the hip new security argot. We can handle emojis but may use old forms. 😉 

This actually tests your ability as a security leader to quickly understand what's being said or asked for, apply your knowledge and not be complaining that you found it hard to understand what's being asked when it comes back onto your plate a week later and you need to solve it.

Quite often knowledge is imperfect, and you need to choose the least bad option. You might even find yourself needing to win another battle to get action on the critical task because that forces an action.

It would be good to have others opinions, but often those one-word signifiers make all the difference, and effective security as you climb the ladder involves all sorts of trade-offs, negotiations and political hurdles. Because of this, it doesn't confirm so much to the black and white you might find if you sat say CompTIA's Security+.

Assuming that ISC2 didn't radically change the questions banks, and all else being equal then I would say that the move to CBT made it a little harder for those with a lot of mental physical stamina who could have gleaned more info from later questions and being able to go back to previous questions based on this and easier for those that were to get more answers correct initially and might get tired during the exam.

Bottom line ISC2 should probably get more data and report back as to whether the rates different, I trust these guys because I voted for them and if you can mark what's interesting on your next sitting ask them directly for comment, they've always taken feedback on.

[Kaity]

Hello @HardCorps88! 

Regarding any issues with the exam, please reach out to Member Services using the Contact Us form on our site. Public discussions of specific exam content is not allowed, due to the NDA & (ISC)² Code of Ethics. But, we certainly want to review any issues you may have experienced, so please share them using that form so that we can properly document and handle. Thank you!  

[JoePete]

---

@HardCorps88 wrote:

The punctuation was terrible on some of these questions (are these not screened and reviewed?) 

The questions answers should not be  a "KEY WORD" test but more of what is your knowledge test and not make a person have to reread a question to lock into a particular word sandwiched between non important phrasing.

---

First consider that some of the questions on the CAT (as well as the traditional test) may in fact be "test" questions. They may not be part of what you or anyone else was scored on. They are simply there to gauge appropriateness/difficulty as an exam question.

Second, part of this may be a sign of the times. I grew up in an era where standardized tests were few and far between. Today everything is a multiple choice test. Testing alone is huge business, but as this very thread postulates, with expansion comes dilution. Here in the U.S. at least, we have shifted to being a multiple choice society (that is in all regards except for politics where 95 percent of us insist on a binary approach, but I digress 😉 ). I think another issue that to write good information security questions takes both an understanding of the subject matter and communications skills. Let's face it, there's not a lot overlap of those skill sets in our industry. How many times have you seen a meeting wasted because Fred and Wilma start correcting each other over whether the term is X or Y? We not only tend to miss the forest for the trees, we miss the trees over debating whether the bug on the bark is an "information security beetle" or a "cybersecurity beetle."