Please help: CISSP Failed after attempting all 150 questions - Need advice

[rvinodh86]

 I am an Info Security professional with 11 years of experience. I took the CISSP CAT 2018 exam day before yesterday after 4 months of tremendous hard work. Studied 8 hours per day for the last one week. I failed after getting 150 questions. It was a nightmare and I was really disappointed. I am trying to figure out where I could have gone wrong. I had known from people that the exam ends at 100 questions if it can determine with certainty that the candidate is above or below the passing standard. I do know of people who had their tests ended at 120ish questions and passed and the others failed at 120 odd questions too. When I crossed the 100th question, I knew I was not doing good enough for the computer to determine that I was above the passing standard. I decided not to lose my cool and answered every question patiently expecting that the exam would end at some point of time. After 125 questions, I seriously lost hope. It was like a roller coaster ride. It kept giving me questions that invariably had key words like "BEST", "MOST", "LEAST" and so on.. While I was able to eliminate 2 choices, it was really hard to narrow down on one. And I also feel like it kept hitting me on areas where I did not fare well. After attempting 150 questions, the test report said that I failed and provided me with a report that said I was above proficiency level in 4 domains and was near or below proficiency in the remaining 4 domains. One thing, I can say for sure was that the exam clearly determined the areas that I personally felt I was weak at. But there are certain confusing aspects that I need clarity on...… My questioning is merely in pursuit of understanding. Please enlighten me on the below

1. If the CAT could fail a person at 120ish and pass another candidate at 120ish, why does it throw 150 questions for a few?. Does that mean that I could have got the last 5 questions and still passed ?? Was my answering so unpredictive that the algorithm was not efficient enough to determine my capability until the 150th question ? If yes, it contradicts with my testing report where it says I was near or below proficiency in 4 domains. Why does an algorithm have to wait till the1 50th question to finally determine I did not reach the proficiency in 4 domains ?

2. Can a candidate be below proficiency in 1-2 domains and still pass at the 150'th question ? I see people posting that they had 150 questions and came out thinking that they surely failed but passed to their surprise. Quite a gray area

3. If a candidate has cleared the exam at 100 questions, is it mandatory that he should have received all 25 unscored questions ? If yes, does it mean that of the 75 that were scored, he/she could have answered merely 70 percentage of them correctly(close to 50+ questions) and still passed ? If yes, this seems not ok.

4. The difficulty level of a question is relative. What is difficult to one could be easier to another. How does ISC2 determine that a question is difficult or easy. 

5. If ISC2 had a way to determine the set of difficult questions, the exam could throw only questions from the difficult pool to all candidates and determine if the candidate scores 70% of ISC2's difficult questions correct? That would be a much fair way. Why even give the easier ones ?

I can read the Sybex 2018 edition twice before my next attempt, but I don't want this scaring experience again 🙂 Can someone enlighten me on the above questions asked and the test taking strategy

[Baechle]

Mr. Ramu,

I'm sorry to hear that you didn't pass after some intense studying.  I'll try to answer some of your questions to the best of my ability.  Several of the things that you asked are details that are not really releasable because they could provide an unfair advantage to folks that work the test algorithm rather than the content.

---

@rvinodh86 wrote:

1. If the CAT could fail a person at 120ish and pass another candidate at 120ish, why does it throw 150 questions for a few?. Does that mean that I could have got the last 5 questions and still passed ?? Was my answering so unpredictive that the algorithm was not efficient enough to determine my capability until the 150th question ? If yes, it contradicts with my testing report where it says I was near or below proficiency in 4 domains. Why does an algorithm have to wait till the1 50th question to finally determine I did not reach the proficiency in 4 domains ? 

---

Yes, you may have failed by only a handful of questions.  Examination security also dictates that the exam collect enough information about your answers to determine if the exam itself was compromised (such as you attempting to use an answer pattern), or determine if you are simply guessing and happen to be getting lucky.  You may have been getting asked additional questions about the domains you already passed in order to make a better evaluation of your answer pattern.

---

@rvinodh86 wrote:

2. Can a candidate be below proficiency in 1-2 domains and still pass at the 150'th question ? I see people posting that they had 150 questions and came out thinking that they surely failed but passed to their surprise. Quite a gray area

---

I'm sure they could for example if they were on the border for those topics.  If you keep meandering across the border between passing and failing, the system is going to keep asking you questions until you wander so far on either side that the exam system can make a determination about your proficiency level.  The system isn't designed to fail you.  It's going to give you an opportunity to pull yourself out and pass if you're close enough.

---

@rvinodh86 wrote:

3. If a candidate has cleared the exam at 100 questions, is it mandatory that he should have received all 25 unscored questions ? If yes, does it mean that of the 75 that were scored, he/she could have answered merely 70 percentage of them correctly(close to 50+ questions) and still passed ? If yes, this seems not ok. 

---

How do you figure?  The system could theoretically ask you 9 questions of increasing difficulty across all 8 domains.  You could reach the most difficult questions by question 4.  That means 6 questions at the most difficult levels, and even if you answered a couple wrong, if you stayed at the passing level by the time it reached question 9 it could pass you for that domain and move on. 

The only reason that it should need to ask you more questions is if you are in the middle zone between passing and failing and is attempting to provide you an opportunity to reach a passing score.  Even if you had to reach 120 questions with no experimental questions, that's still only about 15 questions per domain.  How many questions do you want to be asked?  Even the old paper based, 6 hour, 250 question test (under the 10 domain model) only asked 22.5 questions per domain on average.

---

@rvinodh86 wrote:

4. The difficulty level of a question is relative. What is difficult to one could be easier to another. How does ISC2 determine that a question is difficult or easy.  

---

We do.  The community of CISSPs draft and evaluate the questions for the exam.  When there is an exam version change or an exam update, CISSPs volunteer their time to exam-writing workshops.

@rvinodh86 wrote:

5. If ISC2 had a way to determine the set of difficult questions, the exam could throw only questions from the difficult pool to all candidates and determine if the candidate scores 70% of ISC2's difficult questions correct? That would be a much fair way. Why even give the easier ones ? 

---

Here are some ideas: (1) To determine if the candidate is getting lucky using answer patterns; (2) To collect feedback data to determine the relative fairness and difficulty of several types and subjects of the questions asked; (3) To ease the candidate into the subject material, test format and boost confidence rather than smack them down; (4) To ensure the candidate has some knowledge in the topic rather than there being some technical error preventing the candidate from answering any questions correctly; and several others...

As far as test taking, I can't really help you there.  I never studied for the exam.

Rather than using some gimmick, possibly you could use this community to ask questions about topics you don't quite understand.  You seem to already know what topics you're weak in, and you've probably taken practice tests that tell you what questions you're getting wrong.  So... ask us.  

Sincerely,

Eric B.

[rslade]

> rvinodh86 (Viewer) posted a new topic in Certifications on 07-27-2018 12:08 AM

> I decided not to lose my cool and answered every question patiently expecting
> that the exam would end at some point of time.

Good choice.

> After 125 questions, I seriously
> lost hope.

Oh, dear. A CAT algorithm can go on for some time, if you are close to the line.

> It kept giving me questions that
> invariably had key words like "BEST", "MOST", "LEAST" and so on.

Yeah. Those are looking for your experience, judgement, and critical thinking.
(They are the ones that most of the "practice sets" don't have.)

> And I
> also feel like it kept hitting me on areas where I did not fare well.

Well, don't feel it was "attacking" you in weak areas. Think of it as giving you
another chance to prove you *do* know your stuff.

> After
> attempting 150 questions, the test report said I was above proficiency level in
> 4 domains and was near or below proficiency in the remaining 4 domains. One
> thing, I can say for sure that the exam clearly determined the areas that I felt
> I was weak.

So it's giving you good advice.

>   1. If the CAT could fail a person at 120ish and pass another candidate
> at 120ish, why does it throw 150 questions for a few?

It's that word "adaptive." Instead of throwing all 250 questions at you, it figures
out if you are good enough in some areas, and then stops worrying about those.
Then concentrating on areas where you haven't been able to demonstrate
competence. And, like I said, giving you more chances.

> Does that mean that I
> could have got the last 5 questions and still passed ??

Ummmm, not sure that *anything* you said indicates that ...

> Was my answering so
> unpredictive that the algorithm was not efficient enough to determine my
> capability until the 150th question ?

CAT algorithms are fairly complex at the best of times. In something like the
CISSP they are going to be more so.

> If yes, it contradicts with my testing
> report where it says I was near or below proficiency in 4 domains. Why does an
> algorithm have to wait till the1 50th question to finally determine I did not
> reach the proficiency in 4 domains ?

No, it doesn't contradict. In order to explain why, I think you would have to
know a *lot* more about testing and measurement, let alone psychometircs.

>    2. If a person has cleared the exam at
> 100 questions, is it mandatory that he should have received all 25 unscored
> questions ? If yes, does it mean that of the 75 that were scored, he/she could
> have answered merely 70 percentage of them correctly(close to 50+ questions) and
> still passed ? If yes, this seems not ok.

First off, your math is wrong. You've got eight domains and 100 questions, so it
isn't 25 questions per domain, but twelve and a half. Which is not ridiculous,
because many questions address more than one domain. This was the case even
with the paper based tests, and more so with CAT.

In terms of the "25 unscored questions," are you referring to the 25 unscored
questions on the paper based or linear exam? That's based on a 250 question set,
so, at the very least, I would assume that on the CAT that would drop to 10.

>   3. The difficulty level of a
> question is relative. What is difficult to one could be easier to another. How
> does ISC2 determine that a question is difficult or easy.

Kind of the point of the exam and certification as a whole. What is difficult for
the average working joe should be easy enough for a certified professional.

>    4. If ISC2 had a
> way to determine the set of difficult questions, the exam could throw only
> questions from the difficult pool to all candidates and determine if the
> candidate scores 70% of ISC2's difficult questions correct? That would be a much
> fair way. Why even give the easier ones ?

To give you more chances.

>   I can read the Sybex 2018 edition
> twice before my next attempt, but I don't want this scaring experience again 🙂

Don't stick to one study guide. Obviously you need some breadth. Take the
domains the system said you were weak in, and drill down on those areas. Check
out http://victoria.tc.ca/int-grps/books/techrev/mnbksccd.htm
Also, consider reading "Security Engineering," by Ross Anderson. Best single
volume reference extent.


====================== (quote inserted randomly by Pegasus Mailer)
rslade@vcn.bc.ca slade@victoria.tc.ca rslade@computercrime.org
If you put tomfoolery into a computer, nothing comes out of it
but tomfoolery. But this tomfoolery, having passed through a very
expensive machine, is somehow ennobled and no-one dares criticize
it. - Pierre Gallois
victoria.tc.ca/techrev/rms.htm http://www.infosecbc.org/links
http://blogs.securiteam.com/index.php/archives/author/p1/
http://twitter.com/rslade

[rslade]

> rvinodh86 (Viewer) posted a new topic in Certifications on 07-27-2018 12:31 AM

>  I am an Info Security professional with 11 years of experience. I took
> the CISSP CAT 2018 exam day before yesterday after 4 months of tremendous hard
> work.

Obviously this really bugged you, since you've editted your own post three times in
quick succsession.

> Studied 8 hours per day for the last one week.

OK, there's one problem. I always told my seminar candidates *NOT* to study on
the last night. Give yourself a break. Go in rested. It's going to be hard: don't
make it harder by going in frazzled and exhausted.


====================== (quote inserted randomly by Pegasus Mailer)
rslade@vcn.bc.ca slade@victoria.tc.ca rslade@computercrime.org
Shall we, today, continue to stir out our lives with coffee
spoons, our only monument lost golf balls? - T. S. Eliot
victoria.tc.ca/techrev/rms.htm http://www.infosecbc.org/links
http://blogs.securiteam.com/index.php/archives/author/p1/
http://twitter.com/rslade

[denbesten]

Our colleagues have given some good insight.  Perhaps the most important is to not be frazzled and exhausted going into the test.  For me, I spent the 24 hours before the exam with family and friends doing things I found enjoyable.  Then, as I headed to the test center, I bought high-end cookies so that I had a treat to look forward to on my drive home.   

In your case, I would also recommend spending some google time researching how CAT tests work.  I have made some postings regarding it on this board, and many others have published similar things" across the Internet.  The point being that if CAT is causing angst, learn how it works so that you are no longer afraid of it.

Regarding difficulty, that can be measured by looking at large quantities of completed tests.  If a question is answered correctly by both people who pass and people who fail, it is an easy question (and also useless from a testing perspective) .  If only those who pass get it right, it is hard.  The art of doing this is called psychometrics.  Obviously, one can not do this math until a bunch of people have answered the question on a test, which is pretty much the point behind questions being ungraded until they have a track record. 

Along the same lines, psychometrics help weed out "bad" questions (where those who pass the test tend to get it wrong), such as those that are poorly worded or have multiple "correct" answers because they don't "align" with those who pass the test.

[ro83]

@Baechle thank you very much for this informative and useful overview about calculating CISSP exam results.

[j_M007]

A perplexing and difficult experience. The first thing to do is to forgive, move forward and do a post mortem review of what went right and tweak what went less well.

I was astounded that the questions kept coming, so I decided the time was then to breathe calmly and look for obvious illogic. One I discounted the obvious howlers, I sought to see what I call the "truthiest" answer.

I think the exam is there not just to test for knowledge, but also to test for experience and sharp thinking. In my practice, when I did not know an answer I went through that process. Often, my guess was the right one.

I based myself on the 2015! version of the Sybex book because the 2018 version wasn't available when I started.

As far as test day went, I really relaxed the night before and booked an afternoon appointment so I could be at my peak.

Planning, planning, planning is the way to tackle this sucker. I know you will succeed!

[rvinodh86]

So encouraging to hear. Thank you sir. I read only from Shon Harris last time. I did not have the patience to read the book for more than once. Now I bought the ISC2 Sybex 2018 edition and also the ISC2 test papers. Reading them as slowly as I can. This time, I am going to give a tough fight 🙂

[rvinodh86]

Thank you so much for the detailed response. I am taking your inputs and preparing myself accordingly.

[rvinodh86]

Thank you so much Eric ! Very well explained ! 🙂