Question - What impact do frequent (and varied) session disconnects and concurrent logins have on the CISSP CAT test's evaluation of a student?
This isn't the start, but it's certainly part of my story. Please work with me while I elaborate. I have looked through numerous threads on the forum but haven't found any that enumerate on technical difficulties experienced by the examination facility mid-test and its possible impact on the outcome.
Recently I sat for my first attempt at passing the CISSP exam. Before doing so, I studied using the following references:
All-in-One Study Guide by Harris (7th Ed.),
(ISC)2's Official Study Guide by Chapple (8th Ed.),
11th Hour by Conrad (3rd Ed.), and
Pocket Prep's CISSP module for a different perspective of practice questions.
I will also volunteer that I converted each chapter's list of Exam Essentials (Ref: (ISC)2's Official Study Guide), enhanced the material, and then manually converted them into flashcards. Collectively, there is somewhere near 700 flashcards. It was simply part of my study plan and a way to go hands-on with the material. The short version is, in my opinion, I am committed and well-informed.
During the examination, my workstation experienced several types of delays and fully-locked user interfaces. Towards the latter half of the first 100 questions, I was moved between three different workstations, completed over a dozen hard restarts, and at times was logged on to more than one workstation simultaneously. Without revealing any of the inner workings of the CAT exam algorithm(s) and without listing the numerous, specific concerns I have over the availability and integrity of my exam submissions, what possible impacts could this of had on the exam's evaluation of me?
Needless to say I did not pass. I also have an email in to (ISC)2's Customer Support but have not heard back.
Wow, that sounds like an absolutely horrendous experience - taking exams is stressful enough without all of that happening to you!
They do have a policy covering "exam assistance and breaks", and "technical issues" which I've linked below. Your situation may fall outside of this policy so persevere with contacting Support to get a proper adjudication and hopefully a resit at no additional fee/no wait time.
In terms of the potential impacts:
If the exam timer was still running down during your issues then you would have lost time through no fault of your own.
Stress of the situation causing you not to think clearly or be distracted leading to selection of incorrect answers to questions.
I'm not sure being logged on in multiple places would cause any issues with the scoring unless someone else was sat at one of the terminals answering questions incorrectly. Did you notice the question numbers not being sequential? I don't know if the CAT algorithm takes into account time to answer a question (e.g. extended time to answer through taking voluntary breaks or enforced breaks due to issues being considered a negative thing), but even if it did I think that would potentially only change the number of questions you were asked and the topics. Scoring should only be affected by correct/incorrect answers.
You must remain in your seat during the exam, unless the staff says it’s okay for you to get up. You may not change your computer terminal unless a TA directs you to do so.
Raise your hand to notify the TA if you:
Believe you have a problem with your computer.
Need to change note boards.
Need to take a break.
Need the administrator for any reason.
Your total exam time includes any unscheduled breaks you may take. All breaks count against your testing time. You must leave the testing room during your break. However, you may not leave the building or access any personal belongings unless absolutely necessary (such as for retrieving medication).
When you take a break, you’ll be required to submit to a palm vein scan before and after your break
On rare occasions, the test center may have technical problems. You may need to reschedule your cybersecurity exam.
If technical issues cause you to wait more than 30 minutes after your scheduled exam start time, or your exam starts and then stops with a delay of more than 30 minutes, you’ll have the choice of continuing to wait or rescheduling with no additional fee:
If you choose not to reschedule and take the test after your delay, your test results will count. You’ll have no further recourse.
If you choose to wait but change your mind before starting (or restarting) your exam, you can take the exam at a later date at no extra cost.
If you choose to reschedule or the technical problem can’t be fixed, you’ll be allowed to test at a later date at no extra charge.
We know your time is valuable. If your test center identifies technical problems before your exam, they’ll make every attempt to contact you."
Effectively, the policy covers an issue lasting 30 mins, but doesn't say anything about multiple issues that may add up to 30 mins or more, so you may have some recourse there if you were inconvenienced by more than 30 mins in total. I would also highlight the stress of the situation as this is sure to have had an effect. Starting and stopping multiple times is probably just as bad if not worse than one issue lasting 30 mins.
Although, if you were offered the chance to reschedule at the time and chose to continue then you may have no recourse.