Vladimir,
@vt100 wrote:
Can you shed a bit more light on preferences for manual pen-testing of specialized equipment?
I do not posses a deep enough knowledge to argue for or against it, but am trying to figure out if there is an added risk in going manual.
Sure. These are my thoughts and opinions, and in no way should be considered authoritative best practice.
There is a psychological factor in performing the tests outside of a GUI or automatic tool that assists the testing process, the report writing process, and any oral defense of the findings (that typically occurs when you're evaluating and making claims of deficiency against someone else's configuration). That is, you are much more deliberate in the whole process because it requires much more attention and focus. When using an automated or GUI tool, it has been my experience that folks simply "select all" and then come back when it's done to read the report and often may have no idea of how the tool reached its conclusions.
Although understanding your target is a good fundamental practice to all testing, I believe it is in severe danger of being overlooked when using a push-button automated toolkit. A tester planning a manual test is encouraged to obtain source documentation about the target, such as it's operating system, installed applications, and patch levels. For example, knowledge that a system is relying upon a certain version of a RH kernel and is running custom applications (a), (b), and (c) will help you figure out existing vulnerabilities prior to initiating a fatal or or several hours/days of unnecessary tests.
The next benefit is in the selection of the tool, the switches that are set, and the arguments given when compared to a tool where these processes are handled behind the scenes of check boxes and buttons. Rather than relying upon a "select all" function in GUI tool to do a "bunch of magic", you have direct knowledge of the purpose of the command-line tool, what operations it performed and why you selected that test in the first place.
The final major benefit of running a manual test is that the tester looks at the target system from the vantage point of a threat rather than the perspective of a tool. For example, it's unlikely that running an automated tool will detect a malicious IoT process using a network knock protocol. However, running a sniffer on the final wire to the target that filters out all of the expected traffic from your initial assessment can identify the traffic that is unexpected. Building out the tests based on a threat hunting methodology rather than a tool based methodology provides both a much deeper and wider look at your target.
@vt100 wrote:
Would this, in your opinion, not increase probability that some things may be omitted or, conversely, are likelier to cause the issues you are trying to avoid?
This is why following an established pen test methodology is important; and I suggested hiring out to an experienced pen tester and shadowing them for the first run. If your pent test engagement literally has them coming in, running Nessus and leaving you with a printout of the report (I've seen consultants charge customers $1m for this service) .... then these aren't pros, they're scammers.
Sincerely,
Eric B.
