You need to make a distinction between what you're assessing; the vendors general approach to security or the security of the specific services that you're buying from them, as that should guide how you go about it.
If you look at it in terms of the source of assurance you can use that may help:
1 Supplier assertion - The supplier asserts in the sales material, service descriptions, marketing and in verbal representations that their service is secure (Low)
2 Contractual Commitment - The supplier commits contractually to operate specific security controls and follow good industry practice (High)
3 Off site supplier assessment - The suppliers answers a standardised questionnaire, provides supporting documentary evidence requested and may also demonstrate aspect of their security via Webex, Skype or similar technology. (Medium)
4 On site supplier assessment - The suppliers answers a standardised questionnaire and allows an on site assessment by a security auditor, providing opportunity to observe operation of controls and/or view systems and documentation. (High)
5 Architectural Review A qualified and experienced security architect reviews the architectural design of the service(s) to ensure that they are designed to be technically secure (Medium/Low)
6 Service is built from known secure components - A security architect or auditor reviews the certifications and/or audit reports on the technical components making up the service (Low)
7 Previous Auditor Work - Independent auditors, including certification auditors have previously and recently audited the suppliers service(s) and ISMS and issued either a certificate or unqualified opinion as to it’s security High
8 Independent Penetration Test - An independent penetration test of the service(s) is either carried out by your organisation, its chosen external pen test company or a recent independent CHECK/CREST pen test report is provided by the supplier. (Medium)
So there are multiple methods. You might also want to look at the CSAs STAR scheme.
-----------------------------------------------------------
Steve Wilme CISSP-ISSAP, ISSMP MCIIS
