Increasingly we deploy applications/services as pre-packaged appliances where we have no control or often insight into the inner workings going on inside the black box.
With large appliances this may involve a complex mesh of services which if deployed individually would lead to a number of security controls being put in place.
So the question is really how we deal with these appliances, do we just trust that what is going on inside is done well with a similar care and oversight we would put in place, or do we try and work with the vendor to get some insight / changes made to allow us a greater degree of control in an otherwise black box?
I would ask the vendor if they had any training classes or courses you or your staff could take. Just blindly trusting it is not smart. In one place I worked the previous security guy had set up a (let's just call it a firewall) device and he went through and checked every possible bad actor group in the device to block. There were multiple redundancy of groups that overlapped and overloaded the device. It could only process about 170K items and he had checked over 250K items. Since he didn't understand how the device worked fully he just checked everything which actually made it less secure than it was and promoted a false sense of security.
Even if they don't have training, I would dig into the device and see how it works. Having the hands-on technical experience has never let me down in the long run.
In addition to what @CISOScott suggested, you could compile a list of app security requirements --- covering authentication, cryptography, error-handling, input-handling, logging, session management, and the like --- & fashion this into a questionnaire for the vendor to provide feedback on the app's compliance.
The response will let you ascertain just how secure the application is --- or whether the vendor is even aware of basic requirements --- and you can subsequently present your assessment to management for them to give you a go-ahead to take action, if needed...