Good question. There is no silver bullet answer to this and there is judgment and different interpretations of the standards. I will provide you my stance as an auditor that performs SOC 2 audits.
The key to determining if a CSP is a SOC 2 subservice provider, is if you are relying on the vendor to perform a control and without the control, you would not be meeting a specific SOC 2 requirement. For instance, if you are using a CSP such as AWS and Azure to host the services you provide to customers, they would be a subservice provider, since you cannot meet the requirement for physical security without relying on AWS and Azure. However, if you are using a CSP for tracking changes and software development, this vendor would not be a subservice provider since you are not relying on them for specific controls.
Keep in mind, regardless of whether or not you determine if the vendor is a subservice provider, you are still responsible for monitoring key vendors. In this instance, you should have a vendor management program where key vendors are monitored on a regular basis, subservice orgs and non-subservice orgs.