How to do external API testing (blackbox)

Short answer: you need a test suite for a third-party vendor API - so you will have to develop one.

Don't expect anyone else to do it for you, and don't expect a "magic bullet" for generating automatically the right tests.

Some things you could try additionally:

• ask the vendor if they provide a list of "breaking changes" for each new release
• ask them how they care about API compatibility/inform them that this is an important feature for you
• check if the API provides specific testing hooks, logging output or something like that for parts which could not be tested easily either
• wrap important API calls with your own logging code, writing input and related output of the API to a log file, this will make it easier to debug things if something unexpected happens
• add assertions to API calls to check pre- and postconditions, so if a new release of the API shows up unexpected behaviour within your application, you get informed early by an error message

If these things work or not depends on who is your vendor and what kind of API you have in mind. An API which produces some inspectable output like files is much easier to test than an API which controls some physical device where you have to observe the behaviour of the thing to decide whether the API call was successful or not.

Based on the poster's phrasing, it's more than just testing, IMO. After you write your unit test for the API and make sure everything is working as expected, you need to monitor third-party APIs so you catch issues before the users do. That's the real risk with third-party APIs - it's not your code and you have no control over how much testing was done on the API or when/if it changes.

(Disclaimer: product names used here) If you use soapUI to write your API tests, those tests can be re-used in AlertSite as an operational monitor to make sure the API keeps working as expected. If it fails the test, you can get alerted before your users call you and complain that your app doesn't work.

Implement learning tests for your area of interest (features that you plan to use). Learning tests are integration tests that are written by the developer against the public contract of the API. The tests should not be written against the internal implementation details even if the source code for the API is available. This kind of learning tests serves two purpose -

1. It dramatically improves your understanding of the third-party API.
2. The tests help to verify whether the claimed new version is actually backward compatible or not.