Three Degrees of Mediation: Challenges and Lessons in building cloud-agnostic systems

The paper is based on the work I have done while working as a Technology Principal and Cloud Innovation Lead at Global Technology Consulting Firm. It describes unique challenges encountered during the development of a Cloud Broker. Designed to provide seamless and uniform user experience as well as additional enterprise-level capabilities over a broad range of public and private cloud platforms, the broker had to work with a wide variety of diverse cloud APIs. And, in order to allow rapid onboarding of new target clouds, we had to design the system to operate in a generic cloud-agnostic way. We have reviewed and tried quite a few of available cloud integration technologies, both commercial and open-source as a foundation for our cloud abstraction and mediation layer. Couple seemed to be a good choice and, in theory, should have served all our needs, but in the course of development we found that the problem was far more complex than we (and the rest of the market) had anticipated. As we were building our system, we discovered that there was not just one, but 3 different levels of mediation required to build a truly cloud-agnostic solution: syntactic, semantic and idiosyncratic and even the best off-the-shelf solution will not be able to give us that last two out of the box. The final part of the presentation describes the architecture that allows to realize our goals and build the universal broker.