MICADO

Background

The open-source engine MiCADO extends the default functionalities of state-of-the-art technology, like Kubernetes, to safeguard the implementation of more complex scaling rules and advanced security features. It automates deployment, scaling, management and monitoring of containerised microservices in multiple cloud interfaces, like OpenStack, Google Cloud Platform, Azure, AWS and many more. The Cloud & Container Engine pilot executes “stress-ng” and shows off the usability of the engine, e.g., role-based user management and the machine-aided initiation of up- and downscaling on virtual machine or container level. It will also provide access to the monitoring dashboard and the administrative section.
cloudSME, the University Research Centre for Parallel Computing (University of Westminster, UK) and the Laboratory of Parallel and Distributed Systems (MTA SZTAKI, HU) are responsible for the development of the open-source engine.

Challenge

Setting up and handling of a Kubernetes (K8s) cluster is – even for developers – quite challenging and hardly feasible without experience. Kubernetes as a standalone is not a readymade solution, because it is dependent of other add-on services, like monitoring, Load Balancer, Ingress Controller, the Container Network Interface and others. Therefore, these services carry the real benefits of Kubernetes functionalities ,and at this point MiCADO comes in.

MiCADO is a turnkey solution, and can be thought of as a framework filled with needed open-source technology to compensate for K8’s limitations, such as adaptable scaling rules and security policies. By relying on the Cloud and Container Engine MiCADO, the time for cluster mounting is reduced, time constraints become tradable, and scaling is capable of being customised towards various strategies. The MiCADO pilot aims to showcase the applicability and value of this solution in real time. The pilot provides credentials to access the engine’s dashboard in a “view only mode” (based on role-based policies) so users can get an impression of the service and its options.

Work Plan

We are in the process of developing a cloud resource adapter that will allow MiCADO to run on EGI/EOSC resources. This adapter is required by MiCADO to start and configure cloud instances. In order to avoid vendor lock-in, great care was taken during development to ensure a broad diversity of cloud endpoints, but resource adapters can also be created in-house.The individual goals and milestones of the pilot are as follows:

• The development and integration of a cloud resource adapter for EGI/EOSC resources into MiCADO;
• The re-use of the outcome from an EU founded innovation action within other similar projects;
• Establish a European alternative to the solutions of the market leaders;
• The provision of an example operated in a production environment for demonstration purposes in order to give users a deeper insight into MiCADO. Users can freely visit the sections of the read-only MiCADO dashboard.

Workflow:

To illustrate a MiCADO system in a productive operation, we have created a demonstrator on EGI resources.

What can you do in the demonstrator?

After logging in, you will have read-only access to the MiCADO dashboard, where you can inspect how MiCADO is executing its demo application. You cannot add additional applications, only the demo app can be inspected. You will have access to the Kubernetes dashboard, Grafana and Prometheus.

What is the demo application?

The MiCADO pilot deploys and manages a deliberate stress-testing application called stress-ng. A scheduled job (cron) periodically adjusts the CPU load exerted by stress-ng up and down to simulate an application with dynamically changing computational requirements. MiCADO responds to load increases by scaling out and saves resources when the load is decreased by scaling in.

How to access the pilot?

To illustrate a MiCADO system in productive operation, we have created a demonstrator on EGI resources. This can be accessed under the following URL:
The demonstrator can be accessed through the following url: https://www.micado.eosc-hub.eu.
You can enter the demonstrator and the dashboard with the following credentials:

USER: USERNAME
PASSWORD: PASSWORD

The dashboard is divided into 3 sub-components:

• Prometheus: Prometheus is the monitoring subsystem of MiCADO. It is used to inspect defined scaling metrics. Developers can use the graphical component of Prometheus to assist in the development of new scaling policies.
• Kubernetes: MiCADO’s Kubernetes dashboard visualises the current configuration of the cluster. All components can be monitored and inspected in real time. Due to a modified RBAC (Role Based Access Control), it is not possible to make changes to the cluster through the dashboard.
• Grafana: Grafana is used to further visualise the resources used in the cluster. The available metric allow, among other things:
  • CPU consumption (combined, container, node)
  • Used memory (combined, container, node)
  • Number of nodes
  • Number of containers
  • etc.

Website: www.micado-scale.eu
Documentation: https://micado-scale.readthedocs.io
Repository: https://github.com/micado-scale