(English) Development of a distributed and decentralised monitoring system for Guifi.net

Disculpa, pero esta entrada está disponible sólo en inglés estadounidense. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

Development of a distributed and decentralised monitoring system for Guifi.net

Monitoring the infrastructure of Guifi.net is essential for the operation of the network. A centralized monitoring solution, however, is sensitive to failures and does not follow the spirit of a collaborative effort.

Within the Lightkone project, we have developed a decentralized monitoring solution for Guifi.net, in which the monitoring server software that is hosted on Cloudy nodes can join the overall network monitoring task and in a self-organizing way take over part of that work.

The conceptual system is shown in the next figure. Each device of the network infrastructure of Guifi.net (in the center of the picture) is monitored by several servers.

These monitoring servers coordinate with each other over a distributed database about which server takes care of the monitoring of a set of routers (network devices). This coordination is done continuously and the current assignment is always updated to the database in order to assure that if any server fails, another server will jump in and make sure that all network devices continue to be monitored. In the next figure, the components of the monitoring servers are shown. There is a key component named «assign» to organize the monitoring assignment of each server. The monitoring task is done by the «ping» and «snmp» components.

The implementation of the monitoring servers was done in the Go language and the code is available in our Gitlab repository. We used the AntidoteDB database as a distributed storage service.

You can learn more about the monitoring system in our technical paper published at IEEE SOCA 2019.

(English) Experimenting with Kubernetes in Cloudy

Disculpa, pero esta entrada está disponible sólo en inglés estadounidense. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

We aim to run Kubernetes in a set of Raspberry Pi and to expose the Kubernetes operation in the Cloudy Web GUI.

In order to communicate Kubernetes operations with Cloudy, we implemented the serf-publisher component. Within the Service Controller function, serf-publisher communicates both with the Kubernetes API and through Cloudy’s avahi-ps with Serf.

We create a new service with

kubectl expose deployment/nginx ‑n Cloudy ‑‑type=NodePort

Through serf-publisher, the operations becomes published in Cloudy’s Web GUI:

The code of serf-publisher can be found in this github repository.

Detailed documentation (in Spanish) about this project can be found at here.

(English) Cloudy with IPFS-Cluster over WAN

Disculpa, pero esta entrada está disponible sólo en inglés estadounidense. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

We interconnected three sites with Cloudy nodes over the Internet.

In this setting, we have the situation of a heterogeneous network, with some Cloudy nodes at the same location and others remotely connected over the Internet.

IPFS-Cluster was installed on the Cloudy nodes in order to explore the possibility of having a private IPFS network in this environment.

We observed some timeout issues with IPFS-Cluster and needed to increase for the Raft consensus protocol some network latency related parameter values in the IPFS-Cluster configuration file.

Finally we achieved to successfully connect 7 instances over a heterogeneous network.

Reference (in Spanish):
Leopoldo Álvarez Huerta: Cloud comunitaría y servicios distribuidos sobre IPFS

(English) High availability services in microcloud@home with Cloudy on several Raspberry Pi

Disculpa, pero esta entrada está disponible sólo en inglés estadounidense. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

The goal is to have high availability of the services we deploy at home in case any of the Raspberry Pi gets disconnected.
The hardware we used were three Raspberry Pi.

The solution to achieve high availability consisted of a set of software components, hardware and open technologies, i.e. Raspberry, Cloudy, Docker, Swarm, Node-Red, Mosquitto, keepalived, syncthing, which we integrated to make up the local microcloud, an infrastructure that we can run at home.
We use keepalived to have a floating IP to access the cluster services in the event of a node failure.
The configuration of the containers is replicated with the SyncThing software.

The obtained microcloud offers high availability to any Docker service that we execute on the nodes and has the following characteristics:

– It is scalable and flexible, allowing us to add more nodes, internal or external, to adjust the resources of the microcloud to our needs.
– Thanks to the technology of Docker Swarm, it allows executing complex distributed systems of pre-configured services from a single configuration file.
– It gives us full control over our services.
– It gives us full control over our data.
– The cost is less than 50 Euros per computing node.

Reference (in Spanish):
José Elías Rael Gutierrez: Diseño e implementación de una microCloud abierta para IoT

Cloudy 2017.12 Skywhale + Docker, Docker Compose, IPFS

Lanzamiento de Cloudy 2017.12 Skywhale

¡Estamos muy contentos de anunciar el lanzamiento inmediato de Cloudy 2017.12 Skywhale en diciembre de 2017!

Durante los pasados meses, y gracias al soporte del proyecto netCommons, hemos podido seguir desarrollando Cloudy e incluir varias características nuevas y entusiasmantes. Entre ellas, nos gustaría destacar:

Como resultado de estas integraciones, Cloudy ahora aprovecha la tecnología de contenedores de Docker y los usuarios de Redes Comunitarias pueden arrancar sus imágenes de Docker favoritas o las predefinidas con tan solo unos pocos clics, desde su navegador. Esta rápida provisión de aplicaciones da lugar a nuevas y muy dinámicas maneras de desplegar servicios y compartir recursos en una comunidad digital. Por ejemplo, se puede iniciar una instancia del broker Mosquitto rápidamente para empezar a capturar, gestionar y procesar datos de dispositivos IoT (Internet of Things). En paralelo a todo esto, IPFS se usa ahora en Cloudy como parte del DADS (Distributed Announcement and Discovery Service). Esto permite anunciar desde una instancia Cloudy una cantidad arbitrariamente grande de servicios y datos a otras instancias Cloudy, de una manera eficiente y conveniente.

Además, el código base de Cloudy y sus diferentes paquetes han sido actualizados a la nueva versión de Debian 9 Stretch, mientras que se mantiene la compatibilidad con la anterior versión. Por último, se han añadido varias funcionalidades nuevas a la plataforma o se han mejorado algunas ya existentes y se han corregido algunos errores.

¡Esperamos sus comentarios en las listas de correo de Cloudy!