(English) Development of a distributed and decentralised monitoring system for Guifi.net

Disculpeu, aquesta entrada està disponible només en anglès americà. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

Development of a distributed and decentralised monitoring system for Guifi.net

Monitoring the infrastructure of Guifi.net is essential for the operation of the network. A centralized monitoring solution, however, is sensitive to failures and does not follow the spirit of a collaborative effort.

Within the Lightkone project, we have developed a decentralized monitoring solution for Guifi.net, in which the monitoring server software that is hosted on Cloudy nodes can join the overall network monitoring task and in a self-organizing way take over part of that work.

The conceptual system is shown in the next figure. Each device of the network infrastructure of Guifi.net (in the center of the picture) is monitored by several servers.

These monitoring servers coordinate with each other over a distributed database about which server takes care of the monitoring of a set of routers (network devices). This coordination is done continuously and the current assignment is always updated to the database in order to assure that if any server fails, another server will jump in and make sure that all network devices continue to be monitored. In the next figure, the components of the monitoring servers are shown. There is a key component named “assign” to organize the monitoring assignment of each server. The monitoring task is done by the “ping” and “snmp” components.

The implementation of the monitoring servers was done in the Go language and the code is available in our Gitlab repository. We used the AntidoteDB database as a distributed storage service.

You can learn more about the monitoring system in our technical paper published at IEEE SOCA 2019.

(English) Experimenting with Kubernetes in Cloudy

Disculpeu, aquesta entrada està disponible només en anglès americà. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

We aim to run Kubernetes in a set of Raspberry Pi and to expose the Kubernetes operation in the Cloudy Web GUI.

In order to communicate Kubernetes operations with Cloudy, we implemented the serf-publisher component. Within the Service Controller function, serf-publisher communicates both with the Kubernetes API and through Cloudy’s avahi-ps with Serf.

We create a new service with

kubectl expose deployment/nginx ‑n Cloudy ‑‑type=NodePort

Through serf-publisher, the operations becomes published in Cloudy’s Web GUI:

The code of serf-publisher can be found in this github repository.

Detailed documentation (in Spanish) about this project can be found at here.

(English) Cloudy with IPFS-Cluster over WAN

Disculpeu, aquesta entrada està disponible només en anglès americà. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

We interconnected three sites with Cloudy nodes over the Internet.

In this setting, we have the situation of a heterogeneous network, with some Cloudy nodes at the same location and others remotely connected over the Internet.

IPFS-Cluster was installed on the Cloudy nodes in order to explore the possibility of having a private IPFS network in this environment.

We observed some timeout issues with IPFS-Cluster and needed to increase for the Raft consensus protocol some network latency related parameter values in the IPFS-Cluster configuration file.

Finally we achieved to successfully connect 7 instances over a heterogeneous network.

Reference (in Spanish):
Leopoldo Álvarez Huerta: Cloud comunitaría y servicios distribuidos sobre IPFS

(English) High availability services in microcloud@home with Cloudy on several Raspberry Pi

Disculpeu, aquesta entrada està disponible només en anglès americà. For the sake of viewer convenience, the content is shown below in the alternative language. You may click the link to switch the active language.

The goal is to have high availability of the services we deploy at home in case any of the Raspberry Pi gets disconnected.
The hardware we used were three Raspberry Pi.

The solution to achieve high availability consisted of a set of software components, hardware and open technologies, i.e. Raspberry, Cloudy, Docker, Swarm, Node-Red, Mosquitto, keepalived, syncthing, which we integrated to make up the local microcloud, an infrastructure that we can run at home.
We use keepalived to have a floating IP to access the cluster services in the event of a node failure.
The configuration of the containers is replicated with the SyncThing software.

The obtained microcloud offers high availability to any Docker service that we execute on the nodes and has the following characteristics:

– It is scalable and flexible, allowing us to add more nodes, internal or external, to adjust the resources of the microcloud to our needs.
– Thanks to the technology of Docker Swarm, it allows executing complex distributed systems of pre-configured services from a single configuration file.
– It gives us full control over our services.
– It gives us full control over our data.
– The cost is less than 50 Euros per computing node.

Reference (in Spanish):
José Elías Rael Gutierrez: Diseño e implementación de una microCloud abierta para IoT

Cloudy 2017.12 Skywhale + Docker, Docker Compose, IPFS

Llançament de Cloudy 2017.12 Skywhale

Estem molt contents d’anunciar l’alliberament immediat de Cloudy 2017.12 Skywhale el desembre de 2017!

Durant els mesos passats, i gràcies al suport del projecte netCommons, hem pogut seguir desenvolupant Cloudy i incloure-hi diverses característiques noves i engrescadores. Entre d’altres, ens agradaria destacar:

Com a resultat d’aquestes integracions, Cloudy ara aprofita la tecnologia de contenidors de Docker i els usuaris de Xarxes Comunitàries poden arrencar les seves imatges de Docker preferides, o les predefinides, en només uns pocs clics, des del seu navegador. Aquesta provisió ràpida d’aplicacions dona lloc a noves i molt dinàmiques maneres de desplegar serveis i compartir recursos en una comunitat digital. Per exemple, es pot engegar una instància del broker Mosquitto ràpidament per començar a capturar, gestionar i processar dades de dispositius IoT (Internet of Things). En paral·lel a tot això, IPFs es fa servir ara a Cloudy com a part del DADS (Distributed Announcement and Discovery Service). Això permet anunciar des d’una instància Cloudy una quantitat arbitràriament gran de serveis i dades a d’altres instàncies Cloudy, de manera eficient i convenient.

A més, el codi base de Cloudy i els seus diferents paquets s’han actualitzat a la nova versió de Debian 9 Stretch, mentre que es manté la compatibilitat amb la versió anterior. Per últim, s’han afegit diverses funcionalitats noves a la plataforma, o se n’han millorat algunes de ja existents, i s’han corregit alguns errors.

Esperem els vostres comentaris a les llistes de correu de Cloudy!