GeoDB appStore Architecture

In previous posts we have talked about several areas of our solution. One of them is the appStore, which enhances the community aspect of geoSuite, our ETL and DLT suite. In the appStore, developers provide software pieces to interact with the geoSuite using the GEO as currency. The previous post explored a demo version that provides components to geoSuite. Now, we move on from a prototype to a functional version, which combines the use of blockchain and centralised services to provide applications built in JavaScript. In this post, we detail the architecture of the appStore.

Clarify that the current version of the appStore solution is under development and some elements, such as the web client, have been developed to ease testing of the solution. In the final version, the execution of applications will only be possible using geoSuite.

In the appStore, there are various tasks that need to be performed:

• Purchase and manage application licenses.
• Serve application binaries.
• Serve application metadata.
• Provide methods to upload, modify and update applications.
• Provide an interface to interact with the appStore.
• Provide application ratings from users.
• Install and execute applications.
• Manage developer accounts.

These tasks have different nature. One initial solution could be to create a monolithic solution capable of providing all these features. However, a monolithic solution lacks the scalability and flexibility needed in our scenario. Therefore, we propose a modular solution in which each component is specialised to solve a cohesive subset of these requirements.

Our proposal has five main modules:

• Web client: provides developers and acquirers with an interface for the appStore.
• HyperLedger Fabric (HLF): manages application licenses.
• Ethereum: manages license payment in GEO.
• Centralised backend: interacts with HLF and serves metadata.
• Application Storage Node: serves application binaries.

The appStore web client is a GeoDB provided interface that serves as an entry point to the appStore. It allows users to explore the applications that developers uploaded, pay for app licenses using GEOs and execute applications. From the developer viewpoint, it allows to upload new applications, provide resources related to the apps and withdraw the GEOs they earned selling application licenses.

Going to the details of how the web-client is built, we are building it using React.js (16.8.6), a tool to create web interfaces developed by Facebook. In addition, we needed to use Redux.js (4.0.4) and Semantic-UI (0.87.2). The first one has been used to manage user credentials and store it across screens. Semantic-UI allows us to build websites with fast and concise HTML while providing a mobile responsive experience. The web client is served using Next.js (v9.0.7). Regarding the applications purchased within the appStore, they can be installed locally in the user’s device using the application cache of the navigator.

Those features are only provided by the web client at an interface level. Under the hood, they are outsourced to more specialised pieces of software. This decision has been made because the data that stores the appStore is composed of various pieces that have different natures and security requirements. For example, the management of licenses is entrusted to Hyperledger Fabric. Application license payments, done in GEOs, is entrusted to the appStore contract deployed in Ethereum. And the provision of application binaries is entrusted to application storage nodes.

The core of the appStore, i.e., the management of application licenses, relies on HLF (1.4.1). There are four SDKs for HLF at different stages of development: Java, JavaScript, Go and Python. In the appStore, we use JavaScript and Go SDKs. Most of appStore chaincodes are implemented with Go SDK and comprise the management of applications, application licenses, developer accounts and metadata about where is each application binary stored. On the other hand, we need to develop chaincode that interacts with Ethereum. Currently, the most commonly used client library to interact with Ethereum is Web3js (or Web3) which is developed in JavaScript, hence, JavaScript SDK is used to develop chaincode that needs Web3 library, which is required to communicate with Ethereum network. Within the JavaScript chaincode, Web3js (1.0.0-beta.52) is used to check license payments made in Ethereum with GEOs.

And how are these payments made? Application license payments are performed in GEOs and they are made through a smart contract developed in Solidity (0.5.7). This smart contract provides methods that users of the appStore invoke to make payments and acquire application licenses. A record of each of these payments is stored and a payment event is emitted. Regarding the license cost, most of the payments go to the developer and the remainder is retained as a fee to support appStore maintenance costs. In order to make this payment, the web client searches for metamask or similar, and if it is not available uses Web3js library (1.0.0-beta.37).

So far, we have reviewed three parts of the appStore: the web client, Ethereum and HLF. The web client and HLF interact with Ethereum through Web3. The web client also needs to communicate with the chaincode developed in HLF. An approach for the communication between them would be to develop it inside the web application. This way, both the navigator and the web client must be prepared to manage user certificates. However, for the sake of development speed and simplicity for users, we introduced a middleware to communicate with HLF chaincode and expose its functionality in a REST api. This way the certificate management is done in a Node.js backend, which simplifies the development. The appStore backend is developed in Node.js (v8.16.0) using Express.js (4.16.4).

Once the user has purchased an application license through the Ethereum payment, the application license is created in HLF. For this task, the appStore chaincode in HLF should subscribe to Ethereum events and execute a transaction whenever the payment event is emitted. However, HLF does not support the subscription to external events. The following figure shows how a purchase would be carried out in the case that HLF supported subscription to external events:

Therefore, we need to introduce a layer that is able to subscribe to Ethereum events and notifies HLF that a new payment event happened. With this regard, we introduce a module in the appStore backend to perform this task, which HLF is not able yet to fulfil. The resulting interaction is shown in the following sequence diagram:

At this point, we have reviewed the purchase of application licenses. But once an application license is purchased, how do we manage the serving of application binaries? An initial solution would be to manage it with a traditional file server. However, this solution lacks of reliability in the event of peaks or hardware failure. Therefore, we propose to use a distributed approach. The final objective of the appStore is to use alternatives such as Swarm or IPFS, given that they provide distributed access to content indexed by its hash. In them, popular content is replicated across nodes to ensure that such a content is backed by more providers. These qualities make them suitable to distribute our applications binaries. The only drawback is that the application binaries are accessible by anyone that has its link, therefore, instead of distributing the application binaries only to those who have a valid application license, the application itself must check that the user has a valid application license.

Therefore, our final goal is to use IPFS or Swarm to serve application binaries. Currently, our focus is to develop a functional appStore, and having all the best technologies in every part is not our priority. Therefore, while we get there, we have developed a straightforward solution to serve application binaries: GeoDB federation members can provide an application storage node. This way, developers can upload applications to any of these nodes and tell HLF in which node they stored the binaries. The application storage node has been developed with a Node.js backend that exposes rest api developed with Express.js (4.17.1) to upload and download application binaries. These binaries are internally stored in RocksDB (4.0.1).

But any application store has more resources over the application than just the application binaries and who can execute which application. This additional information range from textual description of the apps, to user ratings passing by resources such as videos or images of the applications. Given that this information does not need great security and need to have low latency, we use a centralised system. For them we plan to develop a REST API developed with Express.js (4.17.1) that relies on MariaDB (10.3.16) to store the structured data (application description and tags, user ratings) and the backend filesystem to store the resources (images, videos).

In summary, all these parts work together to provide a functional appStore that is able to enhance community participation on GeoDB solution and enhance its adoption. Here, developers earn GEOs selling application licenses and data acquirers obtain applications for Big Data analytics.