1. Introduction
ZEDEDA is a simple and scalable cloud-based IoT edge orchestration solution that delivers visibility, control, and security for the distributed IoT edge. ZEDEDA leverages Edge Virtualization Engine (EVE-OS), an open edge computing engine (part of LF Edge), to simplify the orchestration of containers and clusters (Docker and Kubernetes) and virtual machines (VMs) for cloud-native and legacy applications on any hardware (ARM, x86 or GPU). ZEDEDA cloud-based IoT edge orchestration solves many edge analytics challenges (like discovery, interpretation, and communication of meaningful patterns in data, etc.) in a unique manner that many organizations face. ZEDEDA cloud-based IoT edge orchestration brings the power of essential cloud services from the cloud to your on-premise devices.
At ZEDEDA, an Edge Application represents the Metadata manifest of the software application. The Edge Application describes the various software pieces and how they are run on any given Edge Node (every hardware node running ZEDEDA software and under the ZEDEDA platform is represented as an Edge Node). The Metadata manifest is typically defined by the application developer or software provider. It describes the purpose of the application, the intended usage, and the required resources and services to run it.
1.1. Examples of Edge Applications
Video Streaming:
One example is software that uses image recognition and identifies if the safety helmet is worn in hazardous areas. This application must get a video stream from an IP camera and process it with image recognition abilities. The required application can be developed using the C# application, .NET framework, or any other platform and packaged inside a Windows 10 virtual machine.
Application to Sense Environmental Parameters:
Another example is software that does sensing Environmental parameters and determines the water required for agriculture on any given day. These parameters could be inputs from various environmental sensors like temperature, humidity, or luminosity from the deployed area. The data is then processed to determine the amount of water required to be dispensed. The application could be in python language and packaged as a docker-container application.
2. Edge Application Metadata Fields
Now, let’s delve into the Metadata fields required to define an Edge Application in ZEDEDA.
Below is a list of Metadata fields required to define an application.
- Type
- Identity
- Images and Volumes
- Resource requirements
- Storage
-
Adapters and Networks
- Host port mapping
- Security Policy
- Configuration
- Developer Information
Each Metadata field is discussed in detail.
2.1. Edge Application Type
Edge Nodes running EVE-OS runtime supports the ability to run on multiple computing environments such as Virtual Machines (an emulation of a computer system), Containers (a standard unit of software that packages up code and all its dependencies, so the application runs quickly and reliably from one computing environment to another), natively. The type of application in terms of scale, portability, and others will determine the computing environment to choose from. Edge Nodes today do not allow you to combine two types of runtime into a single application. You can either have Virtual Machines or Containers.
2.2. Edge Application Identity
Every object in ZEDEDA has a unique identity. The Edge Application Identity section of manifest includes specific mandatory parameters, described as below;
- 'Name': This parameter uniquely identifies edge applications. With a limit of 256 alphanumeric characters including full stop (.), comma (,), hyphen (-), underscore (_) with no spaces allowed. During the application's life, this parameter cannot be changed or renamed after it is saved. Since it acts as the primary key, it cannot be reused and unique in the given tenants.
- 'Title': This is an additional identity. It acts as a pseudonym, which can be changed during the life of the application. With a limit of 256 characters, it allows special symbols and can be more descriptive.
- 'Category': The category of Application can either be an Operating System, Cloud Gateway, Network Gateway, Infrastructure, End Application, Runtime, or Others.
- 'VM Mode': If the application is run on a Virtual Machine, then it can either be in a para-virtualized (PVM), hardware virtualized (HVM), or field manipulation (FML) environment.
The identity section of manifest also includes additional parameters, which are optional and described as below;
- Description: 256 characters in length, a free form string to capture comments, and a long description of the application. It’s only used in the ZEDEDA management layer.
- Logo: SVGA icon for the application. Supported formats (.png), (.jpeg), (.jpg). Maximum file size–5MB. Recommended Dimensions–200x40 px.
- Company Name: Information about the company.
- License: Every application is required to be licensed under one or the other type of license, depending on the type of application. ZEDEDA lists the various and most commonly used licenses (public licenses could be Apache License 2.0, Mozilla Public License, or any other) supporting the application and allows uploading of custom license or even providing the license URL, where applicable.
- VNC Connection: this is used to control another computer remotely. Uses the Remote Frame Buffer (RFB) protocol. It is basically a graphical desktop sharing system.
2.3. Edge Application Images and Volumes
As discussed in the Edge Application Types section, every application needs to be packaged as one of the following:
-
Virtual Machine images
- QCOW2 format
- RAW format
-
Container Images
- Docker format
-
Unikernel images
- RAW format
Edge Application is made up of one or more images. Edge Application Image binary data can be stored in any Data Store (A Data Store captures the location where you want the device to get the binary application images. It can be hosted anywhere. For example, AWS, Azure, GCP, or native servers running HTTPS/SFTP) of customer’s choice.
- ZEDEDA Owned and operated Data Store (e.g., AWS S3 or Azure)
- Customer-owned and operated Data Store inside the customer’s enterprise data center
- Public Datastore (e.g., Docker Hub, Azure Container Registry, etc.)
ZEDEDA controller itself doesn’t need to have access to the binary data of the images. The controller only needs the location or path from where it can access the binary data of images. It is the Edge Node, which will access the images for the application to run on it.
In manifest Metadata, the developer or software provider specifies each image using the following parameters;
- Type: VM or Container
- Format: Raw, QCOW2, etc.
- Size: The total file size in bytes that needs to be downloaded by the Edge Node. This is an information-only field for the Edge Node.
- SHA256: Exact SHA for binary data. After downloading the image on the Edge Node, EVE-OS calculates the SHA of the binary file. The calculated SHA will have to match the input provided in the manifest. If it doesn’t, then application instances go into an error state and don't run.
- Data Store: This is the enterprise filer or cloud storage, where the actual binary data resides. ZEDEDA supports multiple types of storage protocols and destinations for image download. Please refer to the datastore section for more details.
2.4. Edge Application Runtime Resources
Application developers need to provide EVE-OS information to determine what resources are required for this application to run. Edge Application explicitly requires one to define how much memory and CPU are required. In addition to that, storage volume requirements can also be defined in the manifest. ZEDEDA controller checks if enough resources are available in a given Edge Node to run the application before placing the application. Besides, EVE-OS also performs runtime checks to ascertain the exact resources required, if available and allocated.
Virtual Machine Managers (VMMs) such as QEMU/KVM require memory to perform their virtualization functions. EVE-OS makes a conservative estimation to accommodate for this VMM overhead for both virtual machine and container applications. This results in additional memory being set aside for a given application instance. To ensure consistent and reliable operations, EVE-OS does not allow oversubscription of memory, i.e. the total amount of memory allocated for all application instances on a node cannot exceed the total available node memory. This means that a conservative VMM overhead may artificially limit the amount of applications/instances that can be deployed on a node.For low memory edge devices or high application count use-cases (such as container deployments), it may therefore be beneficial to optimize the VMM overhead for the application at hand.
The EVE-OS standard overhead currently (subject to change) is defined as:
overhead = 370 MB + 2.5% * application vRAM + 3MB * application vCPU count + 1% of MMIO size (memory mapped IO, dependant on the IO devices mapped into the application)
To override the standard EVE calculated overhead a maximum overhead value can be provided at application or application instance level. Customizing these values will involve trial-and-error. It is recommended not to change the default without thorough testing.
For container use-cases, reducing the overhead to 60-200Mb range may be feasible depending on application specifics.
Note that when modify the value for a running application instance, the new value will not take effect until the application is restarted.
2.5. Edge Application Storage
Most Edge Nodes have multiple CPU cores. EVE-OS can allocate one or more CPUs exclusively to a given application. EVE-OS doesn’t allow Statistical Multiplexing (stat mux) of CPU cores. They are dedicated to a given application and cannot be oversubscribed or shared with other applications and base system OS.
Every application is also given a slice of memory to run on the Edge Node. As CPU, memory is also hard allocated and cannot be statmuxed with other applications when they do not run or use their memory.
2.6. Edge Application Adapters and Networks
Every application can connect to the external world using a physical adapter or a logical network.
Let's first talk about the physical adapter method. Edge Node typically has one or more Ethernet ports, COM ports, USB ports, general-purpose input/output (GPIOs), wireless local area network (WLAN) interfaces, Long-Term Evolution (LTE) modems, etc. Edge Application manifest allows you to define the application requires that it own any number of these ports. EVE-OS software then maps the hardware controller associated with these ports directly into the application space. From that point onwards application has full control of hardware I/O spaces and now controls and manages hardware.
In certain cases this is a perfect idea to map the hardware directly into the application, for example.
-
The Application needs an acceleration engine dedicated to processing like a graphics processing unit (GPU), vision processing unit (VPU), video cards, etc.
-
The Application uses proprietary protocols that EVE-OS doesn’t understand.
-
The Application needs a faster transfer rate without taking the hit of going through host OS software.
-
Finally, the application is the legacy mode, and it's easier to map hardware than rewrite it for sharing hardware resources.
Note: EVE-OS can only do physical adapter assignments at the controller level, not individual physical interfaces. For example, if the device has 4 USB ports but only a single USB controller, in that case, all 4 USB ports are given to the application because the USB controller is dedicated.
Now let’s talk about networks. EVE-OS supports defining various connectivity options like Network address translation (NAT), Switch, a virtual private network (VPN), and Mesh networking. Edge application developers can ask for one or more networks to operate his/her application. In addition to network requirements, manifest allows the developer to define a full security policy for network resources and host port mapping. Please refer to the networking sections for details.
Host port mapping:
Developers can add a host port mapping rule for the application to connect to the external network.
Security policy:
Developers can provide access lists (ACLs) to receive any connection from an external network. The port mapping details need to be provided if the application needs to accept the link.
2.7. Edge Application Custom Configuration
ZEDEDA Edge Virtualization platform uses a cloud-init mechanism to configure edge application instances. Customize the configuration script by specifying User Data in a cloud-config format (https://cloudinit.readthedocs.io/en/latest/topics/examples.html). You can use some of the ZEDEDA platform, Webhook, and User input variables to accomplish the custom configuration task.
You need to define the task in a cloud-init format for Ubuntu or a CentOS type of virtual machine instance. See Customize Edge Application for more details.
For Linux-based containers, you must pass environmental variables to the container to be initialized at runtime. For example, the container may need some configs like:
-
Where do the log files need to be stored?
-
Where are the credential files need to be stored? Etc.
The environmental variable value/s can be either given directly or as a variable that can define during the application instance creation. This is of the form 'Variable'=<value>.
2.8. Edge Application Developer Information
ZEDEDA controller captures developer information like name, email, website, and a description of what the application is used for and the support description.