Manage App Instances with the Offline Profile Server

Introduction

An offline profile server is an edge application that can manage (deploy, stop, and more) other edge application instances, both locally and on other edge devices. This allows you to manage applications directly from the edge device without connecting to ZEDEDA Cloud.

Offline profile server functionality includes the following: 

• You can manage the application instances when the edge node is offline or when its management interface is down.
• You can bring up or halt multiple sets of application instances. 
• You can have multiple application instances that require the same physical peripherals (for example, GPU, USB) accommodated in the same Edge Node. Whichever applications are brought up (one at a time) use that peripheral.
• Effective utilization of resources like Memory, CPU, and storage. A different set of application instances can run on the same node at other times.
• Profile server takes precedence over ZEDEDA Cloud while managing the application instances.
• Only application instances that are part of a profile are managed through the profile server. Other application instances are not.

Prerequisites

• You must have at least the SysManager role in your ZEDEDA Cloud enterprise.
• You must have an edge node onboarded with at least two network adapter interfaces.
• Your edge node must be running EVE-OS version 12.0.1 or greater.
• You must build your own application using the Offline Profile Server API doc: https://github.com/lf-edge/eve-api/blob/main/PROFILE.md
• You must add a container or add a virtual machine as an edge app type, making sure both port 22 and 80 are mapped to device ports.
• This article assumes that you have Linux knowledge. 

Deploy Application Instances using Offline Profile Server

To deploy an application instance using an offline profile server follow these steps.

Set up an app shared interface

To manage application instances when the device is offline, you need at least one management interface and one app shared interface. After you onboard an edge node to ZEDEDA Cloud per the prerequisites:

1. Go to Edge Nodes > YOUR_NODE. 
2. Click the Adapters tab and click the Pencil icon to edit.
3. Under Network Adapters, configure Interface Usage: 
   1. Configure at least one Management interface.
   2. Configure at least one App Shared interface.
   3. Click Save. 
4. Go to Edge Nodes > YOUR_NODE.
5. Click the Basic Info tab and click the Pencil icon to edit.
6. Configure two specific Tags on the edge node. The fixed keys are $ztag.local.profile.server.default and $ztag.local.profile.server.host, while the values for the profile name and host are your choice:
   1. Tag key: $ztag.local.profile.server.default
      Tag value: profile1
      This tag key and value pair defines the default profile that needs to be brought up when the profile server is deployed, for example you would swap out  ‘profile1’ with the name of your profile. Whichever profile name you use here must also appear in /mnt/profile.This will make sure that after the application instances are deployed, any application instances that are part of 'profile1' will be brought up by default, until you explicitly change the profile name.
   2. Tag Key: $ztag.local.profile.server.host
      Tag Value: 10.5.20.254
      This tag key and value pair has the IP of the profile server you’re deploying, for example  where '10.5.20.254' is your profile server host. This is an IP from the IP range you will configure in the network instance subnet This is the IP you’re going to specify for the profile server application instance during its deployment. IP only, hostname not supported.

7. Click Save.

Create a network instance for the app shared network interface

Create a network instance for the interface that you configured for the edge node:

1. Go to Library > Network Instances > Add New.
2. In the Details section:
   1. Select the Edge Node you configured during the onboarding prerequisites. 
   2. Select the Kind as Local.
   3. Select the Port you configured as the App Shared interface in the edge node adapters page.  
3. In the IP Configuration section: 
   1. Select Manual.
   2. Configure the Subnet, IP Address Range, and Gateway that contains the IP address of the profile server.
   3. Click Add.

Deploy the offline profile server

Using the app you built in the prerequisites, deploy the app instance.

1. Go to Edge App Instances > Add New.
2. In the Edge App & Edge Nodes section, the Edge App is the app you built in the prerequisites, and the Edge Node is the one you onboarded in the prerequisites. 
3. In the Edge App Instance Identity section, there is nothing specific to Offline Profile Server. Configure it as you choose. 
4. In the Adapters & Networks section:
   1. Select the Network Instance you just created.
   2. Expand the Adapter. 
   3. Expand the DHCP Custom.
   4. Enter the IP Address you provided in the profile server host tag, for example 10.5.20.254.
5. In the Edge App Instance Identity section, there is nothing specific to Offline Profile Server in this example. Configure it as you choose. 
6. In the Review & Deploy section, click Deploy.

Start the offline profile server

The profile server has a public key in it. To ssh into it, you need to get the private key.

1. To get it, run the wget command from a local system where you can reach the profile server application app shared interface. The following shows an example app on github:
   

    https://raw.githubusercontent.com/lf-edge/eden/master/tests/eclient/image/cert/id_rsa

2. Make sure it has right permissions:
   

   chmod 700 id_rsa

3. SSH into it:
   

   ssh -o PasswordAuthentication=no -i id_rsa root@<APP_SHARED_IP> -p 9022

4. When you are inside the local profile server app, start the server with the following:
   

   /root/local_manager &>/dev/null &

Deploy other app instances with other profile names

You can have multiple profiles for edge nodes that need to accommodate different sets of apps that don’t need to run all the time. Some use cases include the following:

• Applications that require the same peripherals, for example: GPU, USB.
  • Two applications that require GPU can be accommodated in the same device and brought up when they’re required.
• Hardware that does not have enough resources to run all applications at once.

While deploying other application instances as part of other profiles, in the Edge App Instance Identity you need to include the new override profile key and value for the tag. This will let the profile server know which profile this application belongs to. 

From the left panel Edge App Instances, configure a Tag on the edge app.

• Tag key: $ztag.local.profile.server.override
• Tag value: profile2

Use Profile Server Commands

You can use the following commands to manage application instances when an edge node has no connectivity to ZEDEDA Cloud. Since you are building your own version of the profile server application that will have its own interface, the following commands are examples and are not necessarily the only way to use this feature.

Switch between profiles

1. Add the profile name that you used in $ztag.local.profile.server.default to the file of the profile server application on the edge node’s app shared interface:
   1. Ssh to the edge node
   2. Check if your profile name is in the profile file (for example, using cat):
      

      cat /mnt/profile

   3. If not, edit the file to include your profile name (for example, using echo):
      

      echo "profile1" > /mnt/profile

2. If the profile you want to switch to is 'profile2', you can write over profile1 with profile2 (for example, using echo):
   

   echo "profile2" > /mnt/profile

Purge

Consider a scenario where you have an application "app1" that you want to purge, run (from inside of the offline profile server):

echo "{\"displayname\": \"app1\", \"timestamp\": $(date +%s), \"command\": \"COMMAND_PURGE\"}" > /mnt/app-command.json

Watch for state changes

You can watch the state of the applications associated with profile1 change to halted, and the applications associated with profile2 change to running. 

watch -n 1 "cat /mnt/app-info-status.json | jq"

Use ctrl-C to exit the watch command.

See edge-node information inside the profile server in this file 

cat /mnt/dev-info-status.json | jq

App stop

App Stop command to be given in file (create new file): /mnt/dev-command.json

{"command":"COMMAND_SHUTDOWN"}

Device power off

{"command":"COMMAND_SHUTDOWN_POWEROFF"}

Reboot

Reboot the device manually to bring up the profile server and the profile of applications.

Watch the Video

You can watch a summary of the process for tagging an edge node, creating a network instance, deploying an app instance, and logging into the app instance to start the offline profile server. The UI has changed a bit since the second half of the demo was done, but the video contains good insights.