Custom Configuration Edge Application

Updated December 20, 2024 17:41

Introduction

ZEDEDA Edge Virtualization platform allows you to optionally configure edge application instances through the cloud-init mechanism. Cloud-init is the industry standard tool for automatically configuring newly deployed virtual machines by handling essential first-boot tasks like setting up networking, SSH keys, hostname, and user accounts.

The most common type of script to pass in is called a cloud-config script. This is a YAML formatted file that provides simple, readable methods of setting up common configuration items by declaration. It also has the ability to run arbitrary commands for other tasks. You can refer to (https://cloudinit.readthedocs.io/en/latest/topics/examples.html) for more details.

Cloud-Init Support for Virtual Machines and Containers

Both VM and container implementations support versioning of the user-data. This means that the cloud-init configuration is only reapplied (on a purge or reactivate) if the version of the user-data has changed. The version is specified in the meta-data file in the instance-id field.

Every cloud-init configuration must begin with the #cloud-config header:

#cloud-config

Support in virtual machines

For VMs, EVE-OS supports cloud-init configuration using the NoCloud data source. With NoCloud, EVE-OS provides both meta-data and user-data received from the controller to the VM. This is done by placing these files on a virtual CD-ROM in the form of an ISO image. If a workload includes cloud-init support, it can consume these files and processes the configurations or scripts defined in them.

Support in containers

As opposed to the VM implementation, cloud-init containers does not rely on a cloud-init daemon being present in the container image. Instead, the cloud-init configuration is parsed by EVE and manually applied to the container. EVE's implementation supports two formats for user-data

Legacy Format (Available in all supported LTS versions): This format only supports the definition of environment variables in the form of a simple key-value map. The equal sign "=" is used as a delimiter:
```
ENV1=value1
ENV2=value2
```
Original Cloud-Init Format (Available since EVE 11.3): In this format, the user-data is specified like in any standard cloud-init configuration. The current EVE implementation only supports two user-data fields: runcmd and write_files.
```
#cloud-config
runcmd:
  - ENV1=value1
  - ENV2=value2
write_files:
  - path: /etc/injected_file.txt
    permissions: '0644'
    encoding: b64
    content: YmxhYmxh
```

In case of containers there’s also a very important setting that is the cmdline startup command. In case of ZEDEDA cmdline may be overridden by setting of EVE_ECO_CMD environment variable. Thus, if defined, cmdline will contain EVE_ECO_CMD value. It may help to run specific command inside application without rebuilding and redeploy of it.In this way user can run entrypoint with specified arguments (i.e. debugging ones). The setting of EVE_ECO_CMD will replace the entrypoint command defined in the container manifest. Please note, that change of environment variables require restart of the application.

See more specific examples for #cloud-config in Custom Configuration Examples.

Custom Configuration

You can specify the application configuration script as part of 'custom configuration' during edge application creation. When you create an edge application instance from this edge application bundle, this custom configuration script is passed to the edge node as part of the edge application Instance configuration.

This allows instance-specific fields to be accepted by the user when they are created while allowing for the single and bulk of the configuration script—the common bits to be defined with the edge application.

You can further customize your edge application configuration script per instance but without manual intervention for each instance. You can define the common part of the configuration script as it is and pass instance-specific parameters using variables.

For example, if you want to set the hostname of your edge application instance to the name of the edge node it is running on, you can still use a single script for all your application instances but specify the hostname using a system variable and ZEDEDA will automatically replace the system variable with respective edge gateway name when it creates a configuration for the edge application instance.

ZEDEDA platform supports the following types of variables for custom configuration scripts:

Note: ZEDEDA supports cloud-init updates for purge/force updates, provided there are no edits/modifications done on the variable groups. After deploying the edge application instance:

If the changes are made on the variable groups, the changes do not get reflected in the application instance (even after the purge/force update).
The only way to update or make changes to cloud-init, in this case, is to delete and redeploy (with updated cloud-init) the application instance.

ZEDEDA platform system variables

ZEDEDA will automatically replace the system variables with values from the ZEDEDA platform.

Example of supported system variables

Key	Description
$zri.system.edge-node.id	System generated unique id for the edge node in Zedcontrol
$zri.system.edge-node.name	User defined name for the edge node in Zedcontrol
$zri.system.edge-node.arch	Hardware architecture of edge node. Values : AMD64, ARM64
$zri.system.edge-app.id	System generated unique id for the edge application instance in Zedcontrol
$zri.system.edge-app.name	User defined name for the edge application instance in Zedcontrol
$zri.system.edge-instance.ip	Edge Application Instance interface IP address in ZedControl
$zri.system.edge-instance.name	Edge Application Instance name in ZedControl
$zri.system.edge-node.serial	Edge Node serial number in ZedControl as configured during onboarding

ZEDEDA webhook variables

If your edge application is interacting with public IoT services like AWS, Azure, etc., you can define IoTHub or Device Provisioning common parameters as part of the webhook profile and ZEDEDA will automatically replace the variables with values from the configured webhook profile.

Example of supported webhook variables

Key	Description
$webhook.azure.dps.scope_id	Azure DPS group enrollment scope
$webhook.azure.dps.shared_key	Azure DPS group enrollment shared key
$webhook.azure.iothub.ca_cert	Azure IoTHub Intermediate certificate for transparent gateway
$webhook.azure.iothub.ca_key	Azure IoTHub Intermediate private key for transparent gateway
$webhook.azure.iothub.ca_password	Azure IoTHub Intermediate certificate password for transparent gateway

User input variables

Ask the user to provide values during edge application instance creation. Specify any variable name between specific delimiters

Example of user input variables


###my_user_variable### (here ### is the delimiter but any other sequence of 3 characters can be defined as delimiter)

Provide Custom Configuration at Application Instance

When deploying the application instance through the WebUI, the user will be asked to provide the values for the input variables defined in the cloud-init application marketplace template.

These values can be in the form of plain text, boolean values, passwords, dropdown options or text files.

The following example shows the variable inputs at application instance deployment:

Provide Custom Configuration at Application Instance Using ZCLI

It’s important to note that when operating with API automation, there’s no such concept of input variables as this is just a mechanism applicable to the WebUI.

When operating with API automation tools like Terraform or ZCLI, the full cloud-init script can be provided at the application instance time.

In order to enable that the user needs to set the option “Allow Edge App deployments to set entire configuration” to Yes in the Application Marketplace template:

With this option, the full cloud-init configuration must be given at the application instance deployment(this is also applicable to the WebUI).

In case of ZCLI the full cloud-init is given as input to the keyword custom-configuration. The user should provide a .json file where the full cloud-init script must be given as base64 encoded to the template leaf element of that .json file.

Example JSON

Starting from the clear text full cloud-init script:

zcli ls
cloud-init.txt

Encode the content of the cloud-init text file with base64:

zclib64cloudinit=$(base64 -w 0 cloud-init.txt)
zcli echo $b64cloudinit
b250ZW50LVR5cGU6IG11bHRpcGFydC9taXhlZDsgYm91b……….--snip–

Update the template leaf in the .json file:

  zcli cat cloud_init_exampe.json
{
        "name": "cloud-init",
        "add": true,
        "override": true,
        "allowStorageResize": true,
        "fieldDelimiter": "",
        "template": “####Update With the Base64 content of the previous step”####”
        "variableGroups": []
 }

Deploy the application instance and provide the above .json as argument to the –custom-configuration:

zcli> zcli edge-app-instance create example-app-instance --edge-app=ubuntu-app-template --edge-node=test-edge-node-1 --network-instance=eth0:defaultLocal-test-edge-node-1 --custom-configuration=./cloud_init_example.json

The application will then be instantiated with the custom cloud-init script.

Custom Configuration Examples

Following are cloud-config examples.

Example of Ubuntu virtual machines cloud-init

This example sets the hostname as the application edge-instance name ($zri.system.edge-instance.name). Creates an user (pocuser) and sets the ssh authorized key, it also defines the password. It also changes the netplan configuration file to enable DHCP on all interfaces in case there are more than one ethernet interface, and then regenerates and applies the netplan configuration.

#cloud-config
ssh_pwauth: Yes
hostname: $zri.system.edge-instance.name
users:
  - name: pocuser
    ssh-authorized-keys:
      - ssh-ed25519 AAAAC3NzaL1lZFI1NTE5AAAAIKzW56UPME0tlYSSuJliChOm0kxGHdO6kVWkos4No6Wm zededa@adminisrsLaptop.home
    shell: /bin/bash
    lock_passwd: false
    sudo: ALL=(ALL) NOPASSWD:ALL
chpasswd:
  list: |
      pocuser:pocuser
  expire: false
write_files:
  - path: /etc/netplan/01-network-config.yaml
    content: |
      network:
       version: 2
       ethernets:
         alleths:
           match:
             name: en*
           dhcp4: true
  - path: /root/parameters
    content: |
      # Standard ZEDEDA variables available through cloudinit:
      system_edge-node_id="$zri.system.edge-node.id"
      system_edge-node_name="$zri.system.edge-node.name"
      system_edge-node_arch="$zri.system.edge-node.arch"
      system_edge-app_id="$zri.system.edge-app.id"
      system_edge-app_name="$zri.system.edge-app.name"
      system_edge-instance_ip="$zri.system.edge-instance.ip"
      system_edge-instance_name="$zri.system.edge-instance.name"
      system_edge-node_serial="$zri.system.edge-node.serial"
    append: true
runcmd:
  - rm /etc/netplan/50-cloud-init.yaml
  - netplan generate
  - netplan apply

Example of Fortinet virtual machines cloud-init

Fortinet requires the cloud-init data to be provided in a MIME multi-part format, using this format, the user can specify more than one data type (in this case Content-Type is text/plain) and the corresponding filename and path where the appliance is expecting to read the data from the cloud-init drive. In this case the cloud-init configuration data is provided in a “text/plain” data type available in the “openstack/latest/user_data” file present in the cloud-init CDROM.

See how to setup ZCLI at ZEDEDA CLI overview. Be aware that to enable MIME multi-part format in the EVE-OS device you need to execute the following command through ZCLI:

zcli>zcli edge-node update myedgedevice --config=process.cloud-init.multipart:true

You can generate the configuration using the write-mime-multipart utility. To install it in a ubuntu system you can run:

$ sudo apt install cloud-image-utils

You should create the following folder structure, and include the configuration data in the user_data file and the license token under 000. Note that file structure is specific to the Fortinet appliance, other appliances might require a different file layout.

  $ ls -R
.:
openstack

./openstack:
content  latest

./openstack/content:
0000 #Fortinet License Token

./openstack/latest:
User_data #Fortinet Configuration Data

Execute the write-mime-multipart that will generate the full cloud-init script can you can then copy and paste under the ZEDEDA application custom configuration:

$write-mime-multipart openstack/latest/user_data:text/plain openstack/content/0000:text/plain


Content-Type: multipart/mixed; boundary="===============####Sep####=="
MIME-Version: 1.0

--===============####Sep####==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="openstack/latest/user_data"

config system interface
    edit "port1"
        set vdom "root"
        set mode static
        set ip 192.168.0.12 255.255.255.0
        set allowaccess ping https ssh http
        set type physical
        set alias "WAN"
        set snmp-index 1
    next
    edit "port2"
        set vdom "root"
        set ip 192.168.4.1 255.255.255.0
        set allowaccess ping https ssh http
        set type physical
        set alias "LAN"
        set snmp-index 2
    next
end

config system admin
   edit admin
       set password admin
   end

config router static
    edit 1
        set gateway 192.168.0.1
        set device "port1"
    next
end

config system dhcp server
    edit 1
        set ntp-service local
        set default-gateway 192.168.4.1
        set netmask 255.255.255.0
        set interface "port2"
        config ip-range
            edit 1
                set start-ip 192.168.4.2
                set end-ip 192.168.4.254
            next
        end
        set vci-match enable
        set vci-string "FortiSwitch" "FortiExtender"
    next
end

config system dns
    set primary 8.8.8.8
    set secondary 1.1.1.1
end

config firewall policy
    edit 1
        set name "Internet access"
        set srcintf "port2"
        set dstintf "port1"
        set action accept
        set srcaddr "all"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set logtraffic all
        set nat enable
    next
end

config system global
   set admin-ssh-grace-time 3600
end

--===============####Sep####==--
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="openstack/content/0000"
 
LICENSE-TOKEN:YOURLICENSETOKEN
 
--===============####Sep####==--

Example of Cisco 8000v virtual machines cloud-init

Similar to the Fortinet appliance example, the Cisco 8000v also requires the cloud-init data to be provided in a MIME multi-part format as the appliance is expecting to read the configuration data from “openstack/latest/user_data” file present in the cloud-init configuration drive.

See how to setup ZCLI at ZEDEDA CLI overview. Be aware that to enable MIME multi-part format in the EVE-OS device you need to execute the following command through ZCLI:

zcli>zcli edge-node update myedgedevice --config=process.cloud-init.multipart:true

You can generate the following configuration using the write-mime-multipart utility. In this case, the configuration must be provided in the “openstack/latest/user_data” file.

Content-Type: multipart/mixed; boundary="===============4459334010561113521=="
MIME-Version: 1.0

--===============4459334010561113521==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="openstack/latest/user_data"

hostname c8kv-ios_cfg
license smart enable
username lab priv 15 secret lab
ip domain-name cisco.com
interface GigabitEthernet1
ip address 10.0.0.5 255.255.255.0
no shut
exit
ip route 0.0.0.0 0.0.0.0 10.0.0.1
line vty 0 4
login local
exit

--===============4459334010561113521==--

In this example above the full device configuration is given in a text format as part of the custom-configuration text snippet.

Example of cloud-config for AWS greengrass runtime

#cloud-config
# Add a user with ztest / zededaI0T credential and sudo permission
users:
- name: ztest
gecos: ZEDEDA Test User
shell: /bin/bash
sudo: ALL=(ALL) NOPASSWD:ALL
lock_passwd: False
passwd: $6$qAI9dWKMTMLUZBB$vpL6vHitSiCVTx8ZHjIkdnerwEsoiWJIffUi65cp6tC/ic1e5GlcbBo0RI90tEvvvolLaUErvIsKyVZkkeyuj1
write_files:
- path: /greengrass/certs/###greengrass_core_name###.cert.pem
permissions: '0644'
encoding: b64
content: ###cloud_cert_pem###
- path: /greengrass/certs/###greengrass_core_name###.key.pem
permissions: '0444'
encoding: b64
content: ###cloud_key_pem###
# Modify AWS Greengrass config file with greengrass core certificates
runcmd:
- sed -i.orig 's|file://certs/\[ROOT_CA_PEM_HERE\]|file:///greengrass/certs/root.ca.pem|' /greengrass/config/config.json
- sed -i.orig 's|file://certs/\[CLOUD_PEM_CRT_HERE\]|file:///greengrass/certs/###greengrass_core_name###.cert.pem|' /greengrass/config/config.json
- sed -i.orig 's|file://certs/\[CLOUD_PEM_KEY_HERE\]|file:///greengrass/certs/###greengrass_core_name###.key.pem|' /greengrass/config/config.json
- sed -i.orig 's|\[ROOT_CA_PEM_HERE\]|root.ca.pem|' /greengrass/config/config.json
- sed -i.orig 's|\[CLOUD_PEM_CRT_HERE\]|###greengrass_core_name###.cert.pem|' /greengrass/config/config.json
- sed -i.orig 's|\[CLOUD_PEM_KEY_HERE\]|###greengrass_core_name###.key.pem|' /greengrass/config/config.json
- sed -i.orig 's|\[yes\|no\]|yes|' /greengrass/config/config.json
- sed -i.orig 's|\[AWS_REGION_HERE\]|###greengrass_region###|' /greengrass/config/config.json
- sed -i.orig 's|\[HOST_PREFIX_HERE\]|###iotcore_name###|' /greengrass/config/config.json
- sed -i.orig 's|\[THING_ARN_HERE\]|arn:aws:iot:###greengrass_region###:###greengrass_account###:thing/###greengrass_core_name###|' /greengrass/config/config.json
- echo -e "Removing backup AWS Greengrass config file"; rm /greengrass/config/config.json.orig
- echo -e "Restarting AWS Greengrass service \n"; sleep 90; systemctl restart greengrass
- echo -e "Checking AWS Greengrass service status \n"; systemctl status greengrass
# - echo -e "Checking AWS Greengrass module status"; sudo iotedge list

final_message: "AWS Greengrass Core gateway is finally up, after $UPTIME seconds"

Example of cloud-config for Azure IoTEdge runtime

#cloud-config
# Add a user with ztest / zededaI0T credential and sudo permission
users:
- name: ztest
gecos: ZEDEDA Test User
shell: /bin/bash
sudo: ALL=(ALL) NOPASSWD:ALL
lock_passwd: False
passwd: $6$qAI9dWKMTMLUZBB$vpL6vHitSiCVTx8ZHjIkdnerwEsoiWJIffUi65cp6tC/ic1e5GlcbBo0RI90tEvvvolLaUErvIsKyVZkkeyuj1
write_files:
- path: /etc/iotedge/trusted_ca.cert.pem
permissions: '0644'
encoding: b64
content: $webhook.azure.iothub.ca_cert
- path: /etc/iotedge/trusted_ca.key.pem
permissions: '0644'
encoding: b64
content: $webhook.azure.iothub.ca_key
- path: /etc/iotedge/openssl_ca.cnf
permissions: '0644'
content: |
[ ca ]
default_ca = CA_default
[ CA_default ]
dir = /etc/iotedge
database = $dir/index.txt
serial = $dir/serial
policy = policy_loose
[ policy_loose ]
[ v3_intermediate_ca ]
# Extensions for a typical CA.
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer:always
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign
# Modify Azure IoTEdge config file with DPS group enrollment settings
runcmd:
- group_key_bytes=$(echo $webhook.azure.dps.shared_key | base64 --decode | xxd -p -u -c 1000)
- symmetric_key=$(echo -n $zri.system.edge-node.name | openssl sha256 -mac HMAC -macopt hexkey:$group_key_bytes -binary | base64)
- symmetric_key_replacement="s|\"{symmetric_key}\"|\"$symmetric_key\"|"
- sed -i.orig '31,33 s/^/#/' /etc/iotedge/config.yaml
- sed -i.orig '45,52 s/^##* //' /etc/iotedge/config.yaml
- sed -i.orig '48s|"{scope_id}"|"$webhook.azure.dps.scope_id"|' /etc/iotedge/config.yaml
- sed -i.orig '51s|"{registration_id}"|"$zri.system.edge-node.name"|' /etc/iotedge/config.yaml
- sed -i.orig $symmetric_key_replacement /etc/iotedge/config.yaml
- echo -e "Installing IoTEdge gateway certificate chains"
- cd /etc/iotedge
- sudo mkdir -p newcerts
- sudo rm index.txt
- sudo touch index.txt
- sudo rm serial
- sudo bash -c 'echo 1000  serial'
- openssl genrsa -out $zri.system.edge-node.name_ca.key.pem 4096
- chmod 444 $zri.system.edge-node.name_ca.key.pem
- openssl req -new -sha256 -key $zri.system.edge-node.name_ca.key.pem -subj "/CN=$zri.system.edge-node.name_ca" -out $zri.system.edge-node.name_ca.csr
- openssl ca -batch -config openssl_ca.cnf -extensions "v3_intermediate_ca" -days 365 -notext -md sha256 -in $zri.system.edge-node.name_ca.csr -cert trusted_ca.cert.pem -keyfile trusted_ca.key.pem -keyform PEM -passin pass:$webhook.azure.iothub.ca_password -out $zri.system.edge-node.name_ca.cert.pem -outdir newcerts
- chmod 444 $zri.system.edge-node.name_ca.cert.pem
- cat $zri.system.edge-node.name_ca.cert.pem trusted_ca.cert.pem  $zri.system.edge-node.name_ca-full-chain.cert.pem
- chmod 444 $zri.system.edge-node.name_ca-full-chain.cert.pem
- cd -
- sed -i.orig 's|# certificates:|certificates:|' /etc/iotedge/config.yaml
- sed -i.orig 's|# device_ca_cert:| device_ca_cert:|' /etc/iotedge/config.yaml
- sed -i.orig 's|""|"/etc/iotedge/$zri.system.edge-node.name_ca-full-chain.cert.pem"|' /etc/iotedge/config.yaml
- sed -i.orig 's|# device_ca_pk:| device_ca_pk:|' /etc/iotedge/config.yaml
- sed -i.orig 's|""|"/etc/iotedge/$zri.system.edge-node.name_ca.key.pem"|' /etc/iotedge/config.yaml
- sed -i.orig 's|# trusted_ca_certs:| trusted_ca_certs:|' /etc/iotedge/config.yaml
- sed -i.orig 's|""|"/etc/iotedge/trusted_ca.cert.pem"|' /etc/iotedge/config.yaml
- echo -e "Removing backup IoTEdge config file"; rm /etc/iotedge/config.yaml.orig
- echo -e "Restarting IoTEdge service \n"; sleep 90; systemctl restart iotedge
- echo -e "Checking IoTEdge service status \n"; systemctl status iotedge
- echo -e "Checking IoTEdge module status"; sudo iotedge list

final_message: "Azure IoT Edge gateway is finally up, after $UPTIME seconds"

Example of cloud-init using input variables

In the previous examples, the full appliance configuration is given in the text body of the cloud-init script defined in the marketplace application. We might have situations where we want to customize the configuration to be applied to each application instance. For that use-case ZEDEDA provides the capability to inject the full cloud-init configuration or parts of it at the application deployment time. Taking the example of the Cisco 8000v appliance from the previous example, we can define the Cisco configuration commands in a text file variable that can then be given as input during the application deployment. The following figure presents how the Application Marketplace template looks like in the WebUI. We can see that the reference to the ###cisco_config_file### is actually a file input variable to be given at the Application Instance deployment time.

The input file configuration file can be a xml or a plain text containing the initial device configuration as described Day 0 Configuration.

Example of cloud-init in a container

In this example we provide the cloud-init for a grafana container application. We provide 2 environment variables “GODEBUG” and “HOSTNAME” along with a custom grafana dashboard(dashboard.json) definition that can be given as input at the time of the application instance deployment.

#cloud-config
runcmd:
 - GODEBUG='netdns=cgo+2'
 - HOSTNAME="grafana-app"
write_files:
  - path: /etc/grafana/provisioning/dashboards/dashboard.json
    permissions: '0777'
    encoding: b64
    content: ###MY_FILE###

If you need to change the entrypoint command and/or its arguments, just include the new entrypoint command and all its variables in the EVE_EVO_CMD variable.

#cloud-config
runcmd:
 - GODEBUG='netdns=cgo+2'
 - HOSTNAME="grafana-app"
 - EVE_ECO_CMD="/run.sh –debug"
write_files:
  - path: /etc/grafana/provisioning/dashboards/dashboard.json
    permissions: '0777'
    encoding: b64
    content: ###MY_FILE###

Next Steps

You can view the status and details of an edge app's custom configuration in the edge app's details page in the Configuration section.

Was this article helpful?

5 out of 11 found this helpful