Docker Guide: Difference between revisions

From Variscite Wiki
No edit summary
No edit summary
 
(76 intermediate revisions by 6 users not shown)
Line 1: Line 1:
<!-- Set release according to "release" parameter in URL and use RELEASE_THUD_V1.0_VAR-SOM-MX6 as default
--> {{INIT_RELEASE_PARAM|RELEASE_THUD_V1.0_VAR-SOM-MX6}}<!--
--> {{#lst:Yocto_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!--
--> {{#lst:B2QT_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!--
--> {{#lst:Debian_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!--
--> {{#vardefine:GUIDE_NAME|{{#if: {{#var:B2QT_GIT}} | B2QT | Yocto }}}} <!--
--> {{#vardefine:DOCKER_INTEGRATION_PREFIX|{{#varexists:DEBIAN_NAME | DEBIAN_ |}}}} <!--
--> {{PageHeader|{{#var:HARDWARE_NAME}} - Docker}} {{DocImage|category1=Yocto|category2=Debian}} <!--
--> {{VAR_FILES_DOMAIN}} <!--
--> [[category:{{#var:HARDWARE_NAME}}]] __toc__
= Docker Introduction =
= Docker Introduction =
<https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-debian-9>
Docker is a software platform that allows the creation, testing and distribution of applications via containers.<br>
Docker is an application that simplifies the process of managing application processes in containers.<br>
Containers let you run your applications in resource-isolated processes. There are some similarities to virtual machines, but  
Containers let you run your applications in resource-isolated processes. They’re similar to virtual machines, but containers<br>
containers are more portable, more resource-friendly, and enables faster software delivery cycles.<br>
are more portable, more resource-friendly, and more dependent on the host operating system.<br>
Often used with docker there's a companion tool known as docker-compose: compose is used for defining and running multi-container Docker applications.<br>
<br>
<br>
For a detailed introduction to the different components of a Docker container, check out The Docker Ecosystem: An Introduction to<br> Common Components.<br>
You can check Docker online documentation at:<br>
<br>
https://docs.docker.com/get-started/<br>
You can check SWUpdate's online documentation at: [http://sbabic.github.io/swupdate http://sbabic.github.io/swupdate]<br>
https://docs.docker.com/compose/<br>
This Introduction was taken from the SWUpdate README.md at: [https://github.com/sbabic/swupdate https://github.com/sbabic/swupdate]<br>
https://training.play-with-docker.com/<br>
<br>
https://docker-curriculum.com/<br>
Example of using the meta-swupdate layer with Variscite i.MX boards is provided in the meta-variscite-fslc layer.<br>
This example uses the double-copy approach, using two rootfs partitions on eMMC.<br>


== Double copy with fall-back ==
{{#switch: {{#var:SOC_SERIES}}
If there is enough space on the storage to save two copies of the whole software, it is possible to guarantee that there is always a working copy even if the software update is interrupted or a power off occurs.
| am6=
|#default=
{{{{#var:DOCKER_INTEGRATION_PREFIX}}DOCKER_INTEGRATION}}
}}


Each copy must contain the kernel, the root file system, and each further component that can be updated. It is required a mechanism to identify which version is running.
= Test Docker =
* Check docker version:<br>
root@{{#var:MACHINE_NAME}}:~# docker --version


SWUpdate should be inserted in the application software, and the application software will trigger it when an update is required. The duty of SWUpdate is to update the stand-by copy, leaving the running copy of the software untouched.
== Test hello-world ==


A synergy with the boot loader is often necessary, because the boot loader must decide which copy should be started. Again, it must be possible to switch between the two copies. After a reboot, the boot loader decides which copy should run.<br>
* Pull the hello-world image from Docker Hub:<br>
[[File:Double_copy_layout.png]]<br>
root@{{#var:MACHINE_NAME}}:~# docker pull hello-world
Summary:
* Requires twice as much space than single copy
* Guarantees there’s always a working copy!
* Bootloader is in charge of booting proper image


== Single copy – running as standalone image ==
* List docker images:<br>
The software upgrade application consists of kernel (maybe reduced dropping not required drivers) and a small root file system, with the application and its libraries. The whole size is much less than a single copy of the system software (2.5MB-8 MB).The system can be put in “upgrade” mode, simply signaling to the boot loader that the upgrading software must be started (either by using boot loader environment or GPIO).
root@{{#var:MACHINE_NAME}}:~# docker images


The boot loader starts “SWUpdate”, booting the SWUpdate kernel and the initrd image as root file system. Because it runs in RAM, it is possible to upgrade the whole storage. Differently as in the double-copy strategy, the systems must reboot to put itself in update mode.
* Run hello-world container:<br>
 
root@{{#var:MACHINE_NAME}}:~# docker run hello-world
This concept consumes less space in storage as having two copies, but it does not guarantee a fall-back without updating again the software. However, it can be guaranteed that the system goes automatically in upgrade mode when the productivity software is not found or corrupted, as well as when the upgrade process is interrupted for some reason. <br>
[[File:Single_copy_layout.png]]<br>
Hello from Docker!<br>
Summary:
This message shows that your installation appears to be working correctly.
* Stand alone image consists of kernel / dt + initrd
* Much smaller than entire system
To generate this message, Docker took the following steps:<br>
* Bootloader in charge of loading standalone image
  1. The Docker client contacted the Docker daemon.<br>
* System must reboot to enter update process
  2. The Docker daemon pulled the "hello-world" image from the Docker Hub.<br>
  3. The Docker daemon created a new container from that image which runs the<br>
    executable that produces the output you are currently reading.<br>
  4. The Docker daemon streamed that output to the Docker client, which sent it<br>
    to your terminal.


== Update image format ==
== Test Ubuntu ==
The main concept is that the manufacturer delivers a single .swu image.<br>
* Pull the ubuntu image from Docker Hub:<br>
The .swu image may contain multiple images and files. In addition, it must contain a sw-description file with meta information about the image.<br>
root@{{#var:MACHINE_NAME}}:~# docker pull ubuntu
[[File:Image_format.png]]


See the following link for info about the sw-description file syntax: [https://sbabic.github.io/swupdate/sw-description.html https://sbabic.github.io/swupdate/sw-description.html]
* Run ubuntu:<br>
root@{{#var:MACHINE_NAME}}:~# docker run -it ubuntu<br>
root@8637148015f5:/# apt update<br>
…<br>
root@8637148015f5:/# apt install nodejs<br>
…<br>
root@8637148015f5:/# node -v<br>
…<br>
root@8637148015f5:/# exit<br>
root@{{#var:MACHINE_NAME}}:~#


== Update methods ==
= Test docker-compose =
=== Update via console ===
* Check the docker-compose version:<br>
To start SWUpdate expecting the image from a file:
  root@{{#var:MACHINE_NAME}}:~# docker-compose --version
swupdate -H {{#var:MACHINE_NAME}}:1.0 -i <filename.swu>
The -H option and its parameter is actually optional. The command can be simplified to
swupdate -i <filename.swu>
as long as a file named
/etc/hwrevision
is present in the rootfs and it's populated with the following content
  {{#var:MACHINE_NAME}} 1.0
Please note that no multiple instances of swupdate can be running simultaneously: if the roofs contains the service to start the swupdate web-server, the server must be stopped before running swupdate from command line
/etc/init.d/swupdate stop
You can schedule an auto update from a removable storage by editing the automount sequence, making it trigger the update command above, each time a removable device is inserted.


=== Update via http ===
== Examples ==
OTA update is available via http protocol:
It is possible to configure the software update agent to include embedded web-server (CONFIG_MONGOOSE). To start SWUpdate with the embedded web-server:
swupdate -w "--document-root ./www"


The default port of the Web-server is 8080. You can then connect to the target with:
=== RESTful-based data-logger ===
<nowiki>http://<target_ip>:8080</nowiki>


And upload the .swu file.<br>
This [{{#var:VAR_FILES_HTTPS}}/docker/Docker_Example01_RestDataLogger.zip archive] provides an example of a RESTful-based data-logger to be deployed on an embedded device acting as an intelligent edge node in large connected networks.<br>
{{#ifeq: {{#var:YOCTO_NAME}} | Pyro |
Through a developed web service application based on the ''Tornado web framework'', data can be easily inserted and retrieved from a ''MongoDB'' instance by remote devices.
[[File:Swupdate-www.png|600px]]
|
{{#ifeq: {{#var:YOCTO_NAME}} | Rocko |
[[File:Swupdate-www.png|600px]]
|
<br>The start page:<br>
[[File:Swupdate-www_sumo2.png|700px]]<br>
And after a successful update:<br>
[[File:Swupdate-www_sumo.png|700px]]
}}}}
<br><br>
To see all of the command line parameters of swupdate:
swupdate -h


= Yocto Integration =
In the tutorial the application is proposed as a '''Docker'''-based solution in which each component is run in a separate container.<br>
'''Docker Compose''' is used to simplify the build of custom Docker images, their deployment, and configuration.<br>
The application can be easily tested on any device which supports Docker and Docker Compose, be it a PC or an embedded device.


Variscite provides an example for the double-copy approach, where the rootfs is on eMMC.
=== PyQt application ===
== Recipes ==


Before you begin, get familiar with the Varicite {{Varlink2|Yocto Build Release|{{#var:RELEASE_LINK}}}} guide.<br>
This [{{#var:VAR_FILES_HTTPS}}/docker/Docker_Example02_PyQtApplication.zip archive] provides a PyQt reference application . The Wayland graphic server is abstracted as resource to be passed to a Docker image in which the application is deployed.
<br>
Variscite provides the following two image recipes:


* '''var-image-swupdate''' - Based on fsl-image-gui (the default Variscite demo image with a GUI without any Qt content) with the following differences:
In the Dockerfile the reference buster-slim image is used as a starting point, then python3 and all dependencies necessary to execute Qt applications are installed. Wayland backend has been chosen because the only one available for all Variscite SoMs.
** swupdate and swupdate-www pacakages are included
** The kernel image and device trees are added to the /boot directory of the rootfs (by including the kernel-image and kernel-devicetree packages), instead of using a BOOT partition
** The rootfs is created only as a tar.gz archive (IMAGE_FSTYPES = "tar.gz")
See the recommended internal storage layout for double-copy approach: <br>
[[File:Swupdate_layout.png]]
<br>
* '''var-image-swu''' - This recipe creates an .swu update image using the var-image-swupdate image
<br>
The following is the sw-description file used by the Variscite sample update image recipe (var-image-swu).<br>
This sample image consists of two components:<br>
- A root file system (including the kernel), which is created by the var-image-swupdate recipe<br>
- A bash script that runs before (called with preinst argument) and after (called with postinst argument) the installation of the root file system.<br>
 
software =
{
version = "0.1.0";
{{#var:MACHINE_NAME}} = {
hardware-compatibility: [ "1.0" ];
files: (
{
filename = "var-image-swupdate-{{#var:MACHINE_NAME}}.tar.gz";
type = "archive";
compressed = true;
device = "/dev/update";
filesystem = "ext4";
path = "/";
}
);
scripts: (
{
filename = "update.sh";
type = "shellscript";
}
);
};
}
<br>
And the following is the aforementioned update.sh bash script: [https://raw.githubusercontent.com/varigit/meta-variscite-fslc/{{#var:YOCTO_BRANCH}}/recipes-support/swupdate/var-image-swu/update.sh update.sh]<br>
This script is being used in order to detect the root device and the U-Boot env. type and location (eMMC vs NAND flash) at runtime.<br>
 
== Create a SWUpdate-ready Image ==
* Follow steps 1-3 of the {{Varlink2|Yocto Build Release|{{#var:RELEASE_LINK}}}} page.
* Build a recovery SD card and the var-image-swupdate image as the recovery image
$ cd {{#var:BUILD_FOLDER}}
{{#ifeq: {{#var:SOC_SERIES}} | imx8 |
$ MACHINE={{#var:MACHINE_NAME}} DISTRO={{#var:BUILD_DISTRO}} {{#var:BUILD_SCRIPT}} -b {{#var:BUILD_FOLDER_XWAYLAND}}
|
$ MACHINE={{#var:MACHINE_NAME}} DISTRO={{#var:BUILD_DISTRO}} {{#var:BUILD_SCRIPT}} {{#var:BUILD_FOLDER_X11}}
}}


$ bitbake fsl-image-gui      (for the recovery SD card)
=== GStreamer application ===
$ bitbake var-image-swupdate (for the rootfs to be installed on the eMMC)
* Create the "SWUpdate recovery SD card"
$ sudo MACHINE={{#var:MACHINE_NAME}} ../sources/meta-variscite-{{#varexists: FSLC_BSP_VERSION |fslc|imx}}/scripts/var_mk_yocto_sdcard/var-create-yocto-sdcard.sh -r tmp/deploy/images/{{#var:MACHINE_NAME}}/var-image-swupdate-{{#var:MACHINE_NAME}} /dev/sdX
(Replace /dev/sdX with your actual device)
* Boot the board using the created SD card. See more info {{Varlink|Yocto Build Release#Boot_the_board_with_a_bootable_SD_card|{{#var:RELEASE_LINK}}|here}}.
* Install the demo image to the eMMC{{#ifeq: {{#var:SOC_SERIES}} | imx8 |&nbsp;using|. Either use the installation icons on the desktop (which will be updated with the "-u" parameter by the var-create-yocto-sdcard.sh script, when the recovery image used is var-image-swupdate), or use}} the install_yocto.sh script {{#ifeq: {{#var:SOC_SERIES}} | imx8||manually,}} with the "-u" parameter.
{{Note|When called with the "-u" parameter, the install_yocto.sh script creates the double-copy compatible layout as shown above}}


== Generate a Field Update file (.swu image) ==
This [{{#var:VAR_FILES_HTTPS}}/docker/Docker_Example03_GStreamer.zip archive] provides a Gstreamer test application . The Wayland graphic server is abstracted as resource to be passed to a Docker image in which the application is deployed.
*  Make your changes in {{#var:BUILD_FOLDER}}
*  Edit the sw-description file
*  Generate .swu file:
$ bitbake var-image-swu


== Test SWUpdate  ==
In the Dockerfile the reference buster-slim image is used as a starting point, then gstreamer1.0 and all dependencies necessary to execute GStreamer applications are installed. Wayland backend has been chosen because the only one available for all Variscite SoMs.
* Get the target's IP address using ifconfig
* Open a web browser on the host and connect to the SWUpdate web server on the target by entering the following in the address bar:
<nowiki>http://<target_ip>:8080</nowiki>
* On the web page, click "Browse..." and open the .swu image which is located at tmp/deploy/images/{{#var:MACHINE_NAME}}/var-image-swu-{{#var:MACHINE_NAME}}.swu

Latest revision as of 14:41, 1 December 2023

Warning: This page is designed to be used with a 'release' URL parameter.

This page is using the default release RELEASE_THUD_V1.0_VAR-SOM-MX6.
To view this page for a specific Variscite SoM and software release, please follow these steps:

  1. Visit variwiki.com
  2. Select your SoM
  3. Select the software release
VAR-SOM-MX6 - Docker

Docker Introduction

Docker is a software platform that allows the creation, testing and distribution of applications via containers.
Containers let you run your applications in resource-isolated processes. There are some similarities to virtual machines, but containers are more portable, more resource-friendly, and enables faster software delivery cycles.
Often used with docker there's a companion tool known as docker-compose: compose is used for defining and running multi-container Docker applications.

You can check Docker online documentation at:
https://docs.docker.com/get-started/
https://docs.docker.com/compose/
https://training.play-with-docker.com/
https://docker-curriculum.com/

Yocto Integration

Recipe

Before you begin, get familiar with the Yocto Build Release guide.

The Yocto BSP provides 2 dedicated images:

  • var-image-docker: a graphic enabled image based on fsl-image-gui with the addition of a few more packages:
    • docker
    • python3-docker-compose
  • var-image-docker-minimal: a console image based on the core-image bbclass with the addition of a few more packages and features:
    • docker
    • python3-docker-compose
    • ssh-server-dropbear
    • basic command line tools

Create a Docker-ready Image

$ cd ~/var-fslc-yocto
$ MACHINE=var-som-mx6 DISTRO=fslc-x11 . setup-environment build_x11


Append the following to the conf/local.conf file in your Yocto build directory:

DISTRO_FEATURES_append = " virtualization"

Build var-image-docker:

$ bitbake var-image-docker

Test Docker

  • Check docker version:
root@var-som-mx6:~# docker --version

Test hello-world

  • Pull the hello-world image from Docker Hub:
root@var-som-mx6:~# docker pull hello-world
  • List docker images:
root@var-som-mx6:~# docker images
  • Run hello-world container:
root@var-som-mx6:~# docker run hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly. To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.

Test Ubuntu

  • Pull the ubuntu image from Docker Hub:
root@var-som-mx6:~# docker pull ubuntu
  • Run ubuntu:
root@var-som-mx6:~# docker run -it ubuntu
root@8637148015f5:/# apt update

root@8637148015f5:/# apt install nodejs

root@8637148015f5:/# node -v

root@8637148015f5:/# exit
root@var-som-mx6:~#

Test docker-compose

  • Check the docker-compose version:
root@var-som-mx6:~# docker-compose --version

Examples

RESTful-based data-logger

This archive provides an example of a RESTful-based data-logger to be deployed on an embedded device acting as an intelligent edge node in large connected networks.
Through a developed web service application based on the Tornado web framework, data can be easily inserted and retrieved from a MongoDB instance by remote devices.

In the tutorial the application is proposed as a Docker-based solution in which each component is run in a separate container.
Docker Compose is used to simplify the build of custom Docker images, their deployment, and configuration.
The application can be easily tested on any device which supports Docker and Docker Compose, be it a PC or an embedded device.

PyQt application

This archive provides a PyQt reference application . The Wayland graphic server is abstracted as resource to be passed to a Docker image in which the application is deployed.

In the Dockerfile the reference buster-slim image is used as a starting point, then python3 and all dependencies necessary to execute Qt applications are installed. Wayland backend has been chosen because the only one available for all Variscite SoMs.

GStreamer application

This archive provides a Gstreamer test application . The Wayland graphic server is abstracted as resource to be passed to a Docker image in which the application is deployed.

In the Dockerfile the reference buster-slim image is used as a starting point, then gstreamer1.0 and all dependencies necessary to execute GStreamer applications are installed. Wayland backend has been chosen because the only one available for all Variscite SoMs.