Docker Build Environment: Difference between revisions

From Variscite Wiki
No edit summary
 
(13 intermediate revisions by 2 users not shown)
Line 18: Line 18:
https://docker-curriculum.com/<br>
https://docker-curriculum.com/<br>
<br>
<br>
This guide describes how to use Docker as a Host build environment for building Variscite Yocto, Debian, Android and Boot2Qt images.
This guide describes how to use Docker as a Host build environment for building Variscite Yocto, Android, Debian and Boot2Qt images.
<br><br>
<br><br>
If you are interested in running Docker containers on top of Yocto, Debian or Boot2Qt, please see our {{Varlink|Docker Guide |{{#var:RELEASE_LINK}}|Docker Guide}}.
If you are interested in running Docker containers on top of Yocto, Debian or Boot2Qt, please see our {{Varlink|Docker Guide |{{#var:RELEASE_LINK}}|Docker Guide}}.
Line 24: Line 24:
= Why Build using Docker? =
= Why Build using Docker? =


Each version of Yocto, Debian, Android, etc. has unique Host requirements (e.g. Ubuntu 16, Ubuntu 18...). This can be cumbersome during development if you are working on legacy projects and new projects at the same time. Over time, we've all come up with our own solutions to this problem by using Virtual Machines or even keeping old computers just to build an occasional image.
Each version of Yocto, Android, Debian and Boot2Qt has unique Host requirements (e.g. Ubuntu 16.04, Ubuntu 18.04...). This can be cumbersome during development if you are working on legacy projects and new projects at the same time. Over time, we've all come up with our own solutions to this problem by using Virtual Machines or even keeping old computers just to build an occasional image.
<br><br>
<br><br>
Docker is a great solution to this problem and provides:
Docker is a great solution to this problem and provides:
Line 47: Line 47:
Install the latest LTS version of Ubuntu: https://ubuntu.com/download
Install the latest LTS version of Ubuntu: https://ubuntu.com/download


After a fresh installation, install Docker
After a fresh installation, install Docker:


  $ sudo apt update && sudo apt install docker.io
  $ sudo apt update && sudo apt install docker.io qemu-user-static


Give permissions to run Docker without sudo
Give permissions to run Docker without sudo:


  $ sudo usermod -aG docker ${USER}
  $ sudo usermod -aG docker ${USER}


Install Linux Headers (required for some Yocto versions)
Install Linux Headers (required for some Yocto versions):


  $ sudo apt install linux-headers-$(uname -r)
  $ sudo apt install linux-headers-$(uname -r)


Logout and login again for permissions to take effect
Logout and login again for permissions to take effect.


= Using Variscite's Docker Container =
= Using Variscite's Docker Container =


Clone Variscite's Docker container repository
== Get the source code ==


  $ git clone https://github.com/varigit/var-host-docker-containers.git ~/var-host-docker-containers
Clone Variscite's Docker container repository:
 
$ git clone https://github.com/varigit/var-host-docker-containers.git ~/var-host-docker-containers
 
== Starting a container ==


Start your first Docker container. The first time you start the container, it will take some time to build the container.
Start your first Docker container. The first time you start the container, it will take some time to build the container.
Line 71: Line 75:
After the container finishes building, it will enter the docker container with an interactive shell in the current working directory.
After the container finishes building, it will enter the docker container with an interactive shell in the current working directory.


Start Ubuntu 16.04 Container:
Start a Ubuntu 14.04 container:
 
$ cd ~/var-host-docker-containers
$ ./run.sh -u 14.04
vari@908b2ecb107c:/workdir$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 14.04.6 LTS"
 
Start a Ubuntu 16.04 container:
 
$ cd ~/var-host-docker-containers
$ ./run.sh -u 16.04
vari@908b2ecb107c:/workdir$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS"
 
Start a Ubuntu 18.04 container:


  $ cd ~/var-host-docker-containers
$ cd ~/var-host-docker-containers
  $ ./run.sh -u 16.04
$ ./run.sh -u 18.04
  vari@908b2ecb107c:/workdir$ grep DESCRIPTION /etc/lsb-release
vari@4822cf0e987c:/workdir$ grep DESCRIPTION /etc/lsb-release
  DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS"
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"


Start Ubuntu 18.04 Container:
Start a Ubuntu 20.04 container:


  $ cd ~/var-host-docker-containers
$ cd ~/var-host-docker-containers
  $ ./run.sh -u 18.04
$ ./run.sh -u 20.04
  vari@4822cf0e987c:/workdir$ grep DESCRIPTION /etc/lsb-release
vari@abe078d0f67c:/workdir$ grep DESCRIPTION /etc/lsb-release
  DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"


Start Ubuntu 20.04 Container:
Start a Ubuntu 22.04 container:


  $ cd ~/var-host-docker-containers
$ cd ~/var-host-docker-containers
  $ ./run.sh -u 20.04
$ ./run.sh -u 22.04
  vari@abe078d0f67c:/workdir$ grep DESCRIPTION /etc/lsb-release
vari@abe078d0f67c:/workdir$ grep DESCRIPTION /etc/lsb-release
  DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"


See run options:
See run options:


  $ cd ~/var-host-docker-containers
$ cd ~/var-host-docker-containers
  $ ./run.sh -h
$ ./run.sh -h
 
== Using the sudo command ==


= Building an Image =
If you need to run a command using 'sudo' inside the container, the default password is 'ubuntu'


This example shows how to build a Yocto Dunfell for the {{#var:HARDWARE_NAME}} using Docker. In general, the process is the same for all target Operating Systems. Please follow Variscite's build guide and host requirements for your specific releases.
= Building an image =
 
== Example: Yocto ==
 
This example shows how to build a Yocto Dunfell image for the {{#var:HARDWARE_NAME}} using Docker. In general, the process is the same for all target Operating Systems. Please follow Variscite's build guide and host requirements for your specific releases.


Make a working directory from your Host OS:
Make a working directory from your Host OS:


  $ mkdir ~/docker-yocto-build
$ mkdir ~/docker-yocto-build
 
Download the source code:
{{#ifeq: {{#var:SOC_SERIES}} | imx8 |
{{#vardefine: SYNC_CMD |repo init -u https://github.com/varigit/variscite-bsp-platform.git -b refs/tags/{{#var:RELEASE_NAME}} -m {{#var:YOCTO_MANIFEST}}}}
|
{{#vardefine: SYNC_CMD |repo init -u https://github.com/varigit/variscite-bsp-platform.git -b refs/tags/{{#var:RELEASE_NAME}}}}
}}
$ cd ~/docker-yocto-build
$ {{#var:SYNC_CMD}}
$ repo sync -j4


Start a Ubuntu 20.04 Docker container:
Start a Ubuntu 20.04 Docker container:


  $ cd ~/var-host-docker-containers
$ cd ~/var-host-docker-containers
  $ ./run.sh -p -u 20.04 -w ~/docker-yocto-build
$ ./run.sh -p -u 20.04 -w ~/docker-yocto-build


Note: -p option starts Docker in privileged mode, which allows access to Host devices and is necessary for building Debian images
Note: the "-p" option starts Docker in privileged mode, which allows access to host devices and is necessary for building Debian images.


{{#ifeq: {{#var:SOC_SERIES}} | imx8 |
Build a Yocto image in the container:
vari@abe078d0f67c:/workdir$ repo init -u https://github.com/varigit/variscite-bsp-platform.git -b refs/tags/{{#var:RELEASE_NAME}} -m {{#var:YOCTO_MANIFEST}}
vari@abe078d0f67c:/workdir$ repo sync -j4
|
vari@abe078d0f67c:/workdir$ repo init -u https://github.com/varigit/variscite-bsp-platform.git -b refs/tags/{{#var:RELEASE_NAME}}
vari@abe078d0f67c:/workdir$ repo sync -j4
}}
  vari@abe078d0f67c:/workdir$ MACHINE={{#var:MACHINE_NAME}} DISTRO=fslc-xwayland . setup-environment build_xwayland
  vari@abe078d0f67c:/workdir$ MACHINE={{#var:MACHINE_NAME}} DISTRO=fslc-xwayland . setup-environment build_xwayland
  vari@abe078d0f67c:/workdir$ source setup-environment build_xwayland
  vari@abe078d0f67c:/workdir$ source setup-environment build_xwayland
Line 126: Line 154:


  $ ls -l ~/docker-yocto-build/build_xwayland/tmp/deploy/images/{{#var:MACHINE_NAME}}/
  $ ls -l ~/docker-yocto-build/build_xwayland/tmp/deploy/images/{{#var:MACHINE_NAME}}/
== Example: Boot2Qt ==
This example shows how to build a B2Qt Dunfell image for the {{#var:HARDWARE_NAME}} using Docker. In general, the process is the same for all target Operating Systems. Please follow Variscite's build guide and host requirements for your specific releases.
Make a working directory from your Host OS:
$ mkdir -p ~/var-b2qt/sources/
Start a Ubuntu 20.04 Docker container:
$ cd ~/var-host-docker-containers
$ ./run.sh -p -u 20.04 -w ~/var-b2qt
Get the source code:
$ cd sources
$ git clone https://github.com/varigit/meta-variscite-boot2qt.git -b dunfell-var01
$ cd ..
$ ./sources/meta-variscite-boot2qt/b2qt-init-build-env init --device {{#var:MACHINE_NAME}}
Build the image:
$ MACHINE=imx8mm-var-dart source ./setup-environment.sh
$ MACHINE=imx8mm-var-dart bitbake b2qt-embedded-qt6-image
The output images can be accessed from your host computer:
$ ls -l ~/var-b2qt/build-imx8mm-var-dart/tmp/deploy/images/imx8mm-var-dart/b2qt-embedded-qt6-image-imx8mm-var-dart.img

Latest revision as of 14:52, 5 December 2022

Docker Build Environment

Docker Introduction

Docker is a software platform that allows the creation, testing and distribution of applications via containers.
Containers let you run your applications in resource-isolated processes. There are some similarities to virtual machines, but containers are more portable, more resource-friendly, and enable faster software delivery cycles.
Often used with docker, there's a companion tool known as docker-compose: compose is used for defining and running multi-container Docker applications.

You can check Docker online documentation at:
https://docs.docker.com/get-started/
https://docs.docker.com/compose/
https://training.play-with-docker.com/
https://docker-curriculum.com/

This guide describes how to use Docker as a Host build environment for building Variscite Yocto, Android, Debian and Boot2Qt images.

If you are interested in running Docker containers on top of Yocto, Debian or Boot2Qt, please see our Docker Guide.

Why Build using Docker?

Each version of Yocto, Android, Debian and Boot2Qt has unique Host requirements (e.g. Ubuntu 16.04, Ubuntu 18.04...). This can be cumbersome during development if you are working on legacy projects and new projects at the same time. Over time, we've all come up with our own solutions to this problem by using Virtual Machines or even keeping old computers just to build an occasional image.

Docker is a great solution to this problem and provides:

  • A git revision controlled build environment
  • Can run on any OS and hardware that supports the Docker Runtime
  • Does not require strict allocation of RAM and hard drive memory
  • Has very little CPU/IO overhead, does not impact build time
  • Containers start and stop in 1-2 seconds


Docker's efficiency allows multiple containers to be run simultaneously without burdening the host OS.

Setting Up Host Computer

Host Hardware

For building Linux / Android images, it is recommended to have a dedicated build computer.
A machine with a 16 cores/32 threads CPU, 64GiB of RAM and a 2TB SSD is recommended. Compile time is generally reduced linearly with the number of CPU cores.

Host OS

Install the latest LTS version of Ubuntu: https://ubuntu.com/download

After a fresh installation, install Docker:

$ sudo apt update && sudo apt install docker.io qemu-user-static

Give permissions to run Docker without sudo:

$ sudo usermod -aG docker ${USER}

Install Linux Headers (required for some Yocto versions):

$ sudo apt install linux-headers-$(uname -r)

Logout and login again for permissions to take effect.

Using Variscite's Docker Container

Get the source code

Clone Variscite's Docker container repository:

$ git clone https://github.com/varigit/var-host-docker-containers.git ~/var-host-docker-containers

Starting a container

Start your first Docker container. The first time you start the container, it will take some time to build the container.

After the container finishes building, it will enter the docker container with an interactive shell in the current working directory.

Start a Ubuntu 14.04 container:

$ cd ~/var-host-docker-containers
$ ./run.sh -u 14.04
vari@908b2ecb107c:/workdir$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 14.04.6 LTS"

Start a Ubuntu 16.04 container:

$ cd ~/var-host-docker-containers
$ ./run.sh -u 16.04
vari@908b2ecb107c:/workdir$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 16.04.7 LTS"

Start a Ubuntu 18.04 container:

$ cd ~/var-host-docker-containers
$ ./run.sh -u 18.04
vari@4822cf0e987c:/workdir$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"

Start a Ubuntu 20.04 container:

$ cd ~/var-host-docker-containers
$ ./run.sh -u 20.04
vari@abe078d0f67c:/workdir$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"

Start a Ubuntu 22.04 container:

$ cd ~/var-host-docker-containers
$ ./run.sh -u 22.04
vari@abe078d0f67c:/workdir$ grep DESCRIPTION /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"

See run options:

$ cd ~/var-host-docker-containers
$ ./run.sh -h

Using the sudo command

If you need to run a command using 'sudo' inside the container, the default password is 'ubuntu'

Building an image

Example: Yocto

This example shows how to build a Yocto Dunfell image for the DART-MX8M-MINI using Docker. In general, the process is the same for all target Operating Systems. Please follow Variscite's build guide and host requirements for your specific releases.

Make a working directory from your Host OS:

$ mkdir ~/docker-yocto-build

Download the source code:

$ cd ~/docker-yocto-build
$ repo init -u https://github.com/varigit/variscite-bsp-platform.git -b refs/tags/dunfell-fslc-5.4-2.1.x-mx8mm-v1.4 -m default.xml
$ repo sync -j4

Start a Ubuntu 20.04 Docker container:

$ cd ~/var-host-docker-containers
$ ./run.sh -p -u 20.04 -w ~/docker-yocto-build

Note: the "-p" option starts Docker in privileged mode, which allows access to host devices and is necessary for building Debian images.

Build a Yocto image in the container:

vari@abe078d0f67c:/workdir$ MACHINE=imx8mm-var-dart DISTRO=fslc-xwayland . setup-environment build_xwayland
vari@abe078d0f67c:/workdir$ source setup-environment build_xwayland
vari@abe078d0f67c:/workdir$ bitbake fsl-image-gui

The output images can be accessed from your host computer:

$ ls -l ~/docker-yocto-build/build_xwayland/tmp/deploy/images/imx8mm-var-dart/

Example: Boot2Qt

This example shows how to build a B2Qt Dunfell image for the DART-MX8M-MINI using Docker. In general, the process is the same for all target Operating Systems. Please follow Variscite's build guide and host requirements for your specific releases.

Make a working directory from your Host OS:

$ mkdir -p ~/var-b2qt/sources/

Start a Ubuntu 20.04 Docker container:

$ cd ~/var-host-docker-containers
$ ./run.sh -p -u 20.04 -w ~/var-b2qt

Get the source code:

$ cd sources
$ git clone https://github.com/varigit/meta-variscite-boot2qt.git -b dunfell-var01
$ cd ..
$ ./sources/meta-variscite-boot2qt/b2qt-init-build-env init --device imx8mm-var-dart

Build the image:

$ MACHINE=imx8mm-var-dart source ./setup-environment.sh
$ MACHINE=imx8mm-var-dart bitbake b2qt-embedded-qt6-image

The output images can be accessed from your host computer:

$ ls -l ~/var-b2qt/build-imx8mm-var-dart/tmp/deploy/images/imx8mm-var-dart/b2qt-embedded-qt6-image-imx8mm-var-dart.img