DART-MX8M GSTREAMER: Difference between revisions

From Variscite Wiki
No edit summary
(23 intermediate revisions by 6 users not shown)
Line 1: Line 1:
<!-- Set release according to "release" parameter in URL and use RELEASE_SUMO_V1.0_DART-MX8M as default
<!-- Set release according to "release" parameter in URL and use RELEASE_SUMO_V1.4_DART-MX8M-MINI as default
--> {{#vardefine:RELEASE_PARAM|{{#urlget:release}}}} <!--
--> {{#vardefine:RELEASE_PARAM|{{#urlget:release|RELEASE_SUMO_V1.4_DART-MX8M-MINI}}}} <!--
--> {{#lst:Yocto_Platform_Customization|{{#var:RELEASE_PARAM|RELEASE_SUMO_V1.0_DART-MX8M}}}} <!--
--> {{#lst:Yocto_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!--
--> {{PageHeader|{{#var:HARDWARE_NAME}} Streaming Video}} {{DocImage|category1=Yocto|category2=DART-MX8M}}[[Category:DART-MX8M-MINI]] __toc__
--> {{#lst:B2QT_Platform_Customization|{{#var:RELEASE_PARAM}}}} <!--
--> {{PageHeader|{{#var:HARDWARE_NAME}} Streaming Video}} {{DocImage|category1=Yocto|category2=Debian}}[[Category:{{#var:HARDWARE_NAME}}]] __toc__


= Introduction to GStreamer =
= Introduction to GStreamer =
Line 279: Line 280:
                         Boolean. Default: true
                         Boolean. Default: true
</pre>
</pre>
|
| {{#ifeq: {{#var:HARDWARE_NAME}}|DART-MX8M-MINI |
=== The i.MX8M-MINI VPU video decoder element ===
=== The i.MX8M-MINI VPU video decoder element ===
<pre>
<pre>
Line 362: Line 363:
                         Boolean. Default: true
                         Boolean. Default: true
</pre>
</pre>
}}
| {{#ifeq: {{#var:HARDWARE_NAME}}|DART-MX8M-PLUS |
=== The i.MX8M-PLUS VPU video decoder element ===
<pre>
Factory Details:
  Rank                    primary + 1 (257)
  Long-name                IMX VPU-based video decoder
  Klass                    Codec/Decoder/Video
  Description              Decode compressed video to raw data
  Author                  Multimedia Team <shmmmw@freescale.com>
 
Plugin Details:
  Name                    vpu
  Description              VPU video codec
  Filename                /usr/lib/gstreamer-1.0/libgstvpu.so
  Version                  4.5.6
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package          Freescle Gstreamer Multimedia Plugins
  Origin URL              http://www.freescale.com
 
GObject
+----GInitiallyUnowned
      +----GstObject
            +----GstElement
                  +----GstVideoDecoder
                        +----GstVpuDec
 
Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
                  width: [ 1, 2147483647 ]
                height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
 
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h265
      video/x-vp9
      video/x-vp8
      video/x-h264
 
Element has no clocking capabilities.
Element has no URI handling capabilities.
 
Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'
 
Element Properties:
  disable-reorder    : disable vpu reorder when end to end streaming
                        flags: readable, writable
                        Boolean. Default: false
  frame-drop          : enable adaptive frame drop for smoothly playback
                        flags: readable, writable
                        Boolean. Default: true
  frame-plus          : set number of addtional frames for smoothly playback
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 16 Default: 3
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "vpudec0"
  output-format      : set raw video format for output (Y42B NV16 Y444 NV24 only for MJPEG)
                        flags: readable, writable
                        Enum "GstVpuDecOutputFormat" Default: 0, "auto"
                          (0): auto            - enable chroma interleave. (default)
                          (1): NV12            - NV12 format
                          (2): I420            - I420 format
                          (3): YV12            - YV12 format
                          (4): Y42B            - Y42B format
                          (5): NV16            - NV16 format
                          (6): Y444            - Y444 format
                          (7): NV24            - NV24 format
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  use-vpu-memory      : use vpu allocate video frame buffer
                        flags: readable, writable
                        Boolean. Default: true
</pre>
|}}}}}}


{{#ifeq: {{#var:HARDWARE_NAME}}|DART-MX8M-MINI|
{{#ifeq: {{#var:HARDWARE_NAME}}|DART-MX8M-MINI|
Line 442: Line 528:
                         Integer. Range: -1 - 51 Default: -1
                         Integer. Range: -1 - 51 Default: -1
</pre>
</pre>
}}
|{{#ifeq: {{#var:HARDWARE_NAME}}|DART-MX8M-PLUS|
=== The i.MX8M-PLUS VPU H.264 video encoder element ===
<pre>
Factory Details:
  Rank                    primary + 1 (257)
  Long-name                IMX VPU-based AVC/H264 video encoder
  Klass                    Codec/Encoder/Video
  Description              Encode raw data to compressed video
  Author                  Multimedia Team <shmmmw@freescale.com>
 
Plugin Details:
  Name                    vpu
  Description              VPU video codec
  Filename                /usr/lib/gstreamer-1.0/libgstvpu.so
  Version                  4.5.6
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package          Freescle Gstreamer Multimedia Plugins
  Origin URL              http://www.freescale.com
 
GObject
+----GInitiallyUnowned
      +----GstObject
            +----GstElement
                  +----GstVideoEncoder
                        +----vpuenc_h264
 
Implemented Interfaces:
  GstPreset
 
Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-h264
          stream-format: { (string)avc, (string)byte-stream }
              alignment: { (string)au, (string)nal }
 
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                format: { (string)NV12, (string)I420, (string)YUY2, (string)UYVY, (string)RGBA, (string)RGBx, (string)RGB16, (string)RGB15, (string)BGRA, (string)BGRx, (string)BGR16 }
                  width: [ 64, 1920 ]
                height: [ 64, 1088 ]
              framerate: [ 0/1, 2147483647/1 ]
 
Element has no clocking capabilities.
Element has no URI handling capabilities.
 
Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'
 
Element Properties:
  bitrate            : set bit rate in kbps (0 for automatic)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  gop-size            : How many frames a group-of-picture shall contain
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 32767 Default: 30
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "vpuenc_h264-0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  qos                : Handle Quality-of-Service events from downstream
                        flags: readable, writable
                        Boolean. Default: false
  quant              : set quant value: H.264(0-51) (-1 for automatic)
                        flags: readable, writable
                        Integer. Range: -1 - 51 Default: -1
  stream-multislice  : the number of slices a picture contains
                        flags: readable, writable
                        Integer. Range: 1 - 2147483647 Default: 1
</pre>
 
=== The i.MX8M-PLUS VPU H.265 video encoder element ===
<pre>
Factory Details:
  Rank                    primary + 1 (257)
  Long-name                IMX VPU-based HEVC video encoder
  Klass                    Codec/Encoder/Video
  Description              Encode raw data to compressed video
  Author                  Multimedia Team <shmmmw@freescale.com>
 
Plugin Details:
  Name                    vpu
  Description              VPU video codec
  Filename                /usr/lib/gstreamer-1.0/libgstvpu.so
  Version                  4.5.6
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package          Freescle Gstreamer Multimedia Plugins
  Origin URL              http://www.freescale.com
 
GObject
+----GInitiallyUnowned
      +----GstObject
            +----GstElement
                  +----GstVideoEncoder
                        +----vpuenc_hevc
 
Implemented Interfaces:
  GstPreset
 
Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-h265
                variant: itu
                  width: [ 64, 1920 ]
                height: [ 64, 1088 ]
              framerate: [ 0/1, 2147483647/1 ]
 
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                format: { (string)NV12, (string)I420, (string)YUY2, (string)UYVY, (string)RGBA, (string)RGBx, (string)RGB16, (string)RGB15, (string)BGRA, (string)BGRx, (string)BGR16 }
                  width: [ 64, 1920 ]
                height: [ 64, 1088 ]
              framerate: [ 0/1, 2147483647/1 ]
 
Element has no clocking capabilities.
Element has no URI handling capabilities.
 
Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'
 
Element Properties:
  bitrate            : set bit rate in kbps (0 for automatic)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  gop-size            : How many frames a group-of-picture shall contain
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 32767 Default: 30
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "vpuenc_hevc0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  qos                : Handle Quality-of-Service events from downstream
                        flags: readable, writable
                        Boolean. Default: false
  quant              : set quant value: HEVC(0-51) (-1 for automatic)
                        flags: readable, writable
                        Integer. Range: -1 - 51 Default: -1
  stream-multislice  : the number of slices a picture contains
                        flags: readable, writable
                        Integer. Range: 1 - 2147483647 Default: 1
</pre>
|}}}}


= Test Pattern Stream =
= Test Pattern Stream =
{{#vardefine:DEFAULT_SINK | {{#varexists:USE_KMSSINK | kmssink can-scale=false | autovideosink }} }}
Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.<br>
Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.<br>
Default test pattern:
Default test pattern:
  # gst-launch-1.0 videotestsrc ! autovideosink
 
  # gst-launch-1.0 videotestsrc ! {{#var:DEFAULT_SINK}}
[[File:gst1.jpg]]<br>
[[File:gst1.jpg]]<br>
<br>
<br>
Test pattern with specific parameters on the input element:
Test pattern with specific parameters on the input element:
  # gst-launch-1.0 videotestsrc pattern=circular ! autovideosink
  # gst-launch-1.0 videotestsrc pattern=circular ! {{#var:DEFAULT_SINK}}
[[File:gst2.jpg]]
[[File:gst2.jpg]]


Line 460: Line 708:
= Video Recording =  
= Video Recording =  
{{#ifeq: {{#var:HARDWARE_NAME}} | DART-MX8M-MINI |
{{#ifeq: {{#var:HARDWARE_NAME}} | DART-MX8M-MINI |
The i.MX8M-MINI can encode videos to H.264 & VP8 at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).<br>
The i.MX8M-MINI can encode videos to AVC(H.264) & VP8 at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).<br>
See the sink pad capabilities in the [[#The i.MX8M-MINI VPU H.264 video encoder element|vpuenc element details]] for the complete list of features supported by the i.MX8M-MINI VPU encoder.<br>
See the src pad capabilities in the [[#The i.MX8M-MINI VPU H.264 video encoder element|vpuenc_264]] and vpuenc_vp8 elements details for the complete list of features supported by the i.MX8M-MINI VPU encoder.<br>
<br>
<br>
Example of encoding and saving a short video stream from a camera to an H.264-encoded AVI file:
Example of encoding and saving a short video stream from a camera to an H.264-encoded AVI file:
  # gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! vpuenc_h264 ! avimux ! filesink location=./test_h264.avi
  # gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! vpuenc_h264 ! avimux ! filesink location=./test_h264.avi
}}
{{#ifeq: {{#var:HARDWARE_NAME}} | DART-MX8M-PLUS |
The i.MX8M-PLUS can encode videos to AVC(H.264) & HEVC(H.265) at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).<br>
See the src pad capabilities in the [[#The i.MX8M-PLUS VPU H.264 video encoder element|vpuenc_264]] and [[#The i.MX8M-PLUS VPU H.265 video encoder element|vpuenc_265]] elements details for the complete list of features supported by the i.MX8M-PLUS VPU encoder.<br>
<br>
Example of encoding and saving a short video stream from a camera to an H.264-encoded AVI file:
# gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! vpuenc_h264 ! avimux ! filesink location=./test_h264.avi
Example of encoding and saving a short video stream from a camera to an H.265-encoded TS file:
# gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! vpuenc_hevc ! mpegtsmux ! filesink location=./test_h265.ts
}}
}}


Line 470: Line 729:
  # gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! jpegenc ! avimux ! filesink location=./test_jpeg.avi
  # gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! jpegenc ! avimux ! filesink location=./test_jpeg.avi


{{#ifeq: {{#var:HARDWARE_NAME}} | VAR-SOM-MX8M-NANO | |
= Video Playback =
= Video Playback =
{{#ifeq: {{#var:HARDWARE_NAME}} | DART-MX8M-MINI |
{{#ifeq: {{#var:HARDWARE_NAME}} | DART-MX8M-MINI |
The i.MX8M-MINI can decode and playback HEVC(H.265), AVC(H.264), VP9, VP8, etc. encoded videos at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).<br>
The i.MX8M-MINI can decode and playback HEVC(H.265), AVC(H.264), VP9, VP8, etc. encoded videos at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).<br>
See the sink pad capabilities in the [[#The i.MX8M-MINI VPU video decoder element|vpudec element details]] for the complete list of codecs supported by the i.MX8M-MINI VPU decoder.<br>
See the sink pad capabilities in the [[#The i.MX8M-MINI VPU video decoder element|vpudec element details]] for the complete list of codecs supported by the i.MX8M-MINI VPU decoder.<br>
| {{#ifeq: {{#var:HARDWARE_NAME}} | DART-MX8M-PLUS |
The i.MX8M-PLUS can decode and playback AVC(H.264), HEVC(H.265), VP9, VP8, etc. encoded videos with at resolutions of up to 1080p60 (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).<br>
See the sink pad capabilities in the [[#The i.MX8M-PLUS VPU video decoder element|vpudec element details]] for the complete list of codecs supported by the i.MX8M-PLUS VPU decoder.<br>
|
|
The i.MX8M can decode and playback HEVC(H.265), VP9, etc. encoded videos with HDR at resolutions of up to 4Kp60 UHD (3840×2160p @ 60fps) using its internal hardware Video Processing Unit (VPU).<br>
The i.MX8M can decode and playback HEVC(H.265), VP9, etc. encoded videos with HDR at resolutions of up to 4Kp60 UHD (3840×2160p @ 60fps) using its internal hardware Video Processing Unit (VPU).<br>
See the sink pad capabilities in the [[#The i.MX8M VPU video decoder element|vpudec element details]] for the complete list of codecs supported by the i.MX8M VPU decoder.<br>
See the sink pad capabilities in the [[#The i.MX8M VPU video decoder element|vpudec element details]] for the complete list of codecs supported by the i.MX8M VPU decoder.<br>
}}
}} }} }}
 
{{#ifeq: {{#var:HARDWARE_NAME}} | VAR-SOM-MX8M-NANO | |
{{#vardefine:PLAYBACK_VIDEO_CMD1 | {{#varexists:USE_KMSSINK | gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4 video-sink="imxvideoconvert_g2d ! queue ! kmssink can-scale=false" audio-sink="alsasink" | gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4 }} }}
{{#vardefine:PLAYBACK_VIDEO_CMD2 | {{#varexists:USE_KMSSINK | gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! imxvideoconvert_g2d ! video/x-raw,width=800,height=480 ! kmssink can-scale=false | gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! autovideosink }} }}
<br>
<br>
Example of a video file playback using the playbin element (making use of the {{#ifeq: {{#var:HARDWARE_NAME}}|DART-MX8M-MINI|i.MX8M-MINI|.MX8M}} VPU decoder):
Example of a video file playback using the playbin element (making use of the VPU decoder):
# gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4
# {{#var:PLAYBACK_VIDEO_CMD1}}  


Example of a video file playback while explicitly specifying a pipeline (making use of the {{#ifeq: {{#var:HARDWARE_NAME}}|DART-MX8M-MINI|i.MX8M-MINI|.MX8M}} VPU decoder):
Example of a video file playback while explicitly specifying a pipeline (making use of the VPU decoder):
# gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! autovideosink
# {{#var:PLAYBACK_VIDEO_CMD2}}
}}


= RTP Video streaming =
= RTP Video streaming =
Line 491: Line 759:
= References =
= References =
* The GStreamer [https://gstreamer.freedesktop.org/ website]
* The GStreamer [https://gstreamer.freedesktop.org/ website]
* NXP BSP Linux Users Guide, Multimedia section [https://www.nxp.com/webapp/Download?colCode=L4.9.51_8MQ_GA_LINUX_DOCS&Parent_nodeId=1513109797983730444282&Parent_pageType=product&Parent_nodeId=1513109797983730444282&Parent_pageType=product download link ]
* NXP BSP Linux Users Guide, Multimedia section [https://www.nxp.com/webapp/Download?colCode=L4.14.78_1.0.0_LINUX_DOCS download link ]

Revision as of 09:10, 26 April 2021

DART-MX8M-MINI Streaming Video

Introduction to GStreamer

GStreamer is a powerful pipeline-based multimedia framework.
It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.
This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.
This release uses the gstreamer-imx set of GStreamer 1.0 plugins for NXP's i.MX platform, which make use of the i.MX multimedia capabilities.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements:

# gst-inspect-1.0

Lists all available sink elements:

# gst-inspect-1.0 | grep sink

For example:

# gst-inspect-1.0 | grep sink
alsa:  alsasink: Audio sink (ALSA)
waylandsink:  waylandsink: wayland video sink
shm:  shmsink: Shared Memory Sink
decklink:  decklinkaudiosink: Decklink Audio Sink
decklink:  decklinkvideosink: Decklink Video Sink
ximagesink:  ximagesink: Video sink
soup:  souphttpclientsink: HTTP client sink
autodetect:  autovideosink: Auto video sink
autodetect:  autoaudiosink: Auto audio sink
gdkpixbuf:  gdkpixbufsink: GdkPixbuf sink
playback:  playsink: Player Sink
video4linux2:  v4l2sink: Video (video4linux2) Sink
fbdevsink:  fbdevsink: fbdev video sink
tcp:  tcpclientsink: TCP client sink
tcp:  tcpserversink: TCP server sink
…

Detailed help information on a specific element:

# gst-inspect-1.0 <name of the element>

For example, the output for waylandsink:

# gst-inspect-1.0 waylandsink
Factory Details:
  Rank                     primary + 2 (258)
  Long-name                wayland video sink
  Klass                    Sink/Video
  Description              Output to wayland surface
  Author                   Sreerenj Balachandran <sreerenj.balachandran@intel.com>, George Kiagiadakis <george.kiagiadakis@collabora.com>

Plugin Details:
  Name                     waylandsink
  Description              Wayland Video Sink
  Filename                 /usr/lib/gstreamer-1.0/libgstwaylandsink.so
  Version                  1.14.0
  License                  LGPL
  Source module            gst-plugins-bad
  Source release date      2018-03-19
  Binary package           GStreamer Bad Plug-ins source release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstWaylandSink

Implemented Interfaces:
  GstVideoOverlay
  GstWaylandVideo

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)BGRx, (string)BGRA, (string)RGBx, (string)xBGR, (string)xRGB, (string)RGBA, (string)ABGR,
                           (string)ARGB, (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)YUY2, (string)YVYU,
                           (string)UYVY, (string)AYUV, (string)NV12, (string)NV21, (string)NV16, (string)YUV9, (string)YVU9,
                           (string)Y41B, (string)I420, (string)YV12, (string)Y42B, (string)v308 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw(memory:DMABuf)
                 format: { (string)BGRx, (string)BGRA, (string)RGBx, (string)xBGR, (string)xRGB, (string)RGBA, (string)ABGR,
                           (string)ARGB, (string)RGB, (string)BGR, (string)RGB16, (string)BGR16, (string)YUY2, (string)YVYU,
                           (string)UYVY, (string)AYUV, (string)NV12, (string)NV21, (string)NV16, (string)YUV9, (string)YVU9,
                           (string)Y41B, (string)I420, (string)YV12, (string)Y42B, (string)v308 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "waylandsink0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000 
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  window-width        : Wayland sink preferred window width in pixel
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  window-height       : Wayland sink preferred window height in pixel
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  display             : Wayland display name to connect to, if not supplied via the GstContext
                        flags: readable, writable
                        String. Default: null
  alpha               : Wayland surface alpha value, apply custom alpha value to wayland surface
                        flags: readable, writable
                        Float. Range:               0 -               1 Default:               0 
  enable-tile         : When enabled, the sink propose VSI tile modifier to VPU
                        flags: readable, writable
                        Boolean. Default: false

The i.MX8M-MINI VPU video decoder element

# gst-inspect-1.0 vpudec
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                IMX VPU-based video decoder
  Klass                    Codec/Decoder/Video
  Description              Decode compressed video to raw data
  Author                   Multimedia Team <shmmmw@freescale.com>

Plugin Details:
  Name                     vpu
  Description              VPU video codec
  Filename                 /usr/lib/gstreamer-1.0/libgstvpu.so
  Version                  4.4.4
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package           Freescle Gstreamer Multimedia Plugins
  Origin URL               http://www.freescale.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstVpuDec

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
  
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h265
      video/x-vp9
      video/x-vp8
      video/x-h264

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "vpudec0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  output-format       : set raw video format for output (Y42B NV16 Y444 NV24 only for MJPEG)
                        flags: readable, writable
                        Enum "GstVpuDecOutputFormat" Default: 0, "auto"
                           (0): auto             - enable chroma interleave. (default)
                           (1): NV12             - NV12 format
                           (2): I420             - I420 format
                           (3): YV12             - YV12 format
                           (4): Y42B             - Y42B format
                           (5): NV16             - NV16 format
                           (6): Y444             - Y444 format
                           (7): NV24             - NV24 format
  frame-drop          : enable adaptive frame drop for smoothly playback
                        flags: readable, writable
                        Boolean. Default: true
  frame-plus          : set number of addtional frames for smoothly playback
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 16 Default: 3 
  use-vpu-memory      : use vpu allocate video frame buffer
                        flags: readable, writable
                        Boolean. Default: true

The i.MX8M-MINI VPU H.264 video encoder element

# gst-inspect-1.0 vpuenc_h264
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                IMX VPU-based AVC/H264 video encoder
  Klass                    Codec/Encoder/Video
  Description              Encode raw data to compressed video
  Author                   Multimedia Team <shmmmw@freescale.com>

Plugin Details:
  Name                     vpu
  Description              VPU video codec
  Filename                 /usr/lib/gstreamer-1.0/libgstvpu.so
  Version                  4.4.4
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package           Freescle Gstreamer Multimedia Plugins
  Origin URL               http://www.freescale.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoEncoder
                         +----vpuenc_h264

Implemented Interfaces:
  GstPreset

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-h264
          stream-format: { (string)avc, (string)byte-stream }
              alignment: { (string)au, (string)nal }
  
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)NV12, (string)I420, (string)YUY2, (string)UYVY, (string)RGBA, (string)RGBx, (string)RGB16,
                           (string)RGB15, (string)BGRA, (string)BGRx, (string)BGR16 }
                  width: [ 64, 1920, 8 ]
                 height: [ 64, 1088, 8 ]
              framerate: [ 0/1, 2147483647/1 ]

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "vpuenc_h264-0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  qos                 : Handle Quality-of-Service events from downstream
                        flags: readable, writable
                        Boolean. Default: false
  bitrate             : set bit rate in kbps (0 for automatic)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0 
  gop-size            : How many frames a group-of-picture shall contain
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 32767 Default: 30 
  quant               : set quant value: H.264(0-51) (-1 for automatic)
                        flags: readable, writable
                        Integer. Range: -1 - 51 Default: -1

Test Pattern Stream

Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:

# gst-launch-1.0 videotestsrc ! autovideosink

Gst1.jpg

Test pattern with specific parameters on the input element:

# gst-launch-1.0 videotestsrc pattern=circular ! autovideosink

Gst2.jpg

Camera Loopback

Stream video from a camera to a display:

# gst-launch-1.0 v4l2src device=/dev/video0 ! autovideosink

Video Recording

The i.MX8M-MINI can encode videos to AVC(H.264) & VP8 at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).
See the src pad capabilities in the vpuenc_264 and vpuenc_vp8 elements details for the complete list of features supported by the i.MX8M-MINI VPU encoder.

Example of encoding and saving a short video stream from a camera to an H.264-encoded AVI file:

# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! vpuenc_h264 ! avimux ! filesink location=./test_h264.avi


Example of encoding and saving a short video stream from a camera to an MJPEG-encoded AVI file (using the SW based jpegenc element - without using the VPU):

# gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! jpegenc ! avimux ! filesink location=./test_jpeg.avi

Video Playback

The i.MX8M-MINI can decode and playback HEVC(H.265), AVC(H.264), VP9, VP8, etc. encoded videos at resolutions of up to 1080p60 Full HD (1920×1080p @ 60fps) using its internal hardware Video Processing Unit (VPU).
See the sink pad capabilities in the vpudec element details for the complete list of codecs supported by the i.MX8M-MINI VPU decoder.


Example of a video file playback using the playbin element (making use of the VPU decoder):

# gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4 

Example of a video file playback while explicitly specifying a pipeline (making use of the VPU decoder):

# gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true ! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! autovideosink

RTP Video streaming

Encode video stream from camera using a software based JPEG encoder and stream it over RTP:

# gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1280,height=720 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.20.60 port=1223

References