DART-MX8M GSTREAMER

From Variscite Wiki
Revision as of 11:52, 12 November 2018 by Admin (talk | contribs) (Created page with "{{PageHeader|VAR-SOM-MX6 - Streaming Video}} {{DocImage|category1=DART-MX8M|category2=Yocto}} __toc__ = Introduction to GStreamer = [https://en.wikipedia.org/wiki/GStreamer G...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
VAR-SOM-MX6 - Streaming Video

Introduction to GStreamer

GStreamer is a powerful pipeline-based multimedia framework.
It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.
This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.
This release uses the gstreamer-imx set of GStreamer 1.0 plugins for Freescale's i.MX platform, which make use of the i.MX multimedia capabilities.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements: 

$ gst-inspect-1.0

Lists all available sink elements: 

$ gst-inspect-1.0 | grep sink

For example: 

root@imx8m-var-dart:~# gst-inspect-1.0 | grep sink
alsa:  alsasink: Audio sink (ALSA)
waylandsink:  waylandsink: wayland video sink
shm:  shmsink: Shared Memory Sink
decklink:  decklinkaudiosink: Decklink Audio Sink
decklink:  decklinkvideosink: Decklink Video Sink
ximagesink:  ximagesink: Video sink
soup:  souphttpclientsink: HTTP client sink
autodetect:  autovideosink: Auto video sink
autodetect:  autoaudiosink: Auto audio sink
gdkpixbuf:  gdkpixbufsink: GdkPixbuf sink
playback:  playsink: Player Sink
video4linux2:  v4l2sink: Video (video4linux2) Sink
fbdevsink:  fbdevsink: fbdev video sink
tcp:  tcpclientsink: TCP client sink
tcp:  tcpserversink: TCP server sink
...

Detailed help information on a specific element: 

$ gst-inspect-1.0 kmssink

For example: 

root@imx8m-var-dart:~# gst-inspect-1.0 kmssink
Factory Details:
  Rank                     secondary (128)
  Long-name                KMS video sink
  Klass                    Sink/Video
  Description              Video sink using the Linux kernel mode setting API
  Author                   Víctor Jáquez <vjaquez@igalia.com>

Plugin Details:
  Name                     kms
  Description              Video sink using the Linux kernel mode setting API
  Filename                 /usr/lib/gstreamer-1.0/libgstkms.so
  Version                  1.12.2
  License                  LGPL
  Source module            gst-plugins-bad
  Source release date      2017-07-14
  Binary package           GStreamer Bad Plug-ins source release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstKMSSink

Implemented Interfaces:
  GstVideoOverlay

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)BGRA, (string)BGRx, (string)RGBA, (string)RGBx, (string)UYVY, (string)YUY2, (string)YVYU, (string)I420, (string)YV12, (string)Y42B, (string)NV12, (string)NV12_10LE, (stri
ng)NV21, (string)NV16 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_kms_sink_change_state

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "kmssink0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000 
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  driver-name         : DRM device driver name
                        flags: readable, writable
                        String. Default: null
  connector-id        : DRM connector id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  plane-id            : DRM plane id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  force-modesetting   : When enabled, the sink try to configure the display mode
                        flags: readable, writable
                        Boolean. Default: false
  global-alpha        : global alpha
                        flags: readable, writable
                        Integer. Range: 0 - 255 Default: 0 
  force-hantrotile    : When enabled, the sink propose hantro tile modifier to VPU
                        flags: readable, writable
                        Boolean. Default: false

gst-launch

For Example launch the camera and display on the monitor: 

$ gst-launch-1.0 -v videotestsrc ! imxv4l2sink

i.MX6 major elements

Prefix
 Element Name
 description
imxaudio imxmp3audioenc imx mp3 audio encoder
imxg2d imxg2dvideosink IMX G2D video sink (blitter-based, zerocopy)
imxipu imxipuvideosink IMX IPU video sink (blitter-based, zerocopy)
imxpxp imxpxpvideosink IMX PxP video sink (blitter-based, zerocopy)
imxeglvivsink imxeglvivsink IMX EGL video sink (zerocopy)
imxv4l2videosrc imxv4l2videosrc IMX V4L2 CSI Video Source (zerocopy)
imxvpu imxvpudec IMX VPU video decoder (zerocopy)
imxvpu imxvpuenc_h263 VPU-based h.263 video encoder (zerocopy)
imxvpu imxvpuenc_h264 VPU-based H264 video encoder (zerocopy)
imxvpu imxvpuenc_h264 VPU-based MPEG video encoder (zerocopy)
imxvpu imxvpuenc_jpeg VPU-based JPEG video encoder (zerocopy)
imxipu imxipuvideotransform IMX IPU video transform
imxpxp imxpxpvideotransform IMX PXP video transform
imxg2d imxg2dvideotransform IMX G2D video transform
imxipu imxipucompositor IMX IPU video compositor
imxg2d imxg2dcompositor IMX G2D video compositor
Only imxeglvivsink will work with X11 or Wayland. More optimized blitter-base PXP,IPU and G2D sinks will work with fslc_framebuffer only
See more detailed description here

Test Pattern Stream

Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern: 

$ export DISPLAY=:0

$ gst-launch-1.0 videotestsrc ! borderless-window="true" force-aspect-ratio="false"

Gst1.jpg

Test pattern with specific paramters on the input element 

$ gst-launch-1.0 videotestsrc pattern=circular ! imxeglvivsink borderless-window="true" force-aspect-ratio="false"

Gst2.jpg

Test pattern with specific paramters on the output element 

$ gst-launch-1.0 videotestsrc ! imxeglvivsink window-width="1280" window-height="720" borderless-window="true" force-aspect-ratio="false"

Gst4.jpg

Test pattern with specific paramters on the input and output elements 

$ gst-launch-1.0 videotestsrc pattern=circular ! imxeglvivsink window-width="1280" window-height="720" borderless-window="true" force-aspect-ratio="false"

Gst3.jpg

Camera Loopback

Stream video from a camera to a display
Command:

$ export DISPLAY=:0

$ gst-launch-1.0 imxv4l2videosrc device=/dev/video0 imx-capture-mode=5 fps-n=30 ! imxeglvivsink  window-width="1920" window-height="1080" borderless-window="true" force-aspect-ratio="false"
Only imxeglvivsink will work with X11 or Wayland with some rendering artifacts.
More optimized blitter-base imxg2dvideosink and imxipuvideosink sinks will properly work with fslc_framebuffer DISTRO only


Get the camera support format and resolution using gst-inspect-1.0 imxv4l2videosrc.

imx-capture-mode for ov5640 example resolutions: 

ov5640_mode_VGA_640_480 = 0,
ov5640_mode_QVGA_320_240 = 1,
ov5640_mode_NTSC_720_480 = 2,
ov5640_mode_PAL_720_576 = 3,
ov5640_mode_720P_1280_720 = 4,
ov5640_mode_1080P_1920_1080 = 5

Video Record

command:
gst-launch-1.0 imxv4l2videosrc device=$DEVICE num-buffers=300 imx-capture-mode=$MODE ! queue ! imxvpuenc_XXX ! $MUXER ! filesink location=output.$EXTENSION

Parameter comments:

• $DEVICE could be set to /dev/video, /dev/video0 or according to the system video input device.
• num-buffers - Number of buffers to output before sending EOS. deafult =-1 (unlimited)
• $MODE should be set according to camera format and resolution
• imxvpuenc_XXX can be imxvpuenc_mpeg4,imxvpuenc_h263, imxvpuenc_h264, and imxvpuenc_jpeg
• MUXER can be set as to qtmux, matroskamux, mp4mux, avimux, or flvmux
• EXTENSION is filename extension according to the muxer type.


Record video from a camera into a file. Encode it to h264 at a bitrate of 10mbit/s (CBR)
720p: 

$ gst-launch-1.0 imxv4l2videosrc device=/dev/video0 imx-capture-mode=4 fps-n=30 num-buffers=300 ! queue ! imxvpuenc_h264 bittrate=10000 ! avimux ! filesink location=test.avi

1080p: 

$ gst-launch-1.0 imxv4l2videosrc device=/dev/video0 imx-capture-mode=5 fps-n=30 num-buffers=300 ! queue ! imxvpuenc_h264 bitrate=10000 ! avimux ! filesink location=test.avi

Simple Movie Play

$ export DISPLAY=:0
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4

Compositing

The compositor is a new feature in gstreamer-imx 0.11.0. Just like with the compositor from gst-plugins-base 1.5.1 and newer, compositor elements support an arbitrary number of request sink pads, and one srcpad. 

$ gst-launch-1.0   \
  imxg2dcompositor name=c background-color=0x223344 \
      sink_0::xpos=0 sink_0::ypos=90 sink_0::width=160 sink_0::height=110 sink_0::zorder=55 sink_0::fill_color=0xff00ff00 sink_0::alpha=0.39 sink_0::rotation=0 \
      sink_1::xpos=0 sink_1::ypos=20 sink_1::width=620 sink_1::height=380 sink_1::fill_color=0x44441133 ! \
    queue2 ! "video/x-raw, width=800, height=600" ! imxg2dvideosink \
  videotestsrc pattern=0 ! "video/x-raw, framerate=30/1" ! c.sink_0 \
  videotestsrc pattern=18 ! "video/x-raw, framerate=30/1" ! c.sink_1


See detailed description here

References

  • The GStreamer website
  • The gstreamer-imx README
  • The gstreamer-imx FAQ
  • A Deep Dive into Image Processing for i.MX 6 Application Processors presentation
Retrieved from "https://variwiki.com/index.php?title=DART-MX8M_GSTREAMER&oldid=10522"
Latest Modules
Support
Variscite Support Center
Contact Support
Product Longevity
Product Compliance Policy
Blog
Variscite Training
Variscite Website
This page was last edited on 12 November 2018, at 11:52.

© 2024 Variscite. All rights reserved