VAR-SOM-MX6 GSTREAM FSLC

VAR-SOM-MX6 - Streaming Video

Introduction to GStreamer

GStreamer
GStreamer is a powerful pipeline-based multimedia framework.
GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products.
This release is based on GStreamer-I.MX community port that allows utilizing internal I.MX VPU and GPU capabilities.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements:

$ gst-inspect-1.0 

Lists all available i.MX6 specific elements:

$ gst-inspect-1.0 | grep imx

For example:

root@var-som-mx6:~# gst-inspect-1.0 | grep imx
imxipu:  imxipuvideotransform: Freescale IPU video transform
imxipu:  imxipuvideosink: Freescale IPU video sink
imxipu:  imxipucompositor: Freescale IPU video compositor
imxg2d:  imxg2dvideosink: Freescale G2D video sink
imxg2d:  imxg2dvideotransform: Freescale G2D video transform
imxg2d:  imxg2dcompositor: Freescale G2D video compositor
imxpxp:  imxpxpvideosink: Freescale PxP video sink
imxpxp:  imxpxpvideotransform: Freescale PxP video transform
imxaudio:  imxuniaudiodec: Freescale i.MX uniaudio decoder
imxaudio:  imxmp3audioenc: Freescale i.MX MP3 encoder
imxeglvivsink:  imxeglvivsink: Freescale EGL video sink
imxvpu:  imxvpudec: Freescale VPU video decoder
imxvpu:  imxvpuenc_h263: Freescale VPU h.263 video encoder
imxvpu:  imxvpuenc_h264: Freescale VPU h.264 video encoder
imxvpu:  imxvpuenc_mpeg4: Freescale VPU MPEG-4 video encoder
imxvpu:  imxvpuenc_mjpeg: Freescale VPU motion JPEG video encoder
imxv4l2videosrc:  imxv4l2videosrc: V4L2 CSI Video Source

Detailed help information on a specific element:

$ gst-inspect-1.0 imxeglvivsink

For example:

root@var-som-mx6:~# gst-inspect-1.0 imxeglvivsink
====== IMXV4L2SINK: 4.0.9 build on Aug  8 2016 12:47:08. ======
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                Freescale EGL video sink
  Klass                    Sink/Video
  Description              Video output using the i.MX6 Vivante GPU
  Author                   Carlos Rafael Giani <dv@pseudoterminal.org>

Plugin Details:
  Name                     imxeglvivsink
  Description              EGL/GLES sink using Vivante direct textures
  Filename                 /usr/lib/gstreamer-1.0/libgstimxeglvivsink.so
  Version                  0.12.2
  License                  LGPL
  Source module            gstreamer-imx
  Binary package           Unknown package release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstImxEglVivSink

Implemented Interfaces:
  GstNavigation
  GstVideoOverlay

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { I420, YV12, YV21, NV12, NV21, UYVY, RGB16, RGBA, BGRA, RGBx, BGRx, BGR, ARGB, ABGR, xRGB, xBGR }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_imx_egl_viv_sink_change_state

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "imxeglvivsink0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  fullscreen          : Whether or not to set the created window to fullscreen mode (ignored if application provides a window handle)
                        flags: readable, writable
                        Boolean. Default: false
  force-aspect-ratio  : When enabled, scaling will respect original aspect ratio
                        flags: readable, writable
                        Boolean. Default: true
  native-display      : String identifying the display to use (default value uses the default display)
                        flags: readable, writable
                        String. Default: null
  window-x-coord      : X coordinate of the window's top left corner, in pixels
                        flags: readable, writable
                        Integer. Range: -2147483648 - 2147483647 Default: 0
  window-y-coord      : Y coordinate of the window's top left corner, in pixels
                        flags: readable, writable
                        Integer. Range: -2147483648 - 2147483647 Default: 0
  window-width        : Window width, in pixels (0 = automatically set to the video input width)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  window-height       : Window height, in pixels (0 = automatically set to the video input height)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 2147483647 Default: 0
  borderless-window   : Disable window borders, bypassing any window manager
                        flags: readable, writable
                        Boolean. Default: false
                   

gst-launch

For Example launch the camera and display on the monitor:

$ gst-launch-1.0 -v videotestsrc ! imxv4l2sink

i.MX6 major elements

Prefix	Element Name	description
imxaudio	imxmp3audioenc	imx mp3 audio encoder
imxg2d	imxg2dvideosink	IMX G2D video sink (blitter-based, zerocopy)
imxipu	imxipuvideosink	IMX IPU video sink (blitter-based, zerocopy)
imxpxp	imxpxpvideosink	IMX PxP video sink (blitter-based, zerocopy)
imxeglvivsink	imxeglvivsink	IMX EGL video sink (zerocopy)
imxv4l2videosrc	imxv4l2videosrc	IMX V4L2 CSI Video Source (zerocopy)
imxvpu	imxvpudec	IMX VPU video decoder (zerocopy)
imxvpu	imxvpuenc_h263	VPU-based h.263 video encoder (zerocopy)
imxvpu	imxvpuenc_h264	VPU-based H264 video encoder (zerocopy)
imxvpu	imxvpuenc_h264	VPU-based MPEG video encoder (zerocopy)
imxvpu	imxvpuenc_jpeg	VPU-based JPEG video encoder (zerocopy)
imxipu	imxipuvideotransform	IMX IPU video transform
imxpxp	imxpxpvideotransform	IMX PXP video transform
imxg2d	imxg2dvideotransform	IMX G2D video transform
imxipu	imxipucompositor	IMX IPU video compositor
imxg2d	imxg2dcompositor	IMX G2D video compositor

Test Pattern Stream

Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:

$ export DISPLAY=:0

$ gst-launch-1.0 videotestsrc ! borderless-window="true" force-aspect-ratio="false"

Test pattern with specific paramters on the input element

$ gst-launch-1.0 videotestsrc pattern=circular ! imxeglvivsink borderless-window="true" force-aspect-ratio="false"

Test pattern with specific paramters on the output element

$ gst-launch-1.0 videotestsrc ! imxeglvivsink window-width="1280" window-height="720" borderless-window="true" force-aspect-ratio="false"

Test pattern with specific paramters on the input and output elements

$ gst-launch-1.0 videotestsrc pattern=circular ! imxeglvivsink window-width="1280" window-height="720" borderless-window="true" force-aspect-ratio="false"

Camera Loopback

Stream video from a camera to a display
Command:
gst-launch-1.0 imxv4l2src ! 'video/x-raw, format=(string)$FORMAT, width=$WIDTH, height=$HEIGHT, framerate=(fraction)30/1' ! imxv4l2sink
For example:

$ gst-launch-1.0 imxv4l2src device=/dev/video0 ! 'video/x-raw,format=(string)I420,width=1920,height=1080,framerate=(fraction)30/1'! imxv4l2sink overlay-width=1920 overlay-height=1080

This is a 3 elements pipe,the element in the middle is a GStreamer capsfilter element.
Parameter comments:

• Get the camera support format and resolution using gst-inspect-1.0 imxv4l2src.

• Set capsfilter according to the camera's supported capabilities if the user needs other format or resolution.

• Ensure that the right caps filter has been set, which also needs to be supported by imxv4l2sink.

ov5640 example resolutions:

width=320,height=240

width=640,height=480

width=720,height=480

width=720,height=576

width=1280,height=720

width=1920,height=1080

Video Record

command:
gst-launch-1.0 imxv4l2src device=$DEVICE num-buffers=300 ! $INPUT_CAPS ! queue ! vpuenc_XXX ! $MUXER ! filesink location=output.$EXTENSION

Parameter comments:

• $DEVICE could be set to /dev/video, /dev/video0 or according to the system video input device.

• num-buffers - Number of buffers to output before sending EOS. deafult =-1 (unlimited)

• INPUT_CAPS should be set according to camera format and resolution

• vpuenc_XXX can be vpuenc_mpeg4,vpuenc_h263, vpuenc_h264, and vpuenc_jpeg

• MUXER can be set as to qtmux, matroskamux, mp4mux, avimux, or flvmux

• EXTENSION is filename extension according to the muxer type.

Record video from a camera into a file.
720p:

$ gst-launch-1.0 imxv4l2src device=/dev/video0 num-buffers=300 ! 'video/x-raw,format=(string)I420,width=1280,height=720,framerate=(fraction)30/1' ! queue ! vpuenc_h264 ! avimux ! filesink location=test.avi

1080p:

$ gst-launch-1.0 imxv4l2src device=/dev/video0 num-buffers=300 ! 'video/x-raw,format=(string)I420,width=1920,height=1080,framerate=(fraction)30/1' ! queue ! vpuenc_h264 ! avimux ! filesink location=test.avi

Simple Movie Play

$ export DISPLAY=:0
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4