Revision as of 18:31, 13 November 2018

DART-MX8M - Streaming Video

Introduction to GStreamer

GStreamer is a powerful pipeline-based multimedia framework.
It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.
This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.
This release uses the gstreamer-imx set of GStreamer 1.0 plugins for Freescale's i.MX platform, which make use of the i.MX multimedia capabilities.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements:

$ gst-inspect-1.0

Lists all available sink elements:

$ gst-inspect-1.0 | grep sink

For example:

root@imx8m-var-dart:~# gst-inspect-1.0 | grep sink
alsa:  alsasink: Audio sink (ALSA)
waylandsink:  waylandsink: wayland video sink
shm:  shmsink: Shared Memory Sink
decklink:  decklinkaudiosink: Decklink Audio Sink
decklink:  decklinkvideosink: Decklink Video Sink
ximagesink:  ximagesink: Video sink
soup:  souphttpclientsink: HTTP client sink
autodetect:  autovideosink: Auto video sink
autodetect:  autoaudiosink: Auto audio sink
gdkpixbuf:  gdkpixbufsink: GdkPixbuf sink
playback:  playsink: Player Sink
video4linux2:  v4l2sink: Video (video4linux2) Sink
fbdevsink:  fbdevsink: fbdev video sink
tcp:  tcpclientsink: TCP client sink
tcp:  tcpserversink: TCP server sink
...

Detailed help information on a specific element:

$ gst-inspect-1.0 kmssink

For example:

root@imx8m-var-dart:~# gst-inspect-1.0 kmssink
Factory Details:
  Rank                     secondary (128)
  Long-name                KMS video sink
  Klass                    Sink/Video
  Description              Video sink using the Linux kernel mode setting API
  Author                   Víctor Jáquez <vjaquez@igalia.com>

Plugin Details:
  Name                     kms
  Description              Video sink using the Linux kernel mode setting API
  Filename                 /usr/lib/gstreamer-1.0/libgstkms.so
  Version                  1.12.2
  License                  LGPL
  Source module            gst-plugins-bad
  Source release date      2017-07-14
  Binary package           GStreamer Bad Plug-ins source release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstKMSSink

Implemented Interfaces:
  GstVideoOverlay

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)BGRA, (string)BGRx, (string)RGBA, (string)RGBx, (string)UYVY, (string)YUY2, (string)YVYU,
                           (string)I420, (string)YV12, (string)Y42B, (string)NV12, (string)NV12_10LE, (string)NV21, (string)NV16 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_kms_sink_change_state

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "kmssink0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000 
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  driver-name         : DRM device driver name
                        flags: readable, writable
                        String. Default: null
  connector-id        : DRM connector id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  plane-id            : DRM plane id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  force-modesetting   : When enabled, the sink try to configure the display mode
                        flags: readable, writable
                        Boolean. Default: false
  global-alpha        : global alpha
                        flags: readable, writable
                        Integer. Range: 0 - 255 Default: 0 
  force-hantrotile    : When enabled, the sink propose hantro tile modifier to VPU
                        flags: readable, writable
                        Boolean. Default: false

Test Pattern Stream

Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:

$ gst-launch-1.0 videotestsrc ! kmssink

Test pattern with specific parameters on the input element

$ gst-launch-1.0 videotestsrc pattern=circular ! kmssink

Camera Loopback

Stream video from a camera to a display

$ gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink

Video Recording

Save video stream from camera as an MJPEG-encoded AVI file

$ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! jpegenc ! avimux ! filesink location=./test0.avi

Video Playback

Play MP4 video file

$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4

RTP Video streaming

Encode video stream from camera using software jpeg encoder and stream it over RTP

 
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1280,height=720 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.20.60 port=1223

References

The GStreamer website

DART-MX8M GSTREAMER: Difference between revisions