DART-MX8M GSTREAMER: Difference between revisions

From Variscite Wiki
No edit summary
Line 38: Line 38:
</pre>
</pre>
Detailed help information on a specific element:
Detailed help information on a specific element:
  $ gst-inspect-1.0 kmssink
  $ gst-inspect-1.0 vpudec
For example:
For example:
<pre>
<pre>
root@imx8m-var-dart:~# gst-inspect-1.0 kmssink
root@imx8m-var-dart:~# gst-inspect-1.0 vpudec
Factory Details:
Factory Details:
   Rank                    secondary (128)
   Rank                    primary + 1 (257)
   Long-name                KMS video sink
   Long-name                VPU-based video decoder
   Klass                    Sink/Video
   Klass                    Codec/Decoder/Video
   Description              Video sink using the Linux kernel mode setting API
   Description              Decode compressed video to raw data
   Author                  Víctor Jáquez <vjaquez@igalia.com>
   Author                  Multimedia Team <shmmmw@freescale.com>


Plugin Details:
Plugin Details:
   Name                    kms
   Name                    vpu.imx
   Description              Video sink using the Linux kernel mode setting API
   Description              VPU video codec
   Filename                /usr/lib/gstreamer-1.0/libgstkms.so
   Filename                /usr/lib/gstreamer-1.0/libgstvpu.so
   Version                  1.12.2
   Version                  4.3.4
   License                  LGPL
   License                  LGPL
   Source module            gst-plugins-bad
   Source module            imx-gst1.0-plugin
  Source release date      2017-07-14
   Binary package          Freescle Gstreamer Multimedia Plugins
   Binary package          GStreamer Bad Plug-ins source release
   Origin URL              http://www.freescale.com
   Origin URL              Unknown package origin


GObject
GObject
Line 64: Line 63:
       +----GstObject
       +----GstObject
             +----GstElement
             +----GstElement
                   +----GstBaseSink
                   +----GstVideoDecoder
                         +----GstVideoSink
                         +----GstVpuDec
                              +----GstKMSSink
 
Implemented Interfaces:
  GstVideoOverlay


Pad Templates:
Pad Templates:
   SINK template: 'sink'
   SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h265
      video/x-vp9
      video/x-vp8
      video/x-vp6-flash
      video/x-h264
      video/mpeg
          systemstream: false
            mpegversion: { (int)1, (int)2 }
      video/mpeg
            mpegversion: 4
      video/x-h263
      video/x-flash-video
            flvversion: 1
      video/x-divx
            divxversion: 3
      video/x-divx
            divxversion: 4
      video/x-divx
            divxversion: { (int)5, (int)6 }
      video/x-xvid
      video/x-cavs
      video/x-wmv
            wmvversion: 3
                format: WMV3
      video/x-wmv
            wmvversion: 3
                format: WVC1
      video/x-pn-realvideo
      image/jpeg
      image/webp
  SRC template: 'src'
     Availability: Always
     Availability: Always
     Capabilities:
     Capabilities:
       video/x-raw
       video/x-raw
                 format: { (string)BGRA, (string)BGRx, (string)RGBA, (string)RGBx, (string)UYVY, (string)YUY2, (string)YVYU,
                 format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
                          (string)I420, (string)YV12, (string)Y42B, (string)NV12, (string)NV12_10LE, (string)NV21, (string)NV16 }
                   width: [ 1, 2147483647 ]
                   width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
Line 87: Line 115:


Element Implementation:
Element Implementation:
   Has change_state() function: gst_kms_sink_change_state
   Has change_state() function: gst_video_decoder_change_state


Element has no clocking capabilities.
Element has no clocking capabilities.
Line 95: Line 123:
   SINK: 'sink'
   SINK: 'sink'
     Pad Template: 'sink'
     Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'


Element Properties:
Element Properties:
   name                : The name of the object
   name                : The name of the object
                         flags: readable, writable
                         flags: readable, writable
                         String. Default: "kmssink0"
                         String. Default: "vpudec0"
   parent              : The parent of the object
   parent              : The parent of the object
                         flags: readable, writable
                         flags: readable, writable
                         Object of type "GstObject"
                         Object of type "GstObject"
   sync                : Sync on the clock
   output-format      : set raw video format for output (Y42B NV16 Y444 NV24 only for MJPEG)
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000
  qos                : Generate Quality-of-Service events upstream
                         flags: readable, writable
                         flags: readable, writable
                         Boolean. Default: true
                         Enum "GstVpuDecOutputFormat" Default: 0, "auto"
   async              : Go asynchronously to PAUSED
                          (0): auto            - enable chroma interleave. (default)
                          (1): NV12            - NV12 format
                          (2): I420            - I420 format
                          (3): YV12            - YV12 format
                          (4): Y42B            - Y42B format
                          (5): NV16            - NV16 format
                          (6): Y444            - Y444 format
                          (7): NV24            - NV24 format
   frame-drop          : enable adaptive frame drop for smoothly playback
                         flags: readable, writable
                         flags: readable, writable
                         Boolean. Default: true
                         Boolean. Default: true
   ts-offset          : Timestamp offset in nanoseconds
   frame-plus          : set number of addtional frames for smoothly playback
                         flags: readable, writable
                         flags: readable, writable
                         Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
                         Unsigned Integer. Range: 0 - 16 Default: 3
   enable-last-sample  : Enable the last-sample property
   use-vpu-memory      : use vpu allocate video frame buffer
                         flags: readable, writable
                         flags: readable, writable
                         Boolean. Default: true
                         Boolean. Default: true
  last-sample        : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize          : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  throttle-time      : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  max-bitrate        : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  driver-name        : DRM device driver name
                        flags: readable, writable
                        String. Default: null
  connector-id        : DRM connector id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1
  plane-id            : DRM plane id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1
  force-modesetting  : When enabled, the sink try to configure the display mode
                        flags: readable, writable
                        Boolean. Default: false
  global-alpha        : global alpha
                        flags: readable, writable
                        Integer. Range: 0 - 255 Default: 0
  force-hantrotile    : When enabled, the sink propose hantro tile modifier to VPU
                        flags: readable, writable
                        Boolean. Default: false
</pre>
</pre>


Line 184: Line 180:
<pre>
<pre>
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4  
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4  
</pre>
Play MP4 video explicitly using i.MX8M VPU decoder
<pre>
gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true \
! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! kmssink
</pre>
</pre>


Line 194: Line 196:
= References =
= References =
* The GStreamer [https://gstreamer.freedesktop.org/ website]
* The GStreamer [https://gstreamer.freedesktop.org/ website]
* NXP BSP Linux Users Guide, Multimedia section [https://www.nxp.com/webapp/Download?colCode=L4.9.51_8MQ_GA_LINUX_DOCS&Parent_nodeId=1513109797983730444282&Parent_pageType=product&Parent_nodeId=1513109797983730444282&Parent_pageType=product download link ]

Revision as of 09:05, 14 November 2018

DART-MX8M - Streaming Video

Introduction to GStreamer

GStreamer is a powerful pipeline-based multimedia framework.
It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.
This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.
This release uses the gstreamer-imx set of GStreamer 1.0 plugins for Freescale's i.MX platform, which make use of the i.MX multimedia capabilities.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements:

$ gst-inspect-1.0

Lists all available sink elements:

$ gst-inspect-1.0 | grep sink

For example:

root@imx8m-var-dart:~# gst-inspect-1.0 | grep sink
alsa:  alsasink: Audio sink (ALSA)
waylandsink:  waylandsink: wayland video sink
shm:  shmsink: Shared Memory Sink
decklink:  decklinkaudiosink: Decklink Audio Sink
decklink:  decklinkvideosink: Decklink Video Sink
ximagesink:  ximagesink: Video sink
soup:  souphttpclientsink: HTTP client sink
autodetect:  autovideosink: Auto video sink
autodetect:  autoaudiosink: Auto audio sink
gdkpixbuf:  gdkpixbufsink: GdkPixbuf sink
playback:  playsink: Player Sink
video4linux2:  v4l2sink: Video (video4linux2) Sink
fbdevsink:  fbdevsink: fbdev video sink
tcp:  tcpclientsink: TCP client sink
tcp:  tcpserversink: TCP server sink
...

Detailed help information on a specific element:

$ gst-inspect-1.0 vpudec

For example:

root@imx8m-var-dart:~# gst-inspect-1.0 vpudec
Factory Details:
  Rank                     primary + 1 (257)
  Long-name                VPU-based video decoder
  Klass                    Codec/Decoder/Video
  Description              Decode compressed video to raw data
  Author                   Multimedia Team <shmmmw@freescale.com>

Plugin Details:
  Name                     vpu.imx
  Description              VPU video codec
  Filename                 /usr/lib/gstreamer-1.0/libgstvpu.so
  Version                  4.3.4
  License                  LGPL
  Source module            imx-gst1.0-plugin
  Binary package           Freescle Gstreamer Multimedia Plugins
  Origin URL               http://www.freescale.com

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstVideoDecoder
                         +----GstVpuDec

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-h265
      video/x-vp9
      video/x-vp8
      video/x-vp6-flash
      video/x-h264
      video/mpeg
           systemstream: false
            mpegversion: { (int)1, (int)2 }
      video/mpeg
            mpegversion: 4
      video/x-h263
      video/x-flash-video
             flvversion: 1
      video/x-divx
            divxversion: 3
      video/x-divx
            divxversion: 4
      video/x-divx
            divxversion: { (int)5, (int)6 }
      video/x-xvid
      video/x-cavs
      video/x-wmv
             wmvversion: 3
                 format: WMV3
      video/x-wmv
             wmvversion: 3
                 format: WVC1
      video/x-pn-realvideo
      image/jpeg
      image/webp

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)NV12, (string)I420, (string)YV12, (string)Y42B, (string)NV16, (string)Y444, (string)NV24, (string)NV12_10LE }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_video_decoder_change_state

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "vpudec0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  output-format       : set raw video format for output (Y42B NV16 Y444 NV24 only for MJPEG)
                        flags: readable, writable
                        Enum "GstVpuDecOutputFormat" Default: 0, "auto"
                           (0): auto             - enable chroma interleave. (default)
                           (1): NV12             - NV12 format
                           (2): I420             - I420 format
                           (3): YV12             - YV12 format
                           (4): Y42B             - Y42B format
                           (5): NV16             - NV16 format
                           (6): Y444             - Y444 format
                           (7): NV24             - NV24 format
  frame-drop          : enable adaptive frame drop for smoothly playback
                        flags: readable, writable
                        Boolean. Default: true
  frame-plus          : set number of addtional frames for smoothly playback
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 16 Default: 3 
  use-vpu-memory      : use vpu allocate video frame buffer
                        flags: readable, writable
                        Boolean. Default: true

Test Pattern Stream

Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:

$ gst-launch-1.0 videotestsrc ! kmssink

Gst1.jpg

Test pattern with specific parameters on the input element

$ gst-launch-1.0 videotestsrc pattern=circular ! kmssink 

Gst2.jpg

Camera Loopback

Stream video from a camera to a display

$ gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink

Video Recording

Save video stream from camera as an MJPEG-encoded AVI file

$ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=100 ! video/x-raw,width=1920,height=1080 ! jpegenc ! avimux ! filesink location=./test0.avi

Video Playback

Play MP4 video file

$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4 

Play MP4 video explicitly using i.MX8M VPU decoder

gst-launch-1.0 filesrc location=/run/media/sda1/big_buck_bunny.mp4 typefind=true \
! video/quicktime ! aiurdemux ! queue max-size-time=0 ! vpudec ! kmssink

RTP Video streaming

Encode video stream from camera using software jpeg encoder and stream it over RTP

 
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,width=1280,height=720 ! jpegenc ! rtpjpegpay ! udpsink host=192.168.20.60 port=1223

References