DART-MX8M GSTREAMER: Difference between revisions

From Variscite Wiki
(Created page with "{{PageHeader|VAR-SOM-MX6 - Streaming Video}} {{DocImage|category1=DART-MX8M|category2=Yocto}} __toc__ = Introduction to GStreamer = [https://en.wikipedia.org/wiki/GStreamer G...")
 
No edit summary
Line 160: Line 160:


=== gst-launch ===
=== gst-launch ===
For Example launch the camera and display on the monitor:
For example launch the camera and display on the monitor:
  $ gst-launch-1.0 -v videotestsrc ! imxv4l2sink
  $ gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink


=== i.MX6 major elements ===
{| class="wikitable"
|-
! scope="col" | Prefix<br/>
! scope="col" | Element Name<br/>
! scope="col" | description<br/>
|-
| imxaudio
| imxmp3audioenc
| imx mp3 audio encoder
|-
| imxg2d
| imxg2dvideosink
| IMX G2D video sink ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/blitter-architecture.md blitter-based], [https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
| imxipu
| imxipuvideosink
| IMX IPU video sink ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/blitter-architecture.md blitter-based], [https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
| imxpxp
| imxpxpvideosink
| IMX PxP video sink ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/blitter-architecture.md blitter-based], [https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
| imxeglvivsink
| imxeglvivsink
| IMX EGL video sink ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
| imxv4l2videosrc
| imxv4l2videosrc
| IMX V4L2 CSI Video Source ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
| imxvpu
| imxvpudec
| IMX VPU video decoder ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
| imxvpu
| imxvpuenc_h263
| VPU-based h.263 video encoder ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
| imxvpu
| imxvpuenc_h264
| VPU-based H264 video encoder ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
|-
| imxvpu
| imxvpuenc_h264
| VPU-based MPEG video encoder ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
|-
| imxvpu
| imxvpuenc_jpeg
| VPU-based JPEG video encoder ([https://github.com/Freescale/gstreamer-imx/blob/master/docs/zerocopy.md zerocopy])
|-
|-
| imxipu
| imxipuvideotransform
| IMX IPU video transform
|-
|-
| imxpxp
| imxpxpvideotransform
| IMX PXP video transform
|-
|-
| imxg2d
| imxg2dvideotransform
| IMX G2D video transform
|-
|-
| imxipu
| imxipucompositor
| IMX IPU video compositor
|-
|-
| imxg2d
| imxg2dcompositor
| IMX G2D video compositor
|-
|}
{{Note| Only imxeglvivsink will work with X11 or Wayland. More optimized blitter-base PXP,IPU and G2D sinks will work with fslc_framebuffer only}}
{{Note| See more detailed description [https://github.com/Freescale/gstreamer-imx/blob/master/README.md#available-plugins here]}}


= Test Pattern Stream =
= Test Pattern Stream =
Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
<br/>Default test pattern:
<br/>Default test pattern:
<pre>
<pre>
$ export DISPLAY=:0
$ gst-launch-1.0 videotestsrc ! kmssink
</pre>
<pre>
$ gst-launch-1.0 videotestsrc ! borderless-window="true" force-aspect-ratio="false"
</pre>
</pre>
[[File:gst1.jpg]]
[[File:gst1.jpg]]
<br/><br/>Test pattern with specific paramters on the input element
<br/><br/>Test pattern with specific parameters on the input element
<pre>$ gst-launch-1.0 videotestsrc pattern=circular ! imxeglvivsink borderless-window="true" force-aspect-ratio="false"</pre>
<pre>$ gst-launch-1.0 videotestsrc pattern=circular ! kmssink </pre>
[[File:gst2.jpg]]
[[File:gst2.jpg]]
<br/><br/>Test pattern with specific paramters on the output element
<pre>$ gst-launch-1.0 videotestsrc ! imxeglvivsink window-width="1280" window-height="720" borderless-window="true" force-aspect-ratio="false"</pre>
[[File:gst4.jpg]]
<br/><br/>Test pattern with specific paramters on the input and output elements
<pre>$ gst-launch-1.0 videotestsrc pattern=circular ! imxeglvivsink window-width="1280" window-height="720" borderless-window="true" force-aspect-ratio="false"</pre>
[[File:gst3.jpg]]


= Camera Loopback  =
= Camera Loopback  =
Stream video from a camera to a display</br>
Stream video from a camera to a display</br>
Command:</br>
Command:</br>
<pre>
<pre>$ gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink</pre>
$ export DISPLAY=:0
</pre>
<pre>$ gst-launch-1.0 imxv4l2videosrc device=/dev/video0 imx-capture-mode=5 fps-n=30 ! imxeglvivsink  window-width="1920" window-height="1080" borderless-window="true" force-aspect-ratio="false"</pre>
{{Note| Only imxeglvivsink will work with X11 or Wayland with some rendering artifacts.<br> More optimized blitter-base '''imxg2dvideosink''' and '''imxipuvideosink''' sinks will properly work with fslc_framebuffer DISTRO only}}<br>
 
Get the camera support format and resolution using gst-inspect-1.0 imxv4l2videosrc.</br>


'''imx-capture-mode''' for ov5640 example resolutions:
= Video Recording =  
<pre>
ov5640_mode_VGA_640_480 = 0,
ov5640_mode_QVGA_320_240 = 1,
ov5640_mode_NTSC_720_480 = 2,
ov5640_mode_PAL_720_576 = 3,
ov5640_mode_720P_1280_720 = 4,
ov5640_mode_1080P_1920_1080 = 5
</pre>
 
= Video Record =  
command:</br>
command:</br>
gst-launch-1.0 imxv4l2videosrc device=$DEVICE num-buffers=300 imx-capture-mode=$MODE ! queue ! imxvpuenc_XXX ! $MUXER ! filesink location=output.$EXTENSION </br>
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! queue ! imxvpuenc_XXX ! $MUXER ! filesink location=output.$EXTENSION </br>
 
Parameter comments:
:• $DEVICE could be set to /dev/video, /dev/video0 or according to the system video input device.
:• num-buffers - Number of buffers to output before sending EOS. deafult =-1 (unlimited)
:• $MODE should be set according to camera format and resolution
:• imxvpuenc_XXX can be imxvpuenc_mpeg4,imxvpuenc_h263, imxvpuenc_h264, and imxvpuenc_jpeg
:• MUXER can be set as to qtmux, matroskamux, mp4mux, avimux, or flvmux
:• EXTENSION is filename extension according to the muxer type.
</br>
Record video from a camera into a file.
Encode it to h264 at a bitrate of 10mbit/s (CBR) </br>
720p:
<pre>$ gst-launch-1.0 imxv4l2videosrc device=/dev/video0 imx-capture-mode=4 fps-n=30 num-buffers=300 ! queue ! imxvpuenc_h264 bittrate=10000 ! avimux ! filesink location=test.avi</pre>
1080p:
<pre>$ gst-launch-1.0 imxv4l2videosrc device=/dev/video0 imx-capture-mode=5 fps-n=30 num-buffers=300 ! queue ! imxvpuenc_h264 bitrate=10000 ! avimux ! filesink location=test.avi</pre>


= Simple Movie Play =  
= Video Playback =  
<pre>
<pre>
$ export DISPLAY=:0
$ export DISPLAY=:0
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4  
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4  
</pre>
</pre>
= Compositing =
The compositor is a new feature in gstreamer-imx 0.11.0. Just like with the compositor from gst-plugins-base 1.5.1 and newer, compositor elements support an arbitrary number of request sink pads, and one srcpad.
<pre>
$ gst-launch-1.0  \
  imxg2dcompositor name=c background-color=0x223344 \
      sink_0::xpos=0 sink_0::ypos=90 sink_0::width=160 sink_0::height=110 sink_0::zorder=55 sink_0::fill_color=0xff00ff00 sink_0::alpha=0.39 sink_0::rotation=0 \
      sink_1::xpos=0 sink_1::ypos=20 sink_1::width=620 sink_1::height=380 sink_1::fill_color=0x44441133 ! \
    queue2 ! "video/x-raw, width=800, height=600" ! imxg2dvideosink \
  videotestsrc pattern=0 ! "video/x-raw, framerate=30/1" ! c.sink_0 \
  videotestsrc pattern=18 ! "video/x-raw, framerate=30/1" ! c.sink_1
</pre>
{{Note|See detailed description [https://github.com/Freescale/gstreamer-imx/blob/master/README.md#compositing here]}}


= References =
= References =
* The GStreamer [https://gstreamer.freedesktop.org/ website]
* The GStreamer [https://gstreamer.freedesktop.org/ website]
* The gstreamer-imx [https://github.com/Freescale/gstreamer-imx/blob/master/README.md README]
* The gstreamer-imx [https://github.com/Freescale/gstreamer-imx/blob/master/docs/faq.md FAQ]
* A Deep Dive into Image Processing for i.MX 6 Application Processors [http://cache.freescale.com/files/training/doc/ftf/2014/FTF-CON-F0119.pdf presentation]

Revision as of 17:00, 12 November 2018

VAR-SOM-MX6 - Streaming Video

Introduction to GStreamer

GStreamer is a powerful pipeline-based multimedia framework.
It allows programmers to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing.
This is not a GStreamer reference manual but an overview for using it on the Variscite i.MX8 products.
This release uses the gstreamer-imx set of GStreamer 1.0 plugins for Freescale's i.MX platform, which make use of the i.MX multimedia capabilities.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements: 

$ gst-inspect-1.0

Lists all available sink elements: 

$ gst-inspect-1.0 | grep sink

For example: 

root@imx8m-var-dart:~# gst-inspect-1.0 | grep sink
alsa:  alsasink: Audio sink (ALSA)
waylandsink:  waylandsink: wayland video sink
shm:  shmsink: Shared Memory Sink
decklink:  decklinkaudiosink: Decklink Audio Sink
decklink:  decklinkvideosink: Decklink Video Sink
ximagesink:  ximagesink: Video sink
soup:  souphttpclientsink: HTTP client sink
autodetect:  autovideosink: Auto video sink
autodetect:  autoaudiosink: Auto audio sink
gdkpixbuf:  gdkpixbufsink: GdkPixbuf sink
playback:  playsink: Player Sink
video4linux2:  v4l2sink: Video (video4linux2) Sink
fbdevsink:  fbdevsink: fbdev video sink
tcp:  tcpclientsink: TCP client sink
tcp:  tcpserversink: TCP server sink
...

Detailed help information on a specific element: 

$ gst-inspect-1.0 kmssink

For example: 

root@imx8m-var-dart:~# gst-inspect-1.0 kmssink
Factory Details:
  Rank                     secondary (128)
  Long-name                KMS video sink
  Klass                    Sink/Video
  Description              Video sink using the Linux kernel mode setting API
  Author                   Víctor Jáquez <vjaquez@igalia.com>

Plugin Details:
  Name                     kms
  Description              Video sink using the Linux kernel mode setting API
  Filename                 /usr/lib/gstreamer-1.0/libgstkms.so
  Version                  1.12.2
  License                  LGPL
  Source module            gst-plugins-bad
  Source release date      2017-07-14
  Binary package           GStreamer Bad Plug-ins source release
  Origin URL               Unknown package origin

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSink
                         +----GstVideoSink
                               +----GstKMSSink

Implemented Interfaces:
  GstVideoOverlay

Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)BGRA, (string)BGRx, (string)RGBA, (string)RGBx, (string)UYVY, (string)YUY2, (string)YVYU, (string)I420, (string)YV12, (string)Y42B, (string)NV12, (string)NV12_10LE, (stri
ng)NV21, (string)NV16 }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: gst_kms_sink_change_state

Element has no clocking capabilities.
Element has no URI handling capabilities.

Pads:
  SINK: 'sink'
    Pad Template: 'sink'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "kmssink0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000 
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 
  enable-last-sample  : Enable the last-sample property
                        flags: readable, writable
                        Boolean. Default: true
  last-sample         : The last sample received in the sink
                        flags: readable
                        Boxed pointer of type "GstSample"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  throttle-time       : The time to keep between rendered buffers (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  max-bitrate         : The maximum bits per second to render (0 = disabled)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  driver-name         : DRM device driver name
                        flags: readable, writable
                        String. Default: null
  connector-id        : DRM connector id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  plane-id            : DRM plane id
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  force-modesetting   : When enabled, the sink try to configure the display mode
                        flags: readable, writable
                        Boolean. Default: false
  global-alpha        : global alpha
                        flags: readable, writable
                        Integer. Range: 0 - 255 Default: 0 
  force-hantrotile    : When enabled, the sink propose hantro tile modifier to VPU
                        flags: readable, writable
                        Boolean. Default: false

gst-launch

For example launch the camera and display on the monitor: 

$ gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink


Test Pattern Stream

Test pattern allows you to display pre-built image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern: 

$ gst-launch-1.0 videotestsrc ! kmssink

Gst1.jpg

Test pattern with specific parameters on the input element 

$ gst-launch-1.0 videotestsrc pattern=circular ! kmssink

Gst2.jpg

Camera Loopback

Stream video from a camera to a display
Command:

$ gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink

Video Recording

command:
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=300 ! queue ! imxvpuenc_XXX ! $MUXER ! filesink location=output.$EXTENSION

Video Playback

$ export DISPLAY=:0
$ gst-launch-1.0 playbin uri=file:/run/media/sda1/big_buck_bunny.mp4

References

Retrieved from "https://variwiki.com/index.php?title=DART-MX8M_GSTREAMER&oldid=10527"
Latest Modules
Support
Variscite Support Center
Contact Support
Product Longevity
Product Compliance Policy
Blog
Variscite Training
Variscite Website
This page was last edited on 12 November 2018, at 17:00.

© 2024 Variscite. All rights reserved