VAR-SOM-MX6 GSTREAM: Difference between revisions

From Variscite Wiki
 
(54 intermediate revisions by 2 users not shown)
Line 3: Line 3:
= Stream Camera with imxcamera application =
= Stream Camera with imxcamera application =
On your host machine install Video Lan VLC [http://www.videolan.org/vlc/index.html VideoLan]
On your host machine install Video Lan VLC [http://www.videolan.org/vlc/index.html VideoLan]
* Find out your host IP address (Windows:ipconfig Linux:ifconfig)
* Find out your host IP address (Windows:ipconfig, Linux:ifconfig)
* Start VLC, Media->Open Network Stream. Set the URL to "rtp://@:5004" and hit "Play".
* Start VLC, Media->Open Network Stream. Set the URL to "rtp://@:5004" and press "Play".
* On target start imxcamera application.
* On target, start the imxcamera application.
** Hit the Settigns button. Select "Enable RTP" box and set the IP to you host machine. Hit OK.
** Press the Settings button. Select "Enable RTP" box and set the IP to you host machine. Press OK.
** Push the Camera button to get a Video camera Image.
** Press the Camera button to get a Video camera Image.
** Hit the blue button and the camera will be streamed to you host VLC.
** Press the blue button and the camera will be streamed to you host VLC.
= Introduction to gstream =
 
= Introduction to GStreamer =
[https://en.wikipedia.org/wiki/GStreamer GStreamer]  
[https://en.wikipedia.org/wiki/GStreamer GStreamer]  
<br/>GStreamer is a powerful pipeline-based multimedia framework.
<br/>GStreamer is a powerful pipeline-based multimedia framework.
<br/>GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but a overview for using in on Variscite i.MX6 products.
<br/>GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products.


== Major GStreamer commands ==
== Major GStreamer commands ==
Two major commands, the first one allow you to to get documentation on available elements and detail information on a specific element.
There are two major commands:<br/>
The second one allow you to create and execute GStreamer  pipelines.
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
<br/>gst-launch - allows you to create and execute GStreamer  pipelines.
=== gst-inspect ===
=== gst-inspect ===
List all available elements:
Lists all available elements:
<pre>$ gst-inspect </pre>
<pre>$ gst-inspect </pre>
List all avilable i.MX6 specific elements:
Lists all available i.MX6 specific elements:
<pre>$ gst-inspect | grep imx</pre>
<pre>$ gst-inspect | grep imx</pre>
Detail help information on a specific element:
For example:
<pre>root@var-som-mx6:~#  gst-inspect | grep imx
mp3enc.imx:  mfw_mp3encoder: mp3 audio encoder
v4lsink.imx:  imxv4l2sink: v4l2 video sink
isink.imx:  mfw_isink: IPU-based video sink
aiur.imx: webm: webm
aiur.imx:  aiurdemux: aiur universal demuxer
tvsrc.imx:  tvsrc: v4l2 based tv src
v4lsrc.imx:  imxv4l2src: v4l2 based camera src
audiopeq.imx:  mfw_audio_pp: audio post equalizer
beep.imx: ac3: ac3
beep.imx: 3ca: ac3
beep.imx:  beepdec: beep audio decoder
beep.imx:  beepdec.vorbis: Vorbis decoder
beep.imx:  beepdec.mp3: MP3 decoder
beep.imx:  beepdec.aac: AAC LC decoder
amrdec.imx:  mfw_amrdecoder: amr audio decoder
vpu.imx:  vpudec: VPU-based video decoder
vpu.imx:  vpuenc: VPU-based video encoder
ipucsc.imx:  mfw_ipucsc: IPU-based video converter
</pre>
Detailed help information on a specific element:
<pre>$ gst-inspect imxv4l2sink</pre>
<pre>$ gst-inspect imxv4l2sink</pre>
For example:
<pre>root@var-som-mx6:~# gst-inspect imxv4l2sink
MFW_GST_V4LSINK_PLUGIN 4.0.3 build on Dec 16 2015 08:44:04.
Factory Details:
  Long name:    v4l2 video sink
  Class:        Sink/Video
  Description:  Display video by using v4l2 interface
  Author(s):    Multimedia Team <shmmmw@freescale.com>
  Rank:        primary + 2 (258)
======================= Was cut ====================
Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "mfw_gst_v4lsink_info_t0"
  preroll-queue-len  : Number of buffers to queue during preroll
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000
  qos                : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async              : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset          : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
  enable-last-buffer  : Enable the last-buffer property
                        flags: readable, writable
                        Boolean. Default: true
  last-buffer        : The last buffer received in the sink
                        flags: readable
                        MiniObject of type "GstBuffer"
  blocksize          : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  throttle-time      : The time to keep between rendered buffers (unused)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  disp-width          : gets the width of the image to be displayed
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  disp-height        : gets the height of the image to be displayed
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  axis-top            : gets the top co-ordinate of the origin of display
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  axis-left          : gets the left co-ordinate of the origin of display
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  rotate              : gets the angle at which display is to be rotated
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  crop-left-by-pixel  : set the input image cropping in the left (width)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  crop-right-by-pixel : set the input image cropping in the right (width)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  crop-top-by-pixel  : set the input image cropping in the top (height)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  crop-bottom-by-pixel: set the input image cropping in the bottom (height)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  tv-out              : set output to TV-OUT
                        flags: readable, writable
                        Boolean. Default: false
  tv-mode            : set mode to TV-OUT, 0: NTSC, 1: PAL, 2: 720p
                        flags: readable, writable
                        Integer. Range: 0 - 2 Default: 3
  dump-location      : Location of the file to write cropped video YUV stream.Enable it will output image to file in
stead of V4L device
                        flags: readable, writable
                        String. Default: null
  setpara            : set parameter of V4L2, 1: Set V4L 2: Set Color
                        flags: readable, writable
                        Integer. Range: 0 - 3 Default: 0
  force-aspect-ratio  : Force Aspect Ratio
                        flags: readable, writable
                        Boolean. Default: false
  x11enable          : Enabled x11 event handle
                        flags: readable, writable
                        Boolean. Default: false
  rendered            : Get the total rendered frames
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0
  device              : V4L device name
                        flags: readable, writable
                        String. Default: "/dev/video17"
  motion              : The interlace motion setting: 0 - low motion, 1 - medium motion, 2 - high motion.
                        flags: readable, writable
                        Integer. Range: 0 - 2 Default: 2
  deinterlace        : set deinterlace enabled
                        flags: readable, writable
                        Boolean. Default: true
[--->FINALIZE v4l_sink
root@var-som-mx6:~#
CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7 | VT102 | Offline | ttyUSB0                                     
</pre>


=== gst-launch ===
=== gst-launch ===
Simple example:
For Example launch the camera and display on the monitor:
<pre>gst-launch -v videotestsrc ! imxv4l2sink</pre>
<pre>gst-launch -v videotestsrc ! imxv4l2sink</pre>
=== i.MX6 specific elements ===


= Stream Camera with gstream =
=== i.MX6 major elements ===
{| class="wikitable"
|-
! scope="col" | Prefix<br/>
! scope="col" | Element Name<br/>
! scope="col" | description<br/>
|-
| mp3enc.imx
| mfw_mp3encoder
| mp3 audio encoder
|-
| v4lsink.imx
| imxv4l2sink
| v4l2 video sink
|-
| isink.imx
| mfw_isink
| IPU-based video sink
|-
| v4lsrc.imx
| imxv4l2src
| v4l2 based camera src
|-
| vpu.imx
| vpudec
| VPU-based video decoder
|-
|vpu.imx
|vpuenc
|VPU-based video encoder
|-
| ipucsc.imx
| mfw_ipucsc
| IPU-based video converter
|-
|}
 
= Test Pattern Stream =
Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
<br/>Default test pattern:
<pre>$ gst-launch -v videotestsrc ! imxv4l2sink</pre>
[[File:gst1.jpg]]
<br/><br/>Test pattern with specific paramters on the input element
<pre>$ gst-launch videotestsrc pattern=circular ! imxv4l2sink</pre>
[[File:gst2.jpg]]
<br/><br/>Test pattern with specific paramters on the output element
<pre>$ gst-launch videotestsrc ! imxv4l2sink disp-width=1280 disp-height=720</pre>
[[File:gst4.jpg]]
<br/><br/>Test pattern with specific paramters on the input and output elements
<pre>$ gst-launch videotestsrc pattern=circular ! imxv4l2sink  disp-width=1280 disp-height=720</pre>
[[File:gst3.jpg]]
 
= Camera Loopback  =
Stream video from a camera to a display
<pre>$ gst-launch imxv4l2src device=/dev/video0 capture-mode=4 fps-n=30 queue-size=7 ! queue max-size-buffers=3 ! imxv4l2sink  disp-width=1280 disp-height=720</pre>
:capture-mode ov5460 options: 
:::ov5640_mode_VGA_640_480 = 0,
:::ov5640_mode_QVGA_320_240 = 1,
:::ov5640_mode_NTSC_720_480 = 2,
:::ov5640_mode_PAL_720_576 = 3,
:::ov5640_mode_720P_1280_720 = 4,
:::ov5640_mode_1080P_1920_1080 = 5 </br>
This is a 3 elements pipe,the queue in the middle is a GStreamer element.
 
= Video Encode =
Encode video from a camera into a file.</br>
720p:
<pre>$ gst-launch imxv4l2src capture-mode=4 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi</pre>
1080p:
<pre>$ gst-launch imxv4l2src capture-mode=5 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi</pre>
:The codec type for encoding
:::flags: readable, writable
:::Enum "vpuenc_codec" Default: 0, "mpeg4"
::::(0): mpeg4            - video/mpeg,
::::(5): h263            - video/x-h263
::::(6): avc              - video/x-h264
::::(12): mjpg            - image/jpeg
 
= Simple Movie Play =
<pre>$ gst-launch playbin2 uri=file:/run/media/sda1/big_buck_bunny.mp4
</pre>

Latest revision as of 14:29, 31 January 2016

VAR-SOM-MX6 - Streaming Video

Under construction!!!

Stream Camera with imxcamera application

On your host machine install Video Lan VLC VideoLan

  • Find out your host IP address (Windows:ipconfig, Linux:ifconfig)
  • Start VLC, Media->Open Network Stream. Set the URL to "rtp://@:5004" and press "Play".
  • On target, start the imxcamera application.
    • Press the Settings button. Select "Enable RTP" box and set the IP to you host machine. Press OK.
    • Press the Camera button to get a Video camera Image.
    • Press the blue button and the camera will be streamed to you host VLC.

Introduction to GStreamer

GStreamer
GStreamer is a powerful pipeline-based multimedia framework.
GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements:

$ gst-inspect 

Lists all available i.MX6 specific elements:

$ gst-inspect | grep imx

For example:

root@var-som-mx6:~#  gst-inspect | grep imx
mp3enc.imx:  mfw_mp3encoder: mp3 audio encoder
v4lsink.imx:  imxv4l2sink: v4l2 video sink
isink.imx:  mfw_isink: IPU-based video sink
aiur.imx: webm: webm
aiur.imx:  aiurdemux: aiur universal demuxer
tvsrc.imx:  tvsrc: v4l2 based tv src
v4lsrc.imx:  imxv4l2src: v4l2 based camera src
audiopeq.imx:  mfw_audio_pp: audio post equalizer
beep.imx: ac3: ac3
beep.imx: 3ca: ac3
beep.imx:  beepdec: beep audio decoder
beep.imx:  beepdec.vorbis: Vorbis decoder
beep.imx:  beepdec.mp3: MP3 decoder
beep.imx:  beepdec.aac: AAC LC decoder
amrdec.imx:  mfw_amrdecoder: amr audio decoder
vpu.imx:  vpudec: VPU-based video decoder
vpu.imx:  vpuenc: VPU-based video encoder
ipucsc.imx:  mfw_ipucsc: IPU-based video converter

Detailed help information on a specific element:

$ gst-inspect imxv4l2sink

For example:

root@var-som-mx6:~# gst-inspect imxv4l2sink
MFW_GST_V4LSINK_PLUGIN 4.0.3 build on Dec 16 2015 08:44:04.
Factory Details:
  Long name:    v4l2 video sink
  Class:        Sink/Video
  Description:  Display video by using v4l2 interface
  Author(s):    Multimedia Team <shmmmw@freescale.com>
  Rank:         primary + 2 (258)

======================= Was cut ====================
Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "mfw_gst_v4lsink_info_t0"
  preroll-queue-len   : Number of buffers to queue during preroll
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000 
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 
  enable-last-buffer  : Enable the last-buffer property
                        flags: readable, writable
                        Boolean. Default: true
  last-buffer         : The last buffer received in the sink
                        flags: readable
                        MiniObject of type "GstBuffer"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  throttle-time       : The time to keep between rendered buffers (unused)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  disp-width          : gets the width of the image to be displayed
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  disp-height         : gets the height of the image to be displayed
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  axis-top            : gets the top co-ordinate of the origin of display
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  axis-left           : gets the left co-ordinate of the origin of display
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  rotate              : gets the angle at which display is to be rotated
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  crop-left-by-pixel  : set the input image cropping in the left (width)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  crop-right-by-pixel : set the input image cropping in the right (width)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  crop-top-by-pixel   : set the input image cropping in the top (height)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  crop-bottom-by-pixel: set the input image cropping in the bottom (height)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  tv-out              : set output to TV-OUT
                        flags: readable, writable
                        Boolean. Default: false
  tv-mode             : set mode to TV-OUT, 0: NTSC, 1: PAL, 2: 720p 
                        flags: readable, writable
                        Integer. Range: 0 - 2 Default: 3 
  dump-location       : Location of the file to write cropped video YUV stream.Enable it will output image to file in
stead of V4L device
                        flags: readable, writable
                        String. Default: null
  setpara             : set parameter of V4L2, 1: Set V4L 2: Set Color
                        flags: readable, writable
                        Integer. Range: 0 - 3 Default: 0 
  force-aspect-ratio  : Force Aspect Ratio
                        flags: readable, writable
                        Boolean. Default: false
  x11enable           : Enabled x11 event handle
                        flags: readable, writable
                        Boolean. Default: false
  rendered            : Get the total rendered frames
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  device              : V4L device name
                        flags: readable, writable
                        String. Default: "/dev/video17"
  motion              : The interlace motion setting: 0 - low motion, 1 - medium motion, 2 - high motion.
                        flags: readable, writable
                        Integer. Range: 0 - 2 Default: 2 
  deinterlace         : set deinterlace enabled
                        flags: readable, writable
                        Boolean. Default: true
[--->FINALIZE v4l_sink
root@var-som-mx6:~# 
CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7 | VT102 | Offline | ttyUSB0                                      

gst-launch

For Example launch the camera and display on the monitor:

gst-launch -v videotestsrc ! imxv4l2sink

i.MX6 major elements

Prefix
Element Name
description
mp3enc.imx mfw_mp3encoder mp3 audio encoder
v4lsink.imx imxv4l2sink v4l2 video sink
isink.imx mfw_isink IPU-based video sink
v4lsrc.imx imxv4l2src v4l2 based camera src
vpu.imx vpudec VPU-based video decoder
vpu.imx vpuenc VPU-based video encoder
ipucsc.imx mfw_ipucsc IPU-based video converter

Test Pattern Stream

Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:

$ gst-launch -v videotestsrc ! imxv4l2sink

Gst1.jpg

Test pattern with specific paramters on the input element

$ gst-launch videotestsrc pattern=circular ! imxv4l2sink

Gst2.jpg

Test pattern with specific paramters on the output element

$ gst-launch videotestsrc ! imxv4l2sink disp-width=1280 disp-height=720

Gst4.jpg

Test pattern with specific paramters on the input and output elements

$ gst-launch videotestsrc pattern=circular ! imxv4l2sink  disp-width=1280 disp-height=720

Gst3.jpg

Camera Loopback

Stream video from a camera to a display

$ gst-launch imxv4l2src device=/dev/video0 capture-mode=4 fps-n=30 queue-size=7 ! queue max-size-buffers=3 ! imxv4l2sink  disp-width=1280 disp-height=720
capture-mode ov5460 options:
ov5640_mode_VGA_640_480 = 0,
ov5640_mode_QVGA_320_240 = 1,
ov5640_mode_NTSC_720_480 = 2,
ov5640_mode_PAL_720_576 = 3,
ov5640_mode_720P_1280_720 = 4,
ov5640_mode_1080P_1920_1080 = 5

This is a 3 elements pipe,the queue in the middle is a GStreamer element.

Video Encode

Encode video from a camera into a file.
720p:

$ gst-launch imxv4l2src capture-mode=4 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi

1080p:

$ gst-launch imxv4l2src capture-mode=5 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi
The codec type for encoding
flags: readable, writable
Enum "vpuenc_codec" Default: 0, "mpeg4"
(0): mpeg4 - video/mpeg,
(5): h263 - video/x-h263
(6): avc - video/x-h264
(12): mjpg - image/jpeg

Simple Movie Play

$ gst-launch playbin2 uri=file:/run/media/sda1/big_buck_bunny.mp4