VAR-SOM-MX6 GSTREAM

VAR-SOM-MX6 - Streaming Video

Under construction!!!

Stream Camera with imxcamera application

On your host machine install Video Lan VLC VideoLan

• Find out your host IP address (Windows:ipconfig, Linux:ifconfig)
• Start VLC, Media->Open Network Stream. Set the URL to "rtp://@:5004" and press "Play".
• On target, start the imxcamera application.
  • Press the Settings button. Select "Enable RTP" box and set the IP to you host machine. Press OK.
  • Press the Camera button to get a Video camera Image.
  • Press the blue button and the camera will be streamed to you host VLC.

Introduction to GStreamer

GStreamer
GStreamer is a powerful pipeline-based multimedia framework.
GStreamer allows a programmer to create a variety of media-handling components, including simple audio playback, audio and video playback, recording, streaming and editing. This is not a GStreamer reference manual but an overview for using it on Variscite i.MX6 products.

Major GStreamer commands

There are two major commands:
gst-inspect - allows you to to get documentation on available elements and detailed information on a specific element.
gst-launch - allows you to create and execute GStreamer pipelines.

gst-inspect

Lists all available elements:

$ gst-inspect 

Lists all available i.MX6 specific elements:

$ gst-inspect | grep imx

For example:

root@var-som-mx6:~#  gst-inspect | grep imx
mp3enc.imx:  mfw_mp3encoder: mp3 audio encoder
v4lsink.imx:  imxv4l2sink: v4l2 video sink
isink.imx:  mfw_isink: IPU-based video sink
aiur.imx: webm: webm
aiur.imx:  aiurdemux: aiur universal demuxer
tvsrc.imx:  tvsrc: v4l2 based tv src
v4lsrc.imx:  imxv4l2src: v4l2 based camera src
audiopeq.imx:  mfw_audio_pp: audio post equalizer
beep.imx: ac3: ac3
beep.imx: 3ca: ac3
beep.imx:  beepdec: beep audio decoder
beep.imx:  beepdec.vorbis: Vorbis decoder
beep.imx:  beepdec.mp3: MP3 decoder
beep.imx:  beepdec.aac: AAC LC decoder
amrdec.imx:  mfw_amrdecoder: amr audio decoder
vpu.imx:  vpudec: VPU-based video decoder
vpu.imx:  vpuenc: VPU-based video encoder
ipucsc.imx:  mfw_ipucsc: IPU-based video converter

Detailed help information on a specific element:

$ gst-inspect imxv4l2sink

For example:

root@var-som-mx6:~# gst-inspect imxv4l2sink
MFW_GST_V4LSINK_PLUGIN 4.0.3 build on Dec 16 2015 08:44:04.
Factory Details:
  Long name:    v4l2 video sink
  Class:        Sink/Video
  Description:  Display video by using v4l2 interface
  Author(s):    Multimedia Team <shmmmw@freescale.com>
  Rank:         primary + 2 (258)

======================= Was cut ====================
Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "mfw_gst_v4lsink_info_t0"
  preroll-queue-len   : Number of buffers to queue during preroll
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 0 
  sync                : Sync on the clock
                        flags: readable, writable
                        Boolean. Default: true
  max-lateness        : Maximum number of nanoseconds that a buffer can be late before it is dropped (-1 unlimited)
                        flags: readable, writable
                        Integer64. Range: -1 - 9223372036854775807 Default: 20000000 
  qos                 : Generate Quality-of-Service events upstream
                        flags: readable, writable
                        Boolean. Default: true
  async               : Go asynchronously to PAUSED
                        flags: readable, writable
                        Boolean. Default: true
  ts-offset           : Timestamp offset in nanoseconds
                        flags: readable, writable
                        Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0 
  enable-last-buffer  : Enable the last-buffer property
                        flags: readable, writable
                        Boolean. Default: true
  last-buffer         : The last buffer received in the sink
                        flags: readable
                        MiniObject of type "GstBuffer"
  blocksize           : Size in bytes to pull per buffer (0 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  render-delay        : Additional render delay of the sink in nanoseconds
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  throttle-time       : The time to keep between rendered buffers (unused)
                        flags: readable, writable
                        Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0 
  show-preroll-frame  : Whether to render video frames during preroll
                        flags: readable, writable
                        Boolean. Default: true
  disp-width          : gets the width of the image to be displayed
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  disp-height         : gets the height of the image to be displayed
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  axis-top            : gets the top co-ordinate of the origin of display
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  axis-left           : gets the left co-ordinate of the origin of display
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  rotate              : gets the angle at which display is to be rotated
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  crop-left-by-pixel  : set the input image cropping in the left (width)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  crop-right-by-pixel : set the input image cropping in the right (width)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  crop-top-by-pixel   : set the input image cropping in the top (height)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  crop-bottom-by-pixel: set the input image cropping in the bottom (height)
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  tv-out              : set output to TV-OUT
                        flags: readable, writable
                        Boolean. Default: false
  tv-mode             : set mode to TV-OUT, 0: NTSC, 1: PAL, 2: 720p 
                        flags: readable, writable
                        Integer. Range: 0 - 2 Default: 3 
  dump-location       : Location of the file to write cropped video YUV stream.Enable it will output image to file in
stead of V4L device
                        flags: readable, writable
                        String. Default: null
  setpara             : set parameter of V4L2, 1: Set V4L 2: Set Color
                        flags: readable, writable
                        Integer. Range: 0 - 3 Default: 0 
  force-aspect-ratio  : Force Aspect Ratio
                        flags: readable, writable
                        Boolean. Default: false
  x11enable           : Enabled x11 event handle
                        flags: readable, writable
                        Boolean. Default: false
  rendered            : Get the total rendered frames
                        flags: readable, writable
                        Integer. Range: 0 - 2147483647 Default: 0 
  device              : V4L device name
                        flags: readable, writable
                        String. Default: "/dev/video17"
  motion              : The interlace motion setting: 0 - low motion, 1 - medium motion, 2 - high motion.
                        flags: readable, writable
                        Integer. Range: 0 - 2 Default: 2 
  deinterlace         : set deinterlace enabled
                        flags: readable, writable
                        Boolean. Default: true
[--->FINALIZE v4l_sink
root@var-som-mx6:~# 
CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.7 | VT102 | Offline | ttyUSB0                                      

gst-launch

For Example launch the camera and display on the monitor:

gst-launch -v videotestsrc ! imxv4l2sink

i.MX6 major elements

Prefix	Element Name	description
mp3enc.imx	mfw_mp3encoder	mp3 audio encoder
v4lsink.imx	imxv4l2sink	v4l2 video sink
isink.imx	mfw_isink	IPU-based video sink
v4lsrc.imx	imxv4l2src	v4l2 based camera src
vpu.imx	vpudec	VPU-based video decoder
vpu.imx	vpuenc	VPU-based video encoder
ipucsc.imx	mfw_ipucsc	IPU-based video converter

Test Pattern Stream

Test pattern allows you to display pre build image/video patterns on the display. It is very useful when you don't have a camera but still want to test GStreamer.
Default test pattern:

$ gst-launch -v videotestsrc ! imxv4l2sink

Test pattern with specific paramters on the input element

$ gst-launch videotestsrc pattern=circular ! imxv4l2sink

Test pattern with specific paramters on the output element

$ gst-launch videotestsrc ! imxv4l2sink disp-width=1280 disp-height=720

Test pattern with specific paramters on the input and output elements

$ gst-launch videotestsrc pattern=circular ! imxv4l2sink  disp-width=1280 disp-height=720

Camera Loopback

Stream video from a camera to a display

$ gst-launch imxv4l2src device=/dev/video0 capture-mode=4 fps-n=30 queue-size=7 ! queue max-size-buffers=3 ! imxv4l2sink  disp-width=1280 disp-height=720

capture-mode ov5460 options:

ov5640_mode_VGA_640_480 = 0,

ov5640_mode_QVGA_320_240 = 1,

ov5640_mode_NTSC_720_480 = 2,

ov5640_mode_PAL_720_576 = 3,

ov5640_mode_720P_1280_720 = 4,

ov5640_mode_1080P_1920_1080 = 5

This is a 3 elements pipe,the queue in the middle is a GStreamer element.

Video Encode

Encode video from a camera into a file.
720p:

$ gst-launch imxv4l2src capture-mode=4 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi

1080p:

$ gst-launch imxv4l2src capture-mode=5 fps-n=30 ! queue ! vpuenc codec=6 ! matroskamux ! filesink location=test.avi

The codec type for encoding

flags: readable, writable

Enum "vpuenc_codec" Default: 0, "mpeg4"

(0): mpeg4 - video/mpeg,

(5): h263 - video/x-h263

(6): avc - video/x-h264

(12): mjpg - image/jpeg

Simple Movie Play

$ gst-launch playbin2 uri=file:/run/media/sda1/big_buck_bunny.mp4