DART-6UL Eth IEEE.1588

From Variscite Wiki
DART-6UL - IEEE 1588 Hardware Timestamp

1 Overview of IEEE 1588 Hardware Timestamping

We recommend getting the basic overview of the IEE1588 and timestamping and basic terminologies.
Refer to https://elinux.org/images/f/f9/Introduction_to_IEEE_1588_Precision_Time_Protocol_%28PTP%29_Using_Embedded_Linux_Systems.pdf

At the very top level, it can be described as below
IEEE 1588 Basic Overview.png
NXP Supports IEE1588 PTP implemented in the Ethernet SOC MAC layer.
File:IEEE Block Diagram IMX6.png

Above diagram depicts the implementation of hardware timestamping at the SOC level.
The programmable Ethernet MAC with IEEE 1588 integrates a standard IEEE 802.3. controller
Ethernet MAC with a time-stamping module, with IEEE 1588 standard provides accurate.
clock synchronization for distributed control nodes for industrial automation applications.

NXP IEE1588 PTP and timer implementation have below features:

  • Allows reference clock to be chosen independently of network speed.
  • Software-programmable precise time-stamping of ingress and egress frames.
  • Timer monitoring capabilities for system calibration and timing accuracy management.
  • Precise time-stamping of external events with programmable interrupt generation.
  • Programmable event and interrupt generation for external system control.
  • Supports hardware and software-controllable timer synchronization.
  • Provides a 4-channel IEEE 1588 timer. Each channel supports input capture and output compare using the 1588 counter.
  • Distribution of precise time information over the packet-based network.
  • Offers high accuracy (in the sub-microsecond range) over the net

2 Hardware Setup

  • Network Setup: Connect the PC/Grand Master Clock Source and the target to the same Ethernet network (and use ping to verify the connection between the PC and the target).

3 Software

In the current setup, we will have

  • PC - Grand Master which acts as the clock source which has the GPS clock synchronized

Note: You may choose Device (i.MX6UL) as grandmaster clock as a device as well.

  • Device - i.MX8 with IEE1588 HW timestamp support - of which local clock to be synchronized with grandmaster

On PC (Master)

Software utilities required for PTP

$ ptp4l
$ phc2sys

For Ubuntu 16.04

$ sudo apt-get install linuxptp

Start the software base timestamping on PC or to become grandmaster clock source which you want to synchronize.

Make sure you have following device tree change present on your kernel.
Note that in this example, ENET2_EVENT0_OUT is supported so only hardware event is feasible.

For more details, see https://www.variscite.com/wp-content/uploads/2018/01/DART-6UL_DART-6UL-5G_Datasheet.pdf
for all combination of IEE1588 events and pinmuxing.

diff --git a/arch/arm/boot/dts/imx6ul-imx6ull-var-dart-common.dtsi b/arch/arm/boot/dts/imx6ul-imx6ull-var-dart-common.dtsi
index f9170af..7182f97 100644
--- a/arch/arm/boot/dts/imx6ul-imx6ull-var-dart-common.dtsi
+++ b/arch/arm/boot/dts/imx6ul-imx6ull-var-dart-common.dtsi
@@ -487,6 +487,7 @@
                                MX6UL_PAD_ENET2_TX_DATA1__ENET2_TDATA01 0x1b0b0
                                MX6UL_PAD_ENET2_TX_CLK__ENET2_REF_CLK2  0x4001b031
                                MX6UL_PAD_JTAG_MOD__GPIO1_IO10          0x1b0b0
+                               MX6UL_PAD_GPIO1_IO05__ENET2_1588_EVENT0_OUT 0x1b0b0
                        >;
                };
$ sudo ptp4l -i  enp5s0 -m -S 

- Here, in this case, the PC sends the timestamp using software timer.
- The timestamp packets are sent over networking infrastructure via switch/router (IEEE 1588 packets)
and then once received on the target, it is used to synchronize the clock with 1μs accuracy.
- Hardware has the test and compares logic to see the drift and jitter.

3.1 Testing on device side

ptp4l works as a daemon service. It must be started first before any system level PTP client can work.
Yocto build already contains the ptp4l required resoureces on the target.
Run following commands on target

root@imx6ul-var-dart:~# ptp4l -H -A -l 7 -q -i eth1 -m -s
ptp4l[151.372]: config item (null).assume_two_step is 0
ptp4l[151.373]: config item (null).check_fup_sync is 0
ptp4l[151.374]: config item (null).tx_timestamp_timeout is 1
ptp4l[151.375]: config item (null).clock_servo is 0
ptp4l[151.376]: config item (null).time_stamping is 1
ptp4l[151.378]: config item (null).clock_servo is 0
ptp4l[151.379]: config item (null).clockClass is 248
ptp4l[151.380]: config item (null).clockAccuracy is 254
ptp4l[151.381]: config item (null).offsetScaledLogVariance is 65535
ptp4l[151.382]: config item (null).productDescription is ';;'
ptp4l[151.383]: config item (null).revisionData is ';;'
ptp4l[151.383]: config item (null).userDescription is ''
ptp4l[151.383]: config item (null).manufacturerIdentity is '00:00:00'
ptp4l[151.384]: config item (null).domainNumber is 0
ptp4l[151.385]: config item (null).slaveOnly is 1
ptp4l[151.385]: config item (null).twoStepFlag is 1
ptp4l[151.385]: config item (null).priority1 is 128
ptp4l[151.385]: config item (null).priority2 is 128
ptp4l[151.385]: config item (null).gmCapable is 1
ptp4l[151.386]: config item (null).gmCapable is 1
ptp4l[151.387]: config item (null).free_running is 0
ptp4l[151.387]: selected /dev/ptp1 as PTP clock
ptp4l[151.387]: config item (null).uds_address is '/var/run/ptp4l'
ptp4l[151.389]: section item /var/run/ptp4l.announceReceiptTimeout now 0
ptp4l[151.389]: section item /var/run/ptp4l.delay_mechanism now 0
ptp4l[151.389]: section item /var/run/ptp4l.network_transport now 0
ptp4l[151.389]: section item /var/run/ptp4l.delay_filter_length now 1
ptp4l[151.390]: config item (null).free_running is 0
ptp4l[151.390]: config item (null).freq_est_interval is 1
ptp4l[151.391]: config item (null).gmCapable is 1
ptp4l[151.392]: config item (null).kernel_leap is 1
ptp4l[151.392]: config item (null).timeSource is 160
ptp4l[151.392]: config item (null).pi_proportional_const is 0.000000
ptp4l[151.393]: config item (null).pi_integral_const is 0.000000
ptp4l[151.393]: config item (null).pi_proportional_scale is 0.000000
ptp4l[151.393]: config item (null).pi_proportional_exponent is -0.300000
ptp4l[151.393]: config item (null).pi_proportional_norm_max is 0.700000
ptp4l[151.393]: config item (null).pi_integral_scale is 0.000000
ptp4l[151.394]: config item (null).pi_integral_exponent is 0.400000
ptp4l[151.394]: config item (null).pi_integral_norm_max is 0.300000
ptp4l[151.394]: config item (null).step_threshold is 0.000000
ptp4l[151.394]: config item (null).first_step_threshold is 0.000020
ptp4l[151.394]: config item (null).max_frequency is 900000000
ptp4l[151.395]: config item (null).tsproc_mode is 0
ptp4l[151.395]: config item (null).delay_filter is 1
ptp4l[151.395]: config item (null).delay_filter_length is 10
ptp4l[151.395]: config item (null).summary_interval is 0
ptp4l[151.395]: config item (null).sanity_freq_limit is 200000000
ptp4l[151.396]: PI servo: sync interval 1.000 kp 0.700 ki 0.300000
ptp4l[151.396]: config item /var/run/ptp4l.boundary_clock_jbod is 0
ptp4l[151.397]: config item /var/run/ptp4l.network_transport is 0
ptp4l[151.397]: config item /var/run/ptp4l.delayAsymmetry is 0
ptp4l[151.397]: config item /var/run/ptp4l.follow_up_info is 0
ptp4l[151.397]: config item /var/run/ptp4l.freq_est_interval is 1
ptp4l[151.398]: config item /var/run/ptp4l.hybrid_e2e is 0
ptp4l[151.398]: config item /var/run/ptp4l.path_trace_enabled is 0
ptp4l[151.398]: config item /var/run/ptp4l.ingressLatency is 0
ptp4l[151.398]: config item /var/run/ptp4l.egressLatency is 0
ptp4l[151.398]: config item /var/run/ptp4l.delay_mechanism is 0
ptp4l[151.399]: config item /var/run/ptp4l.fault_badpeernet_interval is 16
ptp4l[151.399]: config item /var/run/ptp4l.fault_reset_interval is 4
ptp4l[151.399]: config item /var/run/ptp4l.tsproc_mode is 0
ptp4l[151.399]: config item /var/run/ptp4l.delay_filter is 1
ptp4l[151.399]: config item /var/run/ptp4l.delay_filter_length is 1
ptp4l[151.400]: config item eth1.boundary_clock_jbod is 0
ptp4l[151.400]: config item eth1.network_transport is 1
ptp4l[151.401]: config item eth1.delayAsymmetry is 0
ptp4l[151.401]: config item eth1.follow_up_info is 0
ptp4l[151.401]: config item eth1.freq_est_interval is 1
ptp4l[151.401]: config item eth1.hybrid_e2e is 0
ptp4l[151.401]: config item eth1.path_trace_enabled is 0
ptp4l[151.401]: config item eth1.ingressLatency is 0
ptp4l[151.401]: config item eth1.egressLatency is 0
ptp4l[151.401]: config item eth1.delay_mechanism is 0
ptp4l[151.402]: config item eth1.fault_badpeernet_interval is 16
ptp4l[151.403]: config item eth1.fault_reset_interval is 4
ptp4l[151.403]: config item eth1.tsproc_mode is 0
ptp4l[151.403]: config item eth1.delay_filter is 1
ptp4l[151.403]: config item eth1.delay_filter_length is 10
ptp4l[151.404]: config item eth1.logMinDelayReqInterval is 0
ptp4l[151.405]: config item eth1.logAnnounceInterval is 1
ptp4l[151.405]: config item eth1.announceReceiptTimeout is 3
ptp4l[151.405]: config item eth1.syncReceiptTimeout is 0
ptp4l[151.405]: config item eth1.transportSpecific is 0
ptp4l[151.405]: config item eth1.logSyncInterval is 0
ptp4l[151.406]: config item eth1.logMinPdelayReqInterval is 0
ptp4l[151.406]: config item eth1.neighborPropDelayThresh is 20000000
ptp4l[151.406]: config item eth1.min_neighbor_prop_delay is -20000000
ptp4l[151.406]: config item eth1.udp_ttl is 1
ptp4l[151.409]: driver changed our HWTSTAMP options
ptp4l[151.410]: tx_type   1 not 1
ptp4l[151.411]: rx_filter 1 not 12
ptp4l[151.411]: config item (null).dscp_event is 0
ptp4l[151.412]: config item (null).dscp_general is 0
ptp4l[151.412]: port 1: INITIALIZING to LISTENING on INITIALIZE
ptp4l[151.413]: config item /var/run/ptp4l.logMinDelayReqInterval is 0
ptp4l[151.413]: config item /var/run/ptp4l.logAnnounceInterval is 1
ptp4l[151.414]: config item /var/run/ptp4l.announceReceiptTimeout is 0
ptp4l[151.414]: config item /var/run/ptp4l.syncReceiptTimeout is 0
ptp4l[151.414]: config item /var/run/ptp4l.transportSpecific is 0
ptp4l[151.414]: config item /var/run/ptp4l.logSyncInterval is 0
ptp4l[151.414]: config item /var/run/ptp4l.logMinPdelayReqInterval is 0
ptp4l[151.415]: config item /var/run/ptp4l.neighborPropDelayThresh is 20000000
ptp4l[151.415]: config item /var/run/ptp4l.min_neighbor_prop_delay is -20000000
ptp4l[151.416]: config item (null).uds_address is '/var/run/ptp4l'
ptp4l[151.416]: port 0: INITIALIZING to LISTENING on INITIALIZE
ptp4l[151.418]: interface index 1 is up
ptp4l[151.419]: interface index 2 is down
ptp4l[151.419]: interface index 3 is down
ptp4l[151.420]: interface index 4 is up
ptp4l[151.420]: port 1: link up
ptp4l[151.421]: interface index 5 is down
ptp4l[152.319]: port 1: setting asCapable
ptp4l[153.319]: port 1: new foreign master 1831bf.fffe.e0ee96-1
ptp4l[154.621]: port 0: setting asCapable
ptp4l[157.320]: selected best master clock 1831bf.fffe.e0ee96
ptp4l[157.321]: foreign master not using PTP timescale
ptp4l[157.322]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[158.404]: port 1: delay timeout
ptp4l[158.408]: delay   filtered     236138   raw     236138
ptp4l[159.321]: master offset -317886227951 s0 freq      +0 path delay    236138
ptp4l[160.101]: port 1: delay timeout
ptp4l[160.104]: delay   filtered     234415   raw     232693
ptp4l[160.321]: master offset -317886232397 s1 freq   -4444 path delay    234415
ptp4l[160.884]: port 1: delay timeout
ptp4l[161.321]: master offset      -3311 s2 freq   -7755 path delay    234415
ptp4l[161.322]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[162.322]: master offset      -1548 s2 freq   -6985 path delay    234415
ptp4l[163.120]: port 1: delay timeout
ptp4l[163.124]: delay   filtered     232693   raw     231386
ptp4l[163.322]: master offset         38 s2 freq   -5864 path delay    232693
ptp4l[163.936]: port 1: delay timeout
ptp4l[163.939]: delay   filtered     232039   raw      58862
ptp4l[164.322]: master offset       -114 s2 freq   -6004 path delay    232039
ptp4l[164.446]: port 1: delay timeout
ptp4l[164.449]: delay   filtered     231386   raw     230110
ptp4l[165.322]: master offset       -681 s2 freq   -6606 path delay    231386
ptp4l[166.322]: master offset       -374 s2 freq   -6503 path delay    231386
ptp4l[166.348]: port 1: delay timeout
ptp4l[166.351]: delay   filtered     231908   raw     232430
ptp4l[166.878]: port 1: delay timeout
ptp4l[166.882]: delay   filtered     232430   raw     233931
ptp4l[167.322]: master offset      -2300 s2 freq   -8541 path delay    232430
ptp4l[167.740]: port 1: delay timeout
ptp4l[167.743]: delay   filtered     232561   raw     232969
ptp4l[168.323]: master offset       -260 s2 freq   -7191 path delay    232561
ptp4l[168.480]: port 1: delay timeout
ptp4l[168.484]: delay   filtered     232693   raw     236010
ptp4l[169.323]: master offset       -725 s2 freq   -7734 path delay    232693
ptp4l[169.374]: port 1: delay timeout
ptp4l[169.377]: delay   filtered     232831   raw     233908
ptp4l[170.167]: port 1: delay timeout
ptp4l[170.170]: delay   filtered     232561   raw     228615
ptp4l[170.323]: master offset        958 s2 freq   -6269 path delay    232561
ptp4l[171.323]: master offset        284 s2 freq   -6655 path delay    232561
ptp4l[172.038]: port 1: delay timeout
ptp4l[172.042]: delay   filtered     232699   raw     233889
ptp4l[172.324]: master offset       -768 s2 freq   -7622 path delay    232699
ptp4l[172.358]: port 1: delay timeout
ptp4l[172.361]: delay   filtered     233429   raw     235046
ptp4l[173.324]: master offset       -366 s2 freq   -7450 path delay    233429
ptp4l[174.236]: port 1: delay timeout
ptp4l[174.239]: delay   filtered     233429   raw     229687
ptp4l[174.324]: master offset        830 s2 freq   -6364 path delay    233429
ptp4l[174.780]: port 1: delay timeout
ptp4l[174.783]: delay   filtered     233550   raw     233211
ptp4l[175.324]: master offset       -472 s2 freq   -7417 path delay    233550
ptp4l[175.842]: port 1: delay timeout
ptp4l[175.845]: delay   filtered     233550   raw     231953
ptp4l[176.088]: port 1: delay timeout
ptp4l[176.092]: delay   filtered     233550   raw     234050
ptp4l[176.238]: port 1: delay timeout
ptp4l[176.241]: delay   filtered     233550   raw     231870

3.2 Testing on device side - (Local clock Synchronization)

The previous step allows the device to receive and make the HW timestamp available to the local clock source or application.
phyc2sys utility works in conjunction with the ptp4l utility to synchronize the local
system clock with the grand master clock source.

root@imx6ul-var-dart:~# phc2sys -s /dev/ptp0 -w -l 6 -q -m
phc2sys[155.624]: Waiting for ptp4l...
phc2sys[156.626]: Waiting for ptp4l...
phc2sys[157.629]: Waiting for ptp4l...
phc2sys[158.631]: Waiting for ptp4l...
phc2sys[159.634]: Waiting for ptp4l...
phc2sys[160.636]: Waiting for ptp4l...
phc2sys[161.638]: Waiting for ptp4l...
phc2sys[162.640]: phc offset 145614395359 s0 freq      +0 delay   6000
phc2sys[163.641]: phc offset 145614395519 s1 freq    +160 delay   6666
phc2sys[164.642]: phc offset      -317 s2 freq    -157 delay   6000
phc2sys[165.643]: phc offset      -180 s2 freq    -115 delay   6000
phc2sys[166.643]: phc offset       -77 s2 freq     -66 delay   6000
phc2sys[167.644]: phc offset       -11 s2 freq     -23 delay   6000
phc2sys[168.644]: phc offset       380 s2 freq    +364 delay   6333
phc2sys[169.645]: phc offset      -384 s2 freq    -286 delay   6000
phc2sys[170.646]: phc offset       199 s2 freq    +182 delay   6333
phc2sys[171.646]: phc offset      -275 s2 freq    -232 delay   6000
phc2sys[172.647]: phc offset       -20 s2 freq     -60 delay   6000
phc2sys[173.648]: phc offset        61 s2 freq     +15 delay   6000
phc2sys[174.648]: phc offset       188 s2 freq    +161 delay   6333
phc2sys[175.649]: phc offset       133 s2 freq    +162 delay   6333
phc2sys[176.650]: phc offset      -222 s2 freq    -153 delay   6000
phc2sys[177.650]: phc offset      -153 s2 freq    -151 delay   6000
phc2sys[178.651]: phc offset       324 s2 freq    +280 delay   6333
phc2sys[179.652]: phc offset       -22 s2 freq     +32 delay   6333
phc2sys[180.652]: phc offset        20 s2 freq     +67 delay   6333
phc2sys[181.653]: phc offset       -46 s2 freq      +7 delay   6333
phc2sys[182.654]: phc offset       -27 s2 freq     +12 delay   6333
phc2sys[183.654]: phc offset      -392 s2 freq    -361 delay   6000
phc2sys[184.655]: phc offset        16 s2 freq     -70 delay   6000
phc2sys[185.656]: phc offset       103 s2 freq     +21 delay   6000
phc2sys[186.656]: phc offset        55 s2 freq      +4 delay   6000
phc2sys[187.657]: phc offset        38 s2 freq      +4 delay   6000
phc2sys[188.657]: phc offset        88 s2 freq     +65 delay   6000
phc2sys[189.658]: phc offset       -56 s2 freq     -52 delay   6000
phc2sys[190.659]: phc offset        37 s2 freq     +24 delay   6000
phc2sys[191.659]: phc offset       326 s2 freq    +324 delay   6333
phc2sys[192.660]: phc offset      -335 s2 freq    -239 delay   6000
phc2sys[193.661]: phc offset        71 s2 freq     +66 delay   6333
phc2sys[194.661]: phc offset       117 s2 freq    +133 delay   6333
phc2sys[195.662]: phc offset      -220 s2 freq    -168 delay   6000
phc2sys[196.662]: phc offset      -136 s2 freq    -150 delay   6000
phc2sys[197.663]: phc offset       181 s2 freq    +126 delay   6333
phc2sys[198.664]: phc offset       199 s2 freq    +198 delay   6333
phc2sys[199.664]: phc offset      -308 s2 freq    -249 delay   6000
phc2sys[200.665]: phc offset         5 s2 freq     -29 delay   6000
phc2sys[201.666]: phc offset       258 s2 freq    +226 delay   6333
phc2sys[202.666]: phc offset        83 s2 freq    +128 delay   6333
phc2sys[203.667]: phc offset         8 s2 freq     +78 delay   6333
phc2sys[204.667]: phc offset      -489 s2 freq    -416 delay   6000
phc2sys[205.668]: phc offset       328 s2 freq    +254 delay   6333
phc2sys[206.669]: phc offset        -1 s2 freq     +23 delay   6334
phc2sys[207.669]: phc offset      -307 s2 freq    -283 delay   6000
phc2sys[208.670]: phc offset       -27 s2 freq     -95 delay   6000
phc2sys[209.671]: phc offset        67 s2 freq      -9 delay   6000
phc2sys[210.671]: phc offset       459 s2 freq    +403 delay   6333
phc2sys[211.672]: phc offset      -326 s2 freq    -244 delay   6000
phc2sys[212.672]: phc offset      -110 s2 freq    -126 delay   6000
phc2sys[213.673]: phc offset       224 s2 freq    +175 delay   6333
phc2sys[214.674]: phc offset      -117 s2 freq     -99 delay   6000
phc2sys[215.674]: phc offset       -58 s2 freq     -75 delay   6000
phc2sys[216.675]: phc offset       450 s2 freq    +415 delay   6333
phc2sys[217.676]: phc offset        40 s2 freq    +140 delay   6333
phc2sys[218.676]: phc offset      -182 s2 freq     -70 delay   6333
phc2sys[219.677]: phc offset      -164 s2 freq    -106 delay   6333
root@imx6ul-var-dart:~#

4 HW Events

After kernel up, run commands:

$ ptp4l -A -4 -H -m -i eth1  &

$ echo 1 > /sys/class/ptp/ptp0/pps_enable