[Observability] Collect request timestamps to observe the overhead introduced by system

[s5u13b]

No description provided.

[KuilongCui]

RequestStatistics collect should be a choice

[github-actions]

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	168.00 MB	176.00 MB	184.00 MB	192.00 MB	200.00 MB	208.00 MB	224.00 MB	232.00 MB	240.00 MB	312.00 MB	416.00 MB	480.00 MB	488.00 MB	536.00 MB
rpc_speed(GB/s)	1.04	1.53	1.79	1.91	2.03	2.09	2.13	2.21	2.21	2.28	2.31	2.34	2.47	2.49	2.51	2.52	2.49	2.61	2.59	2.48	2.62	2.53	2.64	2.45	2.54	2.57	2.53	2.48	2.50	2.78	3.07	3.03	3.02	3.16

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	168.00 MB	176.00 MB	184.00 MB	192.00 MB	200.00 MB	208.00 MB	232.00 MB	280.00 MB	312.00 MB	320.00 MB	344.00 MB	416.00 MB	440.00 MB	480.00 MB	488.00 MB	536.00 MB	544.00 MB	560.00 MB	912.00 MB
gloo_speed(GB/s)	0.93	1.58	1.99	2.22	2.35	2.60	2.69	2.61	2.66	2.84	2.86	2.63	3.13	2.84	2.71	2.80	2.65	2.05	2.53	2.35	1.21	2.16	2.26	2.49	2.65	3.27	2.64	2.76	2.00	2.48	2.72	2.51	1.77	2.14	2.72	2.81	2.77	2.55	2.47

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	168.00 MB	176.00 MB	184.00 MB	192.00 MB	200.00 MB	224.00 MB	232.00 MB	280.00 MB	288.00 MB	312.00 MB	416.00 MB	424.00 MB	432.00 MB	440.00 MB	456.00 MB	480.00 MB	488.00 MB	536.00 MB	560.00 MB
nccl_speed(GB/s)	0.18	0.47	0.70	0.94	1.16	1.31	1.30	1.65	1.58	2.07	2.12	2.22	2.30	2.37	2.55	2.69	2.91	3.05	3.97	3.15	3.44	3.06	3.70	3.38	3.95	4.61	5.22	2.14	2.72	3.55	6.15	5.44	5.63	6.75	6.01	5.15	5.69	4.69	4.27

[github-actions]

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	144.00 MB	152.00 MB	160.00 MB	176.00 MB	184.00 MB	192.00 MB	200.00 MB	208.00 MB	312.00 MB	328.00 MB	472.00 MB	536.00 MB	560.00 MB	736.00 MB
rpc_speed(GB/s)	1.03	1.50	1.80	1.88	2.02	2.04	2.10	2.18	2.20	2.24	2.38	2.29	2.36	2.32	2.44	2.47	2.52	2.65	2.59	2.48	2.40	2.46	2.46	2.64	2.61	2.70	3.01	3.10	3.28	3.30

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	144.00 MB	160.00 MB	168.00 MB	176.00 MB	192.00 MB	312.00 MB	472.00 MB	560.00 MB	688.00 MB
gloo_speed(GB/s)	0.89	1.60	2.03	2.21	2.39	2.46	2.65	2.54	2.81	2.90	2.74	3.13	3.25	3.43	3.33	2.51	2.62	2.37	3.28	2.47	2.21	2.09	2.42	2.61	2.04

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	168.00 MB	176.00 MB	192.00 MB	312.00 MB	320.00 MB	472.00 MB	560.00 MB	568.00 MB	688.00 MB	728.00 MB
nccl_speed(GB/s)	0.21	0.51	0.82	0.96	1.12	1.32	1.34	1.49	1.54	1.80	2.16	2.34	1.91	2.51	2.15	2.79	1.94	2.98	4.15	3.45	3.12	3.88	4.62	4.83	3.42	4.01	4.30	4.14	4.11	5.19

[s5u13b]

RequestStatistics collect should be a choice

Get

[github-actions]

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	152.00 MB	160.00 MB	168.00 MB	176.00 MB	192.00 MB	280.00 MB	296.00 MB	312.00 MB	368.00 MB	472.00 MB	536.00 MB	560.00 MB	592.00 MB	688.00 MB
rpc_speed(GB/s)	1.06	1.60	1.79	2.00	2.10	2.18	2.15	2.16	2.24	2.33	2.41	2.36	2.38	2.41	2.59	2.45	2.45	2.43	2.45	2.49	2.57	2.42	2.63	2.05	2.70	2.54	3.22	3.40	3.00	3.21	3.25

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	176.00 MB	192.00 MB	280.00 MB	312.00 MB	472.00 MB	480.00 MB	536.00 MB	552.00 MB	592.00 MB	688.00 MB
gloo_speed(GB/s)	0.93	1.60	1.87	2.18	2.19	2.30	2.47	2.05	2.81	2.89	2.65	2.65	3.01	3.46	3.55	2.07	2.35	2.81	3.56	2.61	1.73	1.65	2.54	2.29	2.26	2.64	2.58	2.69	2.68	2.15

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	168.00 MB	192.00 MB	312.00 MB	472.00 MB	480.00 MB	536.00 MB	560.00 MB	688.00 MB
nccl_speed(GB/s)	0.20	0.51	0.83	0.91	1.29	1.36	1.44	1.46	1.71	2.00	1.69	1.88	1.49	2.34	2.50	1.93	3.03	2.98	2.77	3.96	3.16	4.40	4.45	5.82	6.92	3.58	5.56	3.81

[github-actions]

prefill	p25	p50	p75	p95	p99	mean
latency(ms)	15506.21	84160.65	217780.86	236725.72	250184.46	114550.27

decode	p25	p50	p75	p95	p99	mean
latency(ms)	51.89	57.88	70.95	117.01	286.86	69.69

[github-actions]

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	176.00 MB	192.00 MB	200.00 MB	280.00 MB	312.00 MB	472.00 MB	480.00 MB	568.00 MB	576.00 MB	584.00 MB	688.00 MB
rpc_speed(GB/s)	1.04	1.55	1.75	1.93	1.98	2.07	2.13	2.14	2.21	2.26	2.41	2.55	2.35	2.38	2.40	2.48	2.59	2.48	2.54	2.38	2.38	2.41	2.40	2.66	2.52	3.12	3.19	2.98	3.02	3.13	2.97

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	168.00 MB	192.00 MB	280.00 MB	288.00 MB	312.00 MB	472.00 MB	536.00 MB	544.00 MB	688.00 MB
gloo_speed(GB/s)	0.96	1.68	2.04	2.13	2.34	2.77	2.90	2.77	2.68	2.95	3.06	2.59	3.24	3.13	3.20	2.69	2.91	2.30	2.27	0.96	2.37	2.59	2.61	2.50	2.59	2.88	2.25	2.55	2.56

migration_size	8.00 MB	16.00 MB	24.00 MB	32.00 MB	40.00 MB	48.00 MB	56.00 MB	64.00 MB	72.00 MB	80.00 MB	88.00 MB	96.00 MB	104.00 MB	112.00 MB	120.00 MB	128.00 MB	136.00 MB	144.00 MB	152.00 MB	160.00 MB	168.00 MB	176.00 MB	192.00 MB	280.00 MB	312.00 MB	320.00 MB	480.00 MB	536.00 MB	560.00 MB	576.00 MB
nccl_speed(GB/s)	0.18	0.43	0.68	0.89	1.14	1.42	1.62	1.33	1.93	1.82	2.02	1.85	2.02	0.77	2.05	2.63	3.88	3.05	3.02	2.94	3.09	2.83	5.06	2.92	4.38	4.69	5.63	3.90	3.77	4.88