Inference streaming support #3611

Job	Run time
runtimes (ubuntu-latest, 3.9, sklearn, true)	2m 13s
runtimes (ubuntu-latest, 3.9, xgboost, true)	3m 42s
lint (3.9)	50s
runtimes (ubuntu-latest, 3.9, lightgbm, true)	1m 50s
all-runtimes	0s
lint (3.10)	53s
runtimes (ubuntu-latest, 3.9, mlflow, true)	6m 8s
generate (3.9, dataplane)	42s
runtimes (ubuntu-latest, 3.9, huggingface, true)	5m 31s
generate (3.9, model-repository)	41s
generate (3.10, dataplane)	36s
mlserver (ubuntu-latest, 3.9, true)	16m 13s
runtimes (ubuntu-latest, 3.9, alibi-explain, true)	5m 46s
mlserver (ubuntu-latest, 3.10, true)	16m 30s
generate (3.10, model-repository)	31s
runtimes (ubuntu-latest, 3.9, alibi-detect, true)	3m 33s
runtimes (ubuntu-latest, 3.9, catboost, true)	2m 31s
runtimes (ubuntu-latest, 3.10, sklearn, true)	1m 50s
runtimes (ubuntu-latest, 3.10, xgboost, true)	3m 13s
runtimes (ubuntu-latest, 3.10, lightgbm, true)	2m 1s
runtimes (ubuntu-latest, 3.10, mlflow, true)	5m 42s
runtimes (ubuntu-latest, 3.10, huggingface, true)	5m 30s
runtimes (ubuntu-latest, 3.10, alibi-explain, true)	5m 40s
runtimes (ubuntu-latest, 3.10, alibi-detect, true)	3m 46s
runtimes (ubuntu-latest, 3.10, catboost, true)	2m 2s
	1h 37m 54s

Provide feedback