Skip to content

[feat](oss) support oss-hdfs root policy #50678

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 15, 2025

Conversation

morningman
Copy link
Contributor

@morningman morningman commented May 7, 2025

What problem does this PR solve?

The root policy is a feature for aliyun OSS-HDFS, it will mapping a hdfs path to the oss path,
eg:
hdfs://my_root_policy_name/ -> oss://emr-dev-oss.cn-beijing.oss-dls.aliyuncs.com/

So when root policy is enabled, the location of a hive table will be hdfs://my_root_policy_name/xxx
instead of oss://emr-dev-oss.cn-beijing.oss-dls.aliyuncs.com/xxx

This PR covers this case by adding a new catalog property oss.root_policy.
Default is false, if set to true, the location like hdfs://my_root_policy_name/ will be treated separately
to fit the jindofs format.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented May 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman morningman added usercase Important user case type label dev/3.0.x labels May 7, 2025
@morningman morningman marked this pull request as ready for review May 14, 2025 02:47
@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 33949 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 668a4117f215bd942c1ef449584b65305804b65c, data reload: false

------ Round 1 ----------------------------------
q1	25999	5094	5046	5046
q2	2087	280	189	189
q3	10563	1230	680	680
q4	10232	991	525	525
q5	7747	2412	2371	2371
q6	183	160	130	130
q7	909	747	608	608
q8	9301	1249	1042	1042
q9	6783	5147	5135	5135
q10	6854	2330	1912	1912
q11	492	289	272	272
q12	354	352	215	215
q13	17784	3622	3125	3125
q14	233	222	218	218
q15	529	487	473	473
q16	428	438	387	387
q17	604	873	356	356
q18	7743	7290	7114	7114
q19	1481	947	548	548
q20	330	333	222	222
q21	3893	3341	2422	2422
q22	1021	1019	959	959
Total cold run time: 115550 ms
Total hot run time: 33949 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5079	5019	5023	5019
q2	226	322	231	231
q3	2161	2595	2269	2269
q4	1361	1773	1397	1397
q5	4569	4399	4363	4363
q6	209	168	124	124
q7	1961	1890	1743	1743
q8	2557	2585	2542	2542
q9	7155	7114	7246	7114
q10	3020	3118	2699	2699
q11	567	492	483	483
q12	655	751	593	593
q13	3487	3859	3242	3242
q14	281	291	268	268
q15	526	477	476	476
q16	429	511	456	456
q17	1140	1509	1420	1420
q18	7739	7494	7479	7479
q19	802	787	828	787
q20	2037	2061	1875	1875
q21	5259	4538	4529	4529
q22	1030	1024	977	977
Total cold run time: 52250 ms
Total hot run time: 50086 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 186850 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 668a4117f215bd942c1ef449584b65305804b65c, data reload: false

query1	1013	477	491	477
query2	6557	1784	1769	1769
query3	6742	231	211	211
query4	26201	23695	23472	23472
query5	4326	627	462	462
query6	300	207	213	207
query7	4626	495	291	291
query8	320	255	244	244
query9	8619	2655	2657	2655
query10	446	337	289	289
query11	15215	15102	15226	15102
query12	164	108	107	107
query13	1666	521	419	419
query14	9385	6256	6073	6073
query15	199	198	168	168
query16	7206	648	436	436
query17	1207	742	590	590
query18	1975	401	303	303
query19	190	188	162	162
query20	126	118	123	118
query21	215	126	107	107
query22	4099	4136	4060	4060
query23	33981	33145	33179	33145
query24	8513	2394	2373	2373
query25	559	449	392	392
query26	1236	261	152	152
query27	2753	506	343	343
query28	4346	2124	2095	2095
query29	786	589	428	428
query30	289	221	181	181
query31	948	850	740	740
query32	71	64	63	63
query33	552	362	319	319
query34	789	838	514	514
query35	781	805	744	744
query36	958	975	853	853
query37	112	98	85	85
query38	4199	4193	4159	4159
query39	1474	1379	1417	1379
query40	211	118	104	104
query41	70	52	51	51
query42	127	107	108	107
query43	496	486	475	475
query44	1287	813	812	812
query45	180	174	162	162
query46	842	1005	637	637
query47	1791	1782	1725	1725
query48	393	430	322	322
query49	764	522	446	446
query50	653	666	402	402
query51	4129	4211	4073	4073
query52	113	101	100	100
query53	222	251	181	181
query54	595	592	497	497
query55	83	85	87	85
query56	305	303	292	292
query57	1138	1163	1079	1079
query58	273	261	247	247
query59	2552	2583	2543	2543
query60	321	324	331	324
query61	138	125	124	124
query62	785	710	650	650
query63	229	184	188	184
query64	4367	1024	707	707
query65	4321	4227	4265	4227
query66	1140	406	311	311
query67	15783	15604	15463	15463
query68	8038	893	531	531
query69	464	317	264	264
query70	1214	1134	1095	1095
query71	464	325	295	295
query72	5641	4651	4679	4651
query73	727	602	353	353
query74	9152	9116	8918	8918
query75	3864	3208	2739	2739
query76	3655	1180	749	749
query77	788	416	298	298
query78	10073	10152	9362	9362
query79	2159	837	583	583
query80	587	512	458	458
query81	482	256	226	226
query82	472	124	96	96
query83	254	261	271	261
query84	254	97	83	83
query85	795	355	310	310
query86	385	298	280	280
query87	4453	4469	4303	4303
query88	3660	2285	2291	2285
query89	390	311	275	275
query90	1834	212	208	208
query91	145	147	113	113
query92	85	60	55	55
query93	1630	925	586	586
query94	688	413	297	297
query95	377	293	284	284
query96	498	569	282	282
query97	3156	3198	3114	3114
query98	251	204	200	200
query99	1649	1403	1279	1279
Total cold run time: 274736 ms
Total hot run time: 186850 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.71 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 668a4117f215bd942c1ef449584b65305804b65c, data reload: false

query1	0.03	0.04	0.03
query2	0.11	0.11	0.10
query3	0.24	0.20	0.19
query4	1.59	0.18	0.19
query5	0.61	0.59	0.60
query6	1.19	0.72	0.72
query7	0.02	0.02	0.01
query8	0.05	0.04	0.03
query9	0.57	0.54	0.51
query10	0.57	0.57	0.57
query11	0.16	0.11	0.11
query12	0.15	0.12	0.12
query13	0.62	0.59	0.61
query14	0.80	0.81	0.81
query15	0.88	0.84	0.85
query16	0.38	0.38	0.39
query17	1.03	1.03	1.04
query18	0.23	0.21	0.22
query19	1.91	1.91	1.94
query20	0.01	0.01	0.02
query21	15.40	0.92	0.55
query22	0.76	1.14	0.64
query23	15.02	1.39	0.66
query24	7.04	1.94	1.04
query25	0.50	0.13	0.21
query26	0.67	0.16	0.13
query27	0.05	0.05	0.05
query28	10.08	0.90	0.47
query29	12.58	4.00	3.32
query30	0.25	0.09	0.07
query31	2.82	0.60	0.39
query32	3.24	0.55	0.47
query33	3.05	3.06	3.16
query34	15.76	5.10	4.51
query35	4.51	4.49	4.49
query36	0.66	0.50	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.02 s
Total hot run time: 29.71 s

@morningman
Copy link
Contributor Author

run buildall

@morningman
Copy link
Contributor Author

run buildall

@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34901 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit aa1b8e0cafc9ea3a511c1e004ef50f034ab354d0, data reload: false

------ Round 1 ----------------------------------
q1	26088	5467	5417	5417
q2	2143	291	203	203
q3	10350	1232	683	683
q4	10219	1002	520	520
q5	7491	2302	2338	2302
q6	183	164	130	130
q7	911	740	624	624
q8	9332	1315	1052	1052
q9	6781	5047	5093	5047
q10	7060	2313	1945	1945
q11	483	288	268	268
q12	346	348	215	215
q13	17790	3681	3149	3149
q14	225	225	208	208
q15	532	491	494	491
q16	429	418	381	381
q17	589	845	360	360
q18	7647	7319	7210	7210
q19	1365	942	570	570
q20	331	332	221	221
q21	3846	3188	2949	2949
q22	1045	1026	956	956
Total cold run time: 115186 ms
Total hot run time: 34901 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5576	5490	5510	5490
q2	232	336	233	233
q3	2139	2632	2274	2274
q4	1324	1836	1468	1468
q5	4472	4390	4352	4352
q6	222	173	129	129
q7	1992	1942	1762	1762
q8	2563	2491	2608	2491
q9	7272	7068	7214	7068
q10	3040	3212	2789	2789
q11	570	515	478	478
q12	684	744	624	624
q13	3500	3918	3363	3363
q14	295	296	278	278
q15	551	516	505	505
q16	462	502	455	455
q17	1150	1473	1388	1388
q18	7655	7638	7592	7592
q19	810	822	900	822
q20	1979	1997	1830	1830
q21	4961	4606	4577	4577
q22	1128	1103	1036	1036
Total cold run time: 52577 ms
Total hot run time: 51004 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 193204 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit aa1b8e0cafc9ea3a511c1e004ef50f034ab354d0, data reload: false

query1	1406	1125	1069	1069
query2	6432	1812	1786	1786
query3	11153	4588	4650	4588
query4	25675	23669	22965	22965
query5	4264	627	470	470
query6	315	215	210	210
query7	3995	495	291	291
query8	307	256	233	233
query9	8516	2669	2686	2669
query10	470	321	279	279
query11	15429	15178	14812	14812
query12	174	111	120	111
query13	1561	551	426	426
query14	9434	6321	6288	6288
query15	210	187	173	173
query16	7595	643	480	480
query17	1368	816	638	638
query18	2034	412	320	320
query19	241	194	173	173
query20	130	162	124	124
query21	208	126	109	109
query22	4439	4428	4353	4353
query23	34524	33765	33558	33558
query24	8436	2468	2411	2411
query25	516	472	401	401
query26	1176	272	150	150
query27	2883	509	357	357
query28	4732	2216	2217	2216
query29	685	586	456	456
query30	293	221	189	189
query31	896	871	788	788
query32	73	66	67	66
query33	548	363	323	323
query34	821	878	555	555
query35	812	869	757	757
query36	958	983	883	883
query37	118	109	83	83
query38	4183	4305	4167	4167
query39	1528	1464	1456	1456
query40	209	117	146	117
query41	57	58	57	57
query42	125	107	117	107
query43	503	508	480	480
query44	1394	847	839	839
query45	181	181	170	170
query46	863	1022	654	654
query47	1940	1933	1787	1787
query48	417	445	325	325
query49	768	527	438	438
query50	680	705	415	415
query51	4263	4216	4193	4193
query52	116	110	100	100
query53	225	268	187	187
query54	608	589	540	540
query55	84	90	85	85
query56	312	317	302	302
query57	1164	1196	1135	1135
query58	269	262	260	260
query59	2682	2739	2539	2539
query60	345	338	293	293
query61	129	126	123	123
query62	799	781	682	682
query63	230	188	191	188
query64	4440	1054	687	687
query65	4469	4349	4386	4349
query66	978	401	311	311
query67	16402	15603	15406	15406
query68	8905	895	525	525
query69	491	305	274	274
query70	1156	1101	1055	1055
query71	480	319	308	308
query72	5656	4711	4734	4711
query73	715	617	361	361
query74	9166	8839	8741	8741
query75	4215	3260	2793	2793
query76	3755	1216	788	788
query77	789	388	287	287
query78	10147	10244	9481	9481
query79	2273	836	597	597
query80	698	522	447	447
query81	501	249	218	218
query82	474	170	100	100
query83	280	255	241	241
query84	307	109	91	91
query85	801	345	303	303
query86	380	302	272	272
query87	4540	4466	4379	4379
query88	3606	2315	2314	2314
query89	409	318	282	282
query90	1854	209	212	209
query91	146	145	116	116
query92	75	73	54	54
query93	1778	944	585	585
query94	659	411	316	316
query95	379	305	285	285
query96	506	577	282	282
query97	3149	3221	3112	3112
query98	235	212	213	212
query99	1455	1396	1313	1313
Total cold run time: 283170 ms
Total hot run time: 193204 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.05 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit aa1b8e0cafc9ea3a511c1e004ef50f034ab354d0, data reload: false

query1	0.04	0.04	0.03
query2	0.11	0.11	0.11
query3	0.25	0.19	0.19
query4	1.60	0.19	0.20
query5	0.59	0.58	0.60
query6	1.17	0.71	0.74
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.58	0.52	0.51
query10	0.56	0.57	0.57
query11	0.16	0.11	0.11
query12	0.14	0.12	0.11
query13	0.62	0.60	0.59
query14	0.79	0.81	0.81
query15	0.87	0.85	0.85
query16	0.42	0.38	0.39
query17	1.02	1.00	1.04
query18	0.23	0.21	0.21
query19	1.90	1.84	1.81
query20	0.01	0.01	0.01
query21	15.40	0.91	0.55
query22	0.77	1.19	0.74
query23	14.85	1.39	0.64
query24	6.69	1.89	0.38
query25	0.27	0.22	0.06
query26	0.66	0.15	0.14
query27	0.05	0.05	0.04
query28	8.86	0.91	0.46
query29	12.61	4.01	3.40
query30	0.26	0.09	0.07
query31	2.84	0.60	0.39
query32	3.23	0.55	0.47
query33	3.01	3.01	3.12
query34	15.81	5.16	4.50
query35	4.53	4.56	4.57
query36	0.67	0.50	0.48
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.16	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 102.11 s
Total hot run time: 29.05 s

@CalvinKirs CalvinKirs self-assigned this May 15, 2025
Copy link
Contributor

PR approved by anyone and no changes requested.

@CalvinKirs CalvinKirs added the Parameter Refactor https://github.com/apache/doris/issues/50238 label May 15, 2025
@morningman
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 34009 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f1152e681390c904bb5e797d08d90cce5ff8b802, data reload: false

------ Round 1 ----------------------------------
q1	26312	5098	4997	4997
q2	2124	279	190	190
q3	10384	1220	713	713
q4	10222	1033	501	501
q5	7530	2404	2352	2352
q6	178	164	131	131
q7	924	740	618	618
q8	9321	1307	1157	1157
q9	6787	5221	5319	5221
q10	6822	2328	1889	1889
q11	490	278	291	278
q12	361	359	213	213
q13	17782	3647	3064	3064
q14	217	224	218	218
q15	547	493	478	478
q16	425	428	375	375
q17	604	857	358	358
q18	7672	7135	7158	7135
q19	1596	959	551	551
q20	328	331	230	230
q21	3658	3194	2369	2369
q22	1060	1025	971	971
Total cold run time: 115344 ms
Total hot run time: 34009 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5183	5089	5087	5087
q2	237	327	230	230
q3	2175	2650	2245	2245
q4	1321	1750	1357	1357
q5	4599	4446	4400	4400
q6	215	166	126	126
q7	1986	1922	1743	1743
q8	2592	2554	2482	2482
q9	7182	7007	7160	7007
q10	3024	3157	2800	2800
q11	573	508	470	470
q12	708	762	590	590
q13	3485	3901	3357	3357
q14	267	300	258	258
q15	519	473	478	473
q16	426	497	458	458
q17	1188	1568	1402	1402
q18	7644	7501	7384	7384
q19	792	867	1026	867
q20	1932	2048	1854	1854
q21	4765	4332	4186	4186
q22	1058	1041	1016	1016
Total cold run time: 51871 ms
Total hot run time: 49792 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 187845 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f1152e681390c904bb5e797d08d90cce5ff8b802, data reload: false

query1	1033	482	490	482
query2	6581	1869	1899	1869
query3	6768	230	232	230
query4	26203	23569	23212	23212
query5	4371	619	479	479
query6	319	218	209	209
query7	4632	515	289	289
query8	326	296	275	275
query9	8621	2625	2609	2609
query10	471	328	285	285
query11	15264	15313	14901	14901
query12	158	124	120	120
query13	1679	557	436	436
query14	8877	6084	6136	6084
query15	215	187	173	173
query16	7137	635	491	491
query17	1230	744	598	598
query18	1978	441	316	316
query19	202	189	161	161
query20	122	116	118	116
query21	218	135	109	109
query22	4049	4125	4200	4125
query23	34028	33192	33052	33052
query24	8464	2428	2383	2383
query25	573	493	436	436
query26	1226	274	157	157
query27	2761	517	337	337
query28	4407	2119	2092	2092
query29	839	610	479	479
query30	286	222	191	191
query31	928	908	780	780
query32	79	72	66	66
query33	576	383	334	334
query34	817	852	519	519
query35	780	814	790	790
query36	947	960	867	867
query37	120	111	81	81
query38	4206	4206	4098	4098
query39	1683	1439	1411	1411
query40	222	125	113	113
query41	76	75	73	73
query42	130	113	110	110
query43	510	495	463	463
query44	1277	817	808	808
query45	181	177	173	173
query46	842	1027	649	649
query47	1767	1779	1751	1751
query48	388	424	327	327
query49	820	560	480	480
query50	676	707	420	420
query51	4116	4116	4071	4071
query52	113	116	102	102
query53	230	255	192	192
query54	589	588	519	519
query55	85	84	88	84
query56	346	351	328	328
query57	1176	1172	1119	1119
query58	290	286	280	280
query59	2652	2842	2659	2659
query60	362	359	342	342
query61	182	178	176	176
query62	813	747	682	682
query63	234	199	193	193
query64	4669	1303	849	849
query65	4295	4224	4271	4224
query66	1178	438	349	349
query67	15970	15524	15464	15464
query68	8333	894	528	528
query69	482	324	282	282
query70	1252	1103	1095	1095
query71	469	322	308	308
query72	6026	4997	5087	4997
query73	714	620	353	353
query74	8922	9197	8978	8978
query75	3874	3257	2723	2723
query76	3657	1199	764	764
query77	791	399	297	297
query78	9987	10186	9348	9348
query79	2503	819	589	589
query80	645	560	502	502
query81	465	260	226	226
query82	456	134	103	103
query83	303	270	252	252
query84	301	115	94	94
query85	895	440	402	402
query86	341	310	294	294
query87	4483	4528	4327	4327
query88	3013	2294	2324	2294
query89	407	321	293	293
query90	1928	226	220	220
query91	161	212	132	132
query92	86	68	59	59
query93	1168	950	606	606
query94	696	419	323	323
query95	398	310	306	306
query96	500	572	289	289
query97	2718	2788	2716	2716
query98	247	210	205	205
query99	1479	1409	1298	1298
Total cold run time: 275063 ms
Total hot run time: 187845 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.11 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f1152e681390c904bb5e797d08d90cce5ff8b802, data reload: false

query1	0.04	0.03	0.04
query2	0.12	0.10	0.10
query3	0.26	0.20	0.19
query4	1.59	0.20	0.19
query5	0.45	0.46	0.44
query6	1.53	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.57	0.51	0.51
query10	0.58	0.58	0.57
query11	0.16	0.10	0.11
query12	0.15	0.12	0.12
query13	0.61	0.59	0.59
query14	0.78	0.82	0.81
query15	0.87	0.85	0.87
query16	0.39	0.37	0.38
query17	1.02	1.00	1.03
query18	0.22	0.21	0.22
query19	1.86	1.80	1.83
query20	0.02	0.01	0.01
query21	15.44	0.90	0.55
query22	0.74	1.14	0.71
query23	14.93	1.37	0.61
query24	6.79	1.81	0.85
query25	0.49	0.15	0.13
query26	0.65	0.16	0.15
query27	0.05	0.04	0.04
query28	9.66	0.95	0.44
query29	12.57	4.02	3.32
query30	0.26	0.09	0.07
query31	2.82	0.59	0.38
query32	3.23	0.55	0.48
query33	3.09	3.05	3.12
query34	15.84	5.12	4.46
query35	4.51	4.55	4.47
query36	0.67	0.50	0.47
query37	0.08	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.14	0.14
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.04	0.03	0.02
Total cold run time: 103.5 s
Total hot run time: 29.11 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 15, 2025
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@morningman morningman merged commit 727161b into apache:master May 15, 2025
25 of 26 checks passed
github-actions bot pushed a commit that referenced this pull request May 15, 2025
### What problem does this PR solve?

The root policy is a feature for aliyun OSS-HDFS, it will mapping a hdfs
path to the oss path,
eg:
`hdfs://my_root_policy_name/` ->
`oss://emr-dev-oss.cn-beijing.oss-dls.aliyuncs.com/`

So when root policy is enabled, the location of a hive table will be
`hdfs://my_root_policy_name/xxx`
instead of `oss://emr-dev-oss.cn-beijing.oss-dls.aliyuncs.com/xxx`

This PR covers this case by adding a new catalog property
`oss.root_policy`.
Default is false, if set to true, the location like
`hdfs://my_root_policy_name/` will be treated separately
to fit the jindofs format.
dataroaring pushed a commit that referenced this pull request May 17, 2025
Cherry-picked from #50678

Co-authored-by: Mingyu Chen (Rayner) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.6-merged Parameter Refactor https://github.com/apache/doris/issues/50238 reviewed usercase Important user case type label
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants