fix duration conversion error

[Liyixin95]

Which issue does this PR close?

Closes #5625

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

[tustvold]

Could we get a test of this please

[alamb]

I agree -- a test is important to make sure we don't break this fix in some future refactor

[Liyixin95]

Could we get a test of this please

sorry for the noob question, where should I add a test for it？

[alamb]

Could we get a test of this please

sorry for the noob question, where should I add a test for it？

I think we could add a round trip test here perhaps:

arrow-rs/parquet/src/arrow/arrow_writer/mod.rs

Lines 1996 to 2007 in 91f0b17

	#[test]
	fn date32_single_column() {
	required_and_optional::<Date32Array, _>(0..SMALL_SIZE as i32);
	}
	#[test]
	fn date64_single_column() {
	// Date64 must be a multiple of 86400000, see ARROW-10925
	required_and_optional::<Date64Array, _>(
	(0..(SMALL_SIZE as i64 * 86400000)).step_by(86400000),
	);
	}

[Liyixin95]

Could we get a test of this please

sorry for the noob question, where should I add a test for it？

I think we could add a round trip test here perhaps:

arrow-rs/parquet/src/arrow/arrow_writer/mod.rs

Lines 1996 to 2007 in 91f0b17

	#[test]
	fn date32_single_column() {
	required_and_optional::<Date32Array, _>(0..SMALL_SIZE as i32);
	}
	#[test]
	fn date64_single_column() {
	// Date64 must be a multiple of 86400000, see ARROW-10925
	required_and_optional::<Date64Array, _>(
	(0..(SMALL_SIZE as i64 * 86400000)).step_by(86400000),
	);
	}

write duration to parquet get a panic...

arrow-rs/parquet/src/arrow/schema/mod.rs

Line 445 in a999fb8

DataType::Duration(_) => Err(arrow_err!("Converting Duration to parquet not supported",)),

[tustvold]

write duration to parquet get a panic...

Yes I think we would need #1938. Technically polars is doing something a little funky here

[alamb]

or put another way there is no way to write a duration to a parquet file with arrow-rs yet (but polars can do so).

So the ideal solution is support writing duration to parquet too (but I suspect there might be work to sort out what the desired behavior is)

Perhaps for this PR we can just check in a small parquet file that was written by polars and ensure we can read it back correctly 🤔

[tustvold]

Closing as this PR has been inactive for a while

[EddyXorb]

I have the same problem with type pl.Time. When I write a column with pl.Time to a parquet file, it reads the column as int64.

[alamb]

Thanks @EddyXorb -- any chance you can make a PR with a test? It seems that that is all that is missing to be able to get this fixed

[EddyXorb]

I would love to help, but at the moment I am very very busy. Once I have time (mid december), I can try to provide a test.

[alamb]

Thank you!