Support writes to mounted local path with blobfuse

[RobinLin666]

Description

Use Case
Re-open an issue for tracking #1418 (comment). Thank you.

import pandas as pd
from deltalake.writer import write_deltalake

df = pd.DataFrame(
    {
        "a": [1, 2, 3, 4, 5],
        "fruits": ["banana", "orange", "mango", "apple", "banana"],
    }
)

write_deltalake('/lakehouse/default/Files/fruits', df,mode='overwrite')

Related Issue(s)
#1418 (comment)
GlareDB/glaredb#1809

[RobinLin666]

Hi @scsmithr , I tried to solve the rename issue, but got another issue when write to mounted path. Could you please help to check?

from deltalake import DeltaTable, write_deltalake
import pandas as pd
df = pd.DataFrame({"id": [1, 2], "value": ["foo", "boo"]})
write_deltalake("/synfs/test/delta/data/delta3", df)

[scsmithr]

For what it's worth, I was testing with this change to object store: https://github.com/GlareDB/arrow-rs/pull/2/files

The first diff chunk explicitly drops the file so that the metadata gets flushed to blobstore before the rename.

The second chunk I'm not sure is necessary. I was just trying out explicitly syncing the file, but I don't recall it actually fixing the issue.

[RobinLin666]

Thanks @scsmithr, I tried to mitigate the issue from blobfuse side.

for #1 rename issue can be mitigated. I upload the tmp file before renaming.

for #2 you said blobfuse doesn't support hard linking, copy_if_not_exists will fail. I don't quite understand how this aspect is affected and whether it is the cause of the error I encountered above. OSError: Generic DeltaLocalObjectStore error: Function not implemented (os error 38) If possible, please let me know where it need a hard link and we can think about any workarounds available.

I saw the same error when writing to DBFS, perhaps it's one of the reasons?
#987