Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Glob #161

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Glob #161

wants to merge 8 commits into from

Conversation

alex-bo
Copy link

@alex-bo alex-bo commented Sep 29, 2020

Glob module for HDFS client. The source code was ported from Python 3.4 and tested.

@alex-bo
Copy link
Author

alex-bo commented Sep 29, 2020

I resolved all 2.7 compatibility issues. Would appreciate if somebody can point me at the right direction to deal with "Failed to establish a new connection: [Errno 111] Connection refused',))" failures. Looks like it has nothing to do with the source code but rather an environment problem.

Copy link
Owner

@mtth mtth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the slow response @alex-bo. Thank you for the PR, I think this feature would be useful to have. The connection error might be related to #168.

hdfs/glob.py Outdated
import fnmatch
import re

from hdfs import Client
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

from .client import Client



def glob(client, hdfs_path):
"""Return a list of paths matching a pathname pattern.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good doc!

hdfs/glob.py Outdated
Comment on lines 20 to 26
Sample usages:
.. code-block:: python
glob(client, './foo/bar/*')
glob(client, './foo/bar/file[0-9].txt')
glob(client, './foo/bar/file?.txt')
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unindent sample usage? Same for iglob below.

glob(client, './foo/bar/file?.txt')
"""
return list(iglob(client, hdfs_path))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

glob provides little value over iglob: it saves just the list call. What do you think about only exposing iglob, possibly renaming it to glob?

hdfs/glob.py Outdated
yield posixpath.join(dirname, name)


def glob1(client, dirname, pattern):
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefix this (and the other functions below) with an underscore?

@alex-bo
Copy link
Author

alex-bo commented Dec 5, 2021

Hey @mtth any opinion on my latest updates and comments? Are you good with the changes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants