Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with unicode radula output to pipe #13

Open
mustafa1024m opened this issue Jan 7, 2018 · 6 comments
Open

Problem with unicode radula output to pipe #13

mustafa1024m opened this issue Jan 7, 2018 · 6 comments

Comments

@mustafa1024m
Copy link

Using radula ls with a bucket that contains Unicode named objects works fine, but directing it to a pipe results in an error:

radula ls testbucket

works fine, while this doesn't:

radula ls testbucket | grep teststring

Traceback (most recent call last):
File "/usr/bin/radula", line 11, in
sys.exit(main())
File "/usr/lib/python2.7/site-packages/radula/init.py", line 273, in main
_real_main()
File "/usr/lib/python2.7/site-packages/radula/init.py", line 83, in _real_main
getattr(radu, command)(**vars(args))
File "/usr/lib/python2.7/site-packages/radula/proxy.py", line 139, in ls
return self.keys(**kwargs)
File "/usr/lib/python2.7/site-packages/radula/proxy.py", line 153, in keys
print key
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-4: ordinal not in range(128)

@mustafa1024m
Copy link
Author

I could get rid of this error by adding:

reload(sys)
sys.setdefaultencoding('utf8')

to /usr/bin/radula

Now the listing works find, but still can't put a file named "testfile[bracketstest]"

radula put "testfile\[bracketstest\]" mustafa-test/
Error: No file(s) to upload: used testfile\[bracketstest\]

@mustafa1024m
Copy link
Author

Sorry, closed by mistake

@mustafa1024m mustafa1024m reopened this Jan 7, 2018
@mustafa1024m
Copy link
Author

Also without escaping:

radula put "testfile[bracketstest]" mustafa-test/
Error: No file(s) to upload: used testfile[bracketstest]

@mustafa1024m
Copy link
Author

I found this approach highly discouraged:
https://stackoverflow.com/questions/3828723/why-should-we-not-use-sys-setdefaultencodingutf-8-in-a-py-script

Instead, setting environment variable PYTHONIOENCODING="UTF-8" fixes the problems

@bibby
Copy link
Owner

bibby commented Jan 8, 2018

not for nothing, but radula appears to work ok with Japanese characters ok.

$ radula -p abibby up "そうかな" abibby/toss/
2018-01-08 16:31:29,268 INFO:radula: CHUNKS: 1 parts x 100.00 MB
2018-01-08 16:31:30,375 INFO:radula: Finished uploading 9.00 B in 0.44s (20.49 Bps)
2018-01-08 16:31:30,376 INFO:radula: Download URL: https://s3-v2/abibby/toss/%E3%81%9D%E3%81%86%E3%81%8B%E3%81%AA

I think what you've hit is an accidental fnmatch pattern. Characters within brackets will match a single character among the set.

One workaround is to use another fnmatch pattern (the ? char) to avoid typing a bracket ; at least until a better solution reveals itself.

$ radula -p abibby up "brackets?test?" abibby/toss/
2018-01-08 16:35:54,164 INFO:radula: CHUNKS: 1 parts x 100.00 MB
2018-01-08 16:35:55,236 INFO:radula: Finished uploading 10.00 B in 0.38s (26.48 Bps)
2018-01-08 16:35:55,236 INFO:radula: Download URL: https://s3-v2/abibby/toss/brackets%5Btest%5D

@mustafa1024m
Copy link
Author

Thanks
exporting PYTHONIOENCODING="UTF-8" fixed the problem with some characters, but the [ is still problematic, any idea?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants