Post support as in http://aws.amazon.com/articles/1434

[romanbsd]

Looking for some feedback. If it looks fine, then I can invest some time and write some tests.

[yagudaev]

+1, all that is needed is to enable CORS. This can be as simple as exposing a commandline flag --allow-cors which would just return Access-Control-Allow-Origin: *.

EDIT: on a second thought, this might be slightly more complicated. It will need to accept any tokens sent to it and implement a similar interface. And What I was talking about above was direct upload to s3 using ajax. Similar to what Ryan Bates showed here: http://railscasts.com/episodes/383-uploading-to-amazon-s3

[romanbsd]

You can see what was needed to be implemented here: https://github.com/jubos/fake-s3/pull/28/files

[yagudaev]

just tested it out and got: 'can't convert nil into String'. That is responding back to me after it attempts to send the file.

Below are the sent headers:

------WebKitFormBoundary6bGmHjAz1p7332MA
Content-Disposition: form-data; name="utf8"

✓
------WebKitFormBoundary6bGmHjAz1p7332MA
Content-Disposition: form-data; name="key"

uploads/2a9949f55de9f48bc7460737e9b48327/${filename}
------WebKitFormBoundary6bGmHjAz1p7332MA
Content-Disposition: form-data; name="acl"

public-read
------WebKitFormBoundary6bGmHjAz1p7332MA
Content-Disposition: form-data; name="policy"

 eyJleHBpcmF0aW9uIjoiMjAxMy0wNC0xM1QxNjowNjo0MloiLCJjb25kaXRpb25zIjpbWyJzdGFydHMtd2l0aCIsIiR1dGY4IiwiIl0sWyJzdGFydHMtd2l0aCIsIiRrZXkiLCIiXSxbImNvbnRlbnQtbGVuZ3RoLXJhbmdlIiwwLDUyNDI4ODAwMF0seyJidWNrZXQiOiJkZXZlbG9wbWVudC10ZW1wLTBpZGxlIn0seyJhY2wiOiJwdWJsaWMtcmVhZCJ9XX0=
------WebKitFormBoundary6bGmHjAz1p7332MA
Content-Disposition: form-data; name="signature"

2pnHgSCXslHRj6fvBCA3dcZLB+Q=
------WebKitFormBoundary6bGmHjAz1p7332MA
Content-Disposition: form-data; name="AWSAccessKeyId"

AKIAI2DBJT7LDNC2GOOA
------WebKitFormBoundary6bGmHjAz1p7332MA
Content-Disposition: form-data; name="file"; filename="space_27.jpg"
Content-Type: image/jpeg


------WebKitFormBoundary6bGmHjAz1p7332MA--

Never the less, I think this is a great start :)

[romanbsd]

The request looks fine on the first sight. I'm afk, so I will be able to look at it only in a week. We are actually using it for development of our application btw.

[yagudaev]

Well I am sure it is something small. If there are any small debug steps you want me to try, I'll give it a shot.

I also wanted to use it for development, right now I am using fake s3 90% of the way. The other 10% is initial upload to a temporary s3 bucket that gets cleared every two days. Follow the instructions on RailCasts with some modifications to achieve that. http://railscasts.com/episodes/383-uploading-to-amazon-s3

[romanbsd]

Try to add on line 203 in lib/server.rb (above response.status = 400)
puts e.backtrace.join("\n")
This way I can see where the errors occurs.

[yagudaev]

can't convert nil into String
----- stack trace ----
/Users/mike/.rbenv/versions/1.9.3-p362/lib/ruby/gems/1.9.1/gems/fakes3-0.1.5.1/lib/fakes3/file_store.rb:57:in `join'
/Users/mike/.rbenv/versions/1.9.3-p362/lib/ruby/gems/1.9.1/gems/fakes3-0.1.5.1/lib/fakes3/file_store.rb:57:in `create_bucket'
/Users/mike/.rbenv/versions/1.9.3-p362/lib/ruby/gems/1.9.1/gems/fakes3-0.1.5.1/lib/fakes3/server.rb:237:in `get_bucket'
/Users/mike/.rbenv/versions/1.9.3-p362/lib/ruby/gems/1.9.1/gems/fakes3-0.1.5.1/lib/fakes3/server.rb:177:in `do_POST'
/Users/mike/.rbenv/versions/1.9.3-p362/lib/ruby/1.9.1/webrick/httpservlet/abstract.rb:106:in `service'
/Users/mike/.rbenv/versions/1.9.3-p362/lib/ruby/1.9.1/webrick/httpserver.rb:138:in `service'
/Users/mike/.rbenv/versions/1.9.3-p362/lib/ruby/1.9.1/webrick/httpserver.rb:94:in `run'
/Users/mike/.rbenv/versions/1.9.3-p362/lib/ruby/1.9.1/webrick/server.rb:191:in `block in start_thread'

Took a few more debugging steps and discovered the bucket under s_req.bucket is nil for some reason.

[romanbsd]

Interesting. I don't think that I even modified anything related to this code. I think this is due to the way the bucket is deduced, especially when handling POST.
Try the following- add to /etc/hosts something like:
127.0.0.1 my-bucket.localhost
and then use http://my-bucket.localhost/ as an endpoint you're performing the POST to.

[yagudaev]

It is totally because of that. Fakes3 uses a different structure than the real s3 with the bucket after the host name instead of the bucket being a subdomain (to avoid the whole /etc/hosts mess). The code that gets the bucket needs to be patched. So the path to my bucket is http://localhost:4567/my-bucket

Looking at the code it is rather confusing and I am not sure why a GET request would go through different normalization than a POST request. That is where the problem comes from.

[yagudaev]

Ok the following 5 lines need to be added at the end of normalize_post method will solve the problem.

    if s_req.is_path_style
      path = webrick_req.path
      path_len = path.size
      s_req.bucket = path[1,path_len].split("/")
    end

It seems to work quite well now :). If you can commit that @romanbsd that would be great.

This is what my url method looks like (based on the railcast):

    def url
      return "https://#{@options[:bucket]}.s3.amazonaws.com/" unless CONFIG[:download_from] == 'localhost'

      # fakes3
      return "http://#{CONFIG[:aws_s3_host]}:#{CONFIG[:aws_s3_port]}/#{@options[:bucket]}/"
    end

Hope this gets pull into the main branch soon. Very useful, thanks for the hard work @romanbsd.

[romanbsd]

I should better refactor it, to DRY it up.

[mcfiredrill]

I could really use this feature, any progress?

[romanbsd]

You can use my fork: https://github.com/romanbsd/fake-s3.git , we've been successfully using it for almost a year now.

[mcfiredrill]

Thanks, I'll try that.

[rahulinux]

Hi,

Could please help me with Access Key ID, because I'm trying to use fakes3 and It's stuck at access key id.

Error:

ERROR: S3 error: 403 (InvalidAccessKeyId): The AWS Access Key Id you provided does not exist in our records.

[fryzhykau]

The same problem like rahulinux has. How to setup keys for server?

[rahulinux]

I think you need to add s3.amazonaws.com ( or like eu-bucket.s3.amazonaws.com ) to point to your fake s3 server IP

[ghost]

rahulinux or ironmanby, how do we go around this issue? I've been stuck on it for a while now. Any fixes you could help me with that worked for you?