Connecting to datastore emulator doesn't work with v0.7.0+

[enyo]

We have been running on v0.6.1 for a while and just upgraded to the latest version but can't connect to our datastore emulator anymore. We checked which version broke it, and appears to be v0.7.0: Refactored to use package:grpc to talk to Logging and Datastore backends. which fits with the error message we get:

Unhandled exception:
  gRPC Error (14, Error connecting: SocketException: OS Error: Connection refused, errno = 61, address = localhost, port = 53263)
  #0      GrpcDatastoreImpl.query (package:appengine/src/grpc_api_impl/datastore_impl.dart:258:41)
  <asynchronous suspension>
  dart-lang/labs#99      Query.run.<anonymous closure> (package:gcloud/src/db/db.dart:246:12)
  dart-lang/labs#100      StreamFromPages._onListen (package:gcloud/common.dart:72:5)
  dart-lang/labs#101      _runGuarded (dart:async/stream_controller.dart:805:24)
  dart-lang/labs#102      _StreamController._subscribe.<anonymous closure> (dart:async/stream_controller.dart:684:7)
  dart-lang/labs#103      _BufferingStreamSubscription._guardCallback (dart:async/stream_impl.dart:414:13)
  dart-lang/labs#104      _StreamController._subscribe (dart:async/stream_controller.dart:683:18)
  dart-lang/labs#105      _ControllerStream._createSubscription (dart:async/stream_controller.dart:818:19)
  dart-lang/labs#106      _StreamImpl.listen (dart:async/stream_impl.dart:472:9)
  dart-lang/labs#107      new _ForwardingStreamSubscription (dart:async/stream_pipe.dart:123:10)
  dart-lang/labs#108     _ForwardingStream._createSubscription (dart:async/stream_pipe.dart:91:16)
  dart-lang/labs#109     _ForwardingStream.listen (dart:async/stream_pipe.dart:86:12)
  dart-lang/appengine#12     Stream.toList (dart:async/stream.dart:1048:10)
  dart-lang/labs#110     createPageSitemap (package:_server/utils/sitemap.dart:23:12)
  <asynchronous suspension>
  dart-lang/labs#111     _startServer (package:_server/web_server/server.dart:63:9)
  <asynchronous suspension>
  dart-lang/labs#112     startServer.<anonymous closure> (package:_server/web_server/server.dart:34:60)
  dart-lang/labs#113     withAppEngineServices.<anonymous closure> (package:appengine/src/appengine_internal.dart:37:51)
  dart-lang/labs#114     _withAppEngineServicesInternal.<anonymous closure> (package:appengine/src/appengine_internal.dart:110:20)
  <asynchronous suspension>
  dart-lang/labs#115     _ServiceScope._fork.<anonymous closure> (package:gcloud/service_scope.dart:209:19)
  dart-lang/labs#116     _rootRun (dart:async/zone.dart:1124:13)
  dart-lang/labs#117     _CustomZone.run (dart:async/zone.dart:1021:19)
  dart-lang/labs#118     _runZoned (dart:async/zone.dart:1516:10)
  dart-lang/labs#119     runZoned (dart:async/zone.dart:1463:12)
  dart-lang/labs#120     _ServiceScope._fork (package:gcloud/service_scope.dart:208:12)
  dart-lang/labs#121     fork (package:gcloud/service_scope.dart:104:30)
  dart-lang/labs#122     _withAppEngineServicesInternal (package:appengine/src/appengine_internal.dart:101:10)
  dart-lang/labs#123     withAppEngineServices (package:appengine/src/appengine_internal.dart:37:5)
  dart-lang/labs#124     withAppEngineServices (package:appengine/appengine.dart:150:10)
  dart-lang/labs#125     startServer (package:_server/web_server/server.dart:34:28)
  dart-lang/labs#126     main (file:///Users/enyo/Develop/exitlive/server/bin/server.dart:6:9)
  <asynchronous suspension>
  dart-lang/labs#127     _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:303:32)
  dart-lang/labs#128     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)

Do you have any suggestion on how to fix this?

[domesticmouse]

Sounds like the datastore emulator doesn't do gRPC. I don't currently have time to dig into this, but will happily take pointers if you dig into what is breaking and where. A simplified reproduction sample would be a great start.

[enyo]

Thanks. will provide ASAP

[domesticmouse]

Sweet, thanks!

[martflu]

Here is a gist of how to reproduce the problem.

As soon as requestHandler is called and a query is ran on the datastore emulator, the error occurs.

[enyo]

Any update on this? Not being able to use the datastore emulator is a big problem for us.

[enyo]

@jonasfj @domesticmouse sorry to bump this again, but is there a way this could be looked at? We are unable to upgrade any library at the moment because of this, and it's becoming more and more of a problem.

EDIT: Is there a way to connect to the real datastore from my local machine until this is fixed?

[jonasfj]

EDIT: Is there a way to connect to the real datastore from my local machine until this is fixed?

@enyo, Yes, see: https://github.com/dart-lang/appengine#running-without-docker
Basically, create a GCP service account and export the key as json. And make sure to give the service account permissions to datastore.

In pub.dev development we use a fake in-memory gcloud datastore backed for unit tests, and we use a staging site for local development.

---

On topic, if you're interested in debugging this please dive in. We're happy to take contributions.
Maybe it's just something simple... We could also remove this from the readme, so as to not advertize a feature that doesn't work anymore.

[enyo]

Thanks @jonasfj

I think that in the long run, having a local datastore emulator is quite crucial for development, so it would be great to get this fixed

I'm happy to help debugging this, but I have no idea how the communication between these two services is working. It would be great if somebody who made the breaking change could look at this.