New google datastore

[mdipierro]

Tested only with this:

from pydal import DAL, Field
db = DAL("google:datastore")
db.define_table("thing", Field("name"), Field("age", "integer"), Field('price','float'), Field('active', 'boolean'))
print(db.thing.insert(name="table", age=1, price=3.14, active=True))
rows = db(db.thing.name=="table").select()
print(str(rows))
print(db(db.thing.name=="table").count())
rows = db(db.thing.age==1).select()
print(str(rows))
id = rows[-1].id
row = db.thing(id)
print(row)
row.update_record(name="chair")
row.delete_record()
print(db.thing(id))
rows = db(db.thing).select()
print(str(rows))
db(db.thing.name=="chair").delete()

[jparga]

Hi,
Thanks it is promising for py4web. I have made some inicial test in gae environment.

• I had to include google-cloud-datastore in requirements.txt
• I had to create indexes with index.yaml (as usual). In particular this index has to be created to access auth table:

indexes:
# AUTOGENERATED

- kind: auth_user
  properties:
  - name: username
  - name: last_name
  - name: id
  - name: first_name
  - name: email

• The tables were created and the data inserted:

• I could create a grid for any table, and it worked! (the headers orderby not, as it's not implemented)
  
  

• There is a problem with the validation in auth.py, so I couldn't register an user.

"Traceback (most recent call last):
  File "/srv/lib/py4web/core.py", line 908, in wrapper
    ret = func(*func_args, **func_kwargs)
  File "/srv/lib/py4web/core.py", line 859, in wrapper
    ret = func(*args, **kwargs)
  File "/srv/lib/py4web/core.py", line 859, in wrapper
    ret = func(*args, **kwargs)
  File "/srv/lib/py4web/utils/auth.py", line 913, in _
    form=form_factory(), path=path, user=auth.get_user(), **env
  File "/srv/lib/py4web/utils/auth.py", line 1156, in register
    form = Form(fields, submit_value=button_name, formstyle=self.formstyle)
  File "/srv/lib/py4web/utils/form.py", line 792, in __init__
    (value, error) = field.validate(original_value, record_id)
  File "/srv/lib/pydal/objects.py", line 2259, in validate
    (value, error) = validator(value, record_id)
  File "/srv/lib/pydal/validators.py", line 159, in __call__
    return self.validate(value, record_id), None
  File "/srv/lib/pydal/validators.py", line 806, in validate
    if subset.select(limitby=(0, 1)):
  File "/srv/lib/pydal/objects.py", line 2719, in select
    return adapter.select(self.query, fields, attributes)
  File "/srv/lib/pydal/adapters/google.py", line 221, in select
    items, table = self.select_raw(query, fields, attributes)
  File "/srv/lib/pydal/adapters/google.py", line 195, in select_raw
    for item in self.parse_and_query(query):
  File "/srv/lib/pydal/adapters/google.py", line 158, in parse_and_query
    raise RuntimeError("Datastore cannot handle expressions")
RuntimeError: Datastore cannot handle expressions"

Tomorrow more tests. Thanks Massimo!

[jparga]

Hi,

There is another problem with "expresions like or" in auth/verify_email in py4web:

Traceback (most recent call last): File "/srv/lib/py4web/core.py", line 916, in wrapper ret = func(*func_args, **func_kwargs) File "/srv/lib/py4web/core.py", line 867, in wrapper ret = func(*args, **kwargs) File "/srv/lib/py4web/core.py", line 867, in wrapper ret = func(*args, **kwargs) File "/srv/lib/py4web/utils/auth.py", line 942, in _ form=form_factory(), path=path, user=auth.get_user(), **env File "/srv/lib/py4web/utils/auth.py", line 1435, in verify_email verified = self.auth.verify_email(token) File "/srv/lib/py4web/utils/auth.py", line 618, in verify_email n = self.db(self._query_from_token(token)).update(action_token=action_token) File "/srv/lib/pydal/objects.py", line 2786, in update ret = db._adapter.update(table, self.query, row.op_values()) File "/srv/lib/pydal/adapters/google.py", line 266, in update items, table = self.select_raw(query) File "/srv/lib/pydal/adapters/google.py", line 195, in select_raw for item in self.parse_and_query(query): File "/srv/lib/pydal/adapters/google.py", line 158, in parse_and_query raise RuntimeError("Datastore cannot handle expressions like %s" % query.op.__name__) RuntimeError: Datastore cannot handle expressions like _or

[jparga]

There is a problem with the auth register. I does not storage the password correctly. I had to declare a password field directly in datastore, and then I coudl login.

The password change arises this error:
Traceback (most recent call last): File "/srv/lib/py4web/core.py", line 916, in wrapper ret = func(*func_args, **func_kwargs) File "/srv/lib/py4web/core.py", line 867, in wrapper ret = func(*args, **kwargs) File "/srv/lib/py4web/core.py", line 867, in wrapper ret = func(*args, **kwargs) File "/srv/lib/py4web/utils/auth.py", line 930, in _ form=form_factory(), path=path, user=auth.get_user(), **env File "/srv/lib/py4web/utils/auth.py", line 1377, in change_password self._process_change_password_form(form, user, True) File "/srv/lib/py4web/utils/auth.py", line 1388, in _process_change_password_form res = self.auth.change_password( File "/srv/lib/py4web/utils/auth.py", line 669, in change_password num = db(db.auth_user.id == user.get("id")).update( File "/srv/lib/pydal/objects.py", line 2786, in update ret = db._adapter.update(table, self.query, row.op_values()) File "/srv/lib/pydal/adapters/google.py", line 271, in update self.client.put(item) File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/datastore/client.py", line 575, in put self.put_multi(entities=[entity], retry=retry, timeout=timeout) File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/datastore/client.py", line 612, in put_multi current.put(entity) File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/datastore/batch.py", line 227, in put _assign_entity_to_pb(entity_pb, entity) File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/datastore/batch.py", line 373, in _assign_entity_to_pb bare_entity_pb = helpers.entity_to_protobuf(entity) File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/datastore/helpers.py", line 215, in entity_to_protobuf _set_protobuf_value(value_pb, value) File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/datastore/helpers.py", line 449, in _set_protobuf_value attr, val = _pb_attr_value(val) File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/cloud/datastore/helpers.py", line 366, in _pb_attr_value raise ValueError("Unknown protobuf attr type", type(val)) ValueError: ('Unknown protobuf attr type', <class 'pydal.validators.LazyCrypt'>)

[jparga]

When the fields of a table are modified, I mean, when a new field is added, grid.py in that table stops working with this error:

Traceback (most recent call last): File "/srv/lib/py4web/core.py", line 916, in wrapper ret = func(*func_args, **func_kwargs) File "/srv/lib/py4web/core.py", line 867, in wrapper ret = func(*args, **kwargs) File "/srv/apps/best/controllers.py", line 118, in thinggrid grid = Grid( File "/srv/lib/py4web/utils/grid.py", line 447, in __init__ self.process() File "/srv/lib/py4web/utils/grid.py", line 655, in process self.rows = db(query).select(*self.needed_fields, **select_params) File "/srv/lib/pydal/objects.py", line 2719, in select return adapter.select(self.query, fields, attributes) File "/srv/lib/pydal/adapters/google.py", line 235, in select items, table = self.select_raw(query, fields, attributes) File "/srv/lib/pydal/adapters/google.py", line 217, in select_raw first, op, second = item.op(item.first, item.second) AttributeError: 'list' object has no attribute 'op'

[mdipierro]

We added support for the Firestore instead