Support for Amazon Neptune

[kkom]

Hi all!

I'm running into a few issues using Gobling with Amazon Neptune.

My environment

I am testing the following package versions with Neptune 1.0.2.2 (see: https://docs.aws.amazon.com/neptune/latest/userguide/access-graph-gremlin-differences.html):

aiogremlin==3.3.5
goblin==2.2.3
gremlinpython==3.4.3

(aiogremlin includes a fix from goblin-ogm/aiogremlin#2 to enable SSL connection, hence the unreleased version 3.3.5)

I am defining an object using the example from http://goblin-ogm.com/ frontpage:

class Person(element.Vertex):
    __label__ = 'person'
    name = properties.Property(properties.String)
    age = properties.Property(properties.Integer)

Impossible to add a vertex

Trying to add a vertex results in this error:

GremlinServerError: 499: {"requestId":"3e83b817-e993-409c-afc0-693a676e533b","code":"UnsupportedOperationException","detailedMessage":"Properties on a vertex property is not supported"}

I was just playing with the object schema, and after changing it to:

class Person(element.Vertex):
    __label__ = 'person'
    name = properties.String
    age = properties.Integer

Adding the vertex succeeds, but I think the property is not stored on the graph? At least I can't see it, even if using the official gremlin console...

Traversing the graph

Traversing the graph in this way:

async def execute_person_traversal(
    session: Session,
) -> List[element.GenericVertex]:
    vertices = []

    async for vertex in session.traversal(Person):
        vertices.append(vertex)

    return vertices

results in the same exception as when adding the vertex before:

GremlinServerError: 500: 499: {"requestId":"47fa052d-8602-4d25-b0e0-6f51d009bd0e","code":"UnsupportedOperationException","detailedMessage":"Properties on a vertex property is not supported"}

Interestingly, this is regardless of how I define the object.

I think I previously could get it to work somehow, but I don't remember how. Maybe it was related to the state of the graph? Seems weird, so probably not though.

Other notes

Anyway, I would like to find out if it's possible to reconfigure Goblin to support Neptune. And if not, how much of an effort it would be to do so? Something like this was mentioned on the original ticket: davebshow/aiogremlin#14 (comment)

Also, what kind of limitations would that introduce in general? Maybe Neptune is so limited, that it's not even worth considering it as a practical Gremlin-backed graph database?

Thanks in advance for any help!

PS: This is me reposting my comment from a repository that contains an abandoned, older version of the library: davebshow/aiogremlin#14 (comment)

[kkom]

Here is my full code:

import asyncio
import os
from typing import List, Sequence

from goblin import Goblin, element, properties
from goblin.session import Session

NEPTUNE_CLUSTER_ENDPOINT = os.getenv("NEPTUNE_CLUSTER_ENDPOINT")
if NEPTUNE_CLUSTER_ENDPOINT is None:
    raise Exception("Environment variable NEPTUNE_CLUSTER_ENDPOINT not set")

NEPTUNE_PORT = os.getenv("NEPTUNE_PORT")
if NEPTUNE_PORT is None:
    raise Exception("Environment variable NEPTUNE_PORT not set")
NEPTUNE_PORT_INT = int(NEPTUNE_PORT)

loop = asyncio.get_event_loop()


# Object representing a Person vertex
class Person(element.Vertex):
    __label__ = 'person'
    name = properties.Property(properties.String)
    age = properties.Property(properties.Integer)


async def create_app() -> Goblin:
    return await Goblin.open(
        loop,
        scheme="https",
        hosts=[NEPTUNE_CLUSTER_ENDPOINT],
        port=NEPTUNE_PORT_INT,
        ssl_certfile="/etc/ssl/certs/SFSRootCAG2.pem",
    )


async def create_session() -> Session:
    app = await create_app()
    return await app.session()


people: List[Person] = []

for i in range(5):
    person = Person()
    person.name = f"Person {i}"
    person.age = i

    people.append(person)


async def add_people(session: Session, people: Sequence[Person]) -> None:
    for person in people:
        session.add(person)

    await session.flush()


async def execute_person_traversal(
    session: Session,
) -> List[element.GenericVertex]:
    vertices = []

    async for vertex in session.traversal(Person):
        vertices.append(vertex)

    return vertices

I'm testing it using the ipython console (it supports inline await)