Hook into existing database without migrations

[lastmjs]

I'm not sure if this is possible in graphback, but I would love it to be. One essential thing that Prisma 1 does is allow users to hook into an existing database, and allow other tools to handle migrations. This allowed me to generate all of my GraphQL infrastructure using Prisma, but for an already-existing Postgres DB whose migrations are handled with Rails' Active Record. Because we're still in the process of migrating away from our Rails app, it is convenient to allow the Rails app to handle database migrations.

Does graphback support this type of workflow? Could it? As Prisma 2 seems to be going in a direction that I don't agree with, I'm assessing the viability of graphback as a replacement

[lastmjs]

It would also be nice if graphback could be used entirely without Docker, and just hook into an existing Postgres database through a config file that points to the database URL, username, password, etc

[wtrocki]

Graphback is designed as a set of tools rather than closed ecosystem. It consists:

• A number of code generators for schema, resolvers etc.
  Each of those can be swapped for developer needs
• Runtime layer (similar to Prisma) resolves the challenge of making performant queries to the database without generating a lots of complex codebase.
• DB migrations package that makes changes in db

Each of those can be used separately or together. I will add more info into the docs to make this clear.

As for the "bring your own database" - technically this is really simple to achieve however, graphback at the moment is not providing db to graphql mapping package, which means that data from the database needs to conform with the graphql fields and types. This limitation can be resolved by:

1. simply providing your own runtime layer that will understand how your database is structured
   https://graphback.dev/docs/runtime

2. writing some generator that will create some ORM types etc.

In both cases there will be some extra work needed that might depend on the structure of your database etc.
Our team can check if we can build some example that does that kind of mapping to understand if runtime will allow easily to support those cases. We have written custom solutions for these cases in the past (with a strong focus on performance).
I'm going to pin this issue to see if we can get some opinions about this and possible approaches.

To support that we can have a community runtime layer that works with Bookshelf. See: knex/knex#498

When we think about it we are closer to AWS App Sync rather than Prisma2 only because Schema really drives the data.

In TL;DR Graphback will give much more flexibility and freedom (and migration patterns) but it will mean much more work required on developer side to figure out the best pattern

As a team we are really interested in getting some community taction in those cases as we can do much better job at making Graphback successful and something that community really wants.

It would also be nice if graphback could be used entirely without Docker, and just hook into an existing Postgres database through a config file that points to the database URL, username, password, etc

Graphback can be used without docker. We provide docker for convenience and also support SQL Lite. You can choose your database when executing graphback config or manually by supplying knex compatible configuration for any database (we going to document that). Our default runtime layer uses knex and it will support every database

EDIT: To really make this successful we will need to know your take on my response and more details about your setup like database type and difference between database field and graphql type

[lastmjs]

Very detailed response, thank you.

1. simply providing your own runtime layer that will understand how your database is structured
   https://graphback.dev/docs/runtime

2. writing some generator that will create some ORM types etc.

I would love some more documentation or direction on how to do 1 and 2. As for my setup, I'm using Postgres, and a GraphQL schema that was originally generated using Prisma's introspection command. I can post some select examples here if you need them. I generated the GraphQL schema with I believe Prisma v1.21.1. Once the original GraphQL schema was generated, I transitioned to using manual updates to the schema, not using the introspection command every time a database migration occurred. Prisma is able to understand my custom GraphQL schema and map it to the underlying Postgres types. This is very convenient.

Ideally, I as a developer would not need to write any custom mapping code to get my GraphQL schema to work with a Postgres, MySQL, or other database. I believe each of the database types should have their own plugins, adapters, runtimes, or whatever you might want to call them for resolving GraphQL queries against them.

Prisma was working on things like OpenCRUD, trying to create standards for many of these problems. I know they are moving away from that, but is anyone picking up the slack here? Seems like graphback is just going to be creating yet another opinionated set of generated CRUD queries. I'm not saying graphback's would be worse, but it would be nice if we could standardize this.

I haven't dug into graphback yet to really see how things work, I should be able to dedicate some significant time to it sometime in the next month. I have a number of requirements that I'm looking for, not sure if this is the best time or place to list these, but perhaps:

1. I as a developer should have to write one canonical GraphQL schema for all of the persistent types (such as DB types) in my application. Queries and resolvers for all basic CRUD operations should be generated from this schema (I believe graphback does this well)
2. There should be no extra infrastructure outside of the Node.js application to resolve queries. Prisma created an entirely separate set of processes that had to run in a docker container, accessible only through http requests accessed through a specialized API. This created a lot of extra complexity. The resolver system should run in the same process or be booted up in the same process as the Node.js app that is resolving queries (I believe graphback does this well)
3. Extending the canonical GraphQL schema with custom queries and adding custom resolvers should be simple (I believe graphback does this well)
4. On the server, I should be able to write actual GraphQL queries that are then resolved against the database. I don't want to have to go through a generated JavaScript API on the server, just GraphQL queries. Writing queries should be the same on the server as it is on the client, pure GraphQL. (I believe graphback will generate a schema object which I can pass to something like graphql-js to resolve queries, correct?)
5. Advanced custom directives on generated schema. Allowing custom directives in the canonical schema is great, but for advanced permissions you may also need to generate directives based on the directives from the main type schema, to add to the generated schema. For example, adding permissions on record creation, or any mutations for that matter. This directive will need to apply behavior to the create mutations for a type. The create mutations are only visible after CRUD generation, and so there needs to be a way to apply custom directives to those mutations. We also need to be able to support per-field permissions on creating, updating, deleting, etc. None of this can be done with simple custom directives in the main typed schema, but must be done in the generated schema
6. There are more, I'm sure they will come up the deeper I get into graphback. Prisma had many unresolved issues that I'm hoping graphback will be able to solve. Some of the main themes relating to Prisma's limitations from my point of view are spelled out here: https://medium.com/@lastmjs/advanced-graphql-directive-permissions-with-prisma-fdee6f846044

Perhaps graphback is already doing a lot of these things, and perhaps the documentation just isn't in a state where all of these things are obviously understood without further inquiry. I'm really excited about this project, I hope it can be great!

[wtrocki]

Thank you so much for investing your time for such amazing feedback. We will need more time to digest all the points and come up with a solid plan (and direct actions/issues). I would like to be able to provide info on what we could do and when. Graphback is still in early beta phase and we are working towards an official release that will clarify docs.
We going to improve our documentation in very short timeframe and work on the path that could make graphback (and entire graphql-cli ecosystem) an alternative to resolver first approach)

Your assumptions are correct. I think we need to clarify better what graphback does and how to use it for different use cases in docs.

[craicoverflow]

Hi @lastmjs - thanks for your detailed response which shows us some really strong uses cases for Graphback. We are now in the design phase of an architectural change, which will make Graphback more flexible and easier to add more customisation cases in the future. You can gain more context of this refactor in #635.

I would love some more documentation or direction on how to do 1 and 2.

Our documentation does not have directions for this. We will be adding a runtime mapping solution that should resolve this though. Tracked in #637

Ideally, I as a developer would not need to write any custom mapping code to get my GraphQL schema to work with a Postgres, MySQL, or other database. I believe each of the database types should have their own plugins, adapters, runtimes, or whatever you might want to call them for resolving GraphQL queries against them.

This is something we are aware is missing - the current custom mapping code was really only a temporary solution to document until we can map fields during runtime. We have a @db.name annotation which is used during database migration. We can use the same annotation at runtime to map fields in the resolvers. This is definitely something we will be looking to implement soon. I've created #637 to track this feature proposal.

Prisma was working on things like OpenCRUD, trying to create standards for many of these problems. I know they are moving away from that, but is anyone picking up the slack here? Seems like graphback is just going to be creating yet another opinionated set of generated CRUD queries. I'm not saying graphback's would be worse, but it would be nice if we could standardize this.

We had a look into OpenCRUD some more and there really doesn't seem to be much activity or adoption. I've created #638 for us to take a deeper dive into the specification, try and get some opinions from the wider community and see if we should adopt it.

On the server, I should be able to write actual GraphQL queries that are then resolved against the database. I don't want to have to go through a generated JavaScript API on the server, just GraphQL queries. Writing queries should be the same on the server as it is on the client, pure GraphQL. (I believe graphback will generate a schema object which I can pass to something like graphql-js to resolve queries, correct?)

Our new plugin architecture will allow us to bring those use cases basing on libraries that are available in the community.

Advanced custom directives on generated schema. Allowing custom directives in the canonical schema is great, but for advanced permissions you may also need to generate directives based on the directives from the main type schema, to add to the generated schema. For example, adding permissions on record creation, or any mutations for that matter. This directive will need to apply behavior to the create mutations for a type. The create mutations are only visible after CRUD generation, and so there needs to be a way to apply custom directives to those mutations. We also need to be able to support per-field permissions on creating, updating, deleting, etc. None of this can be done with simple custom directives in the main typed schema, but must be done in the generated schema

We are in the process of refactoring to use a plugin-based approach, which will use the generated schema (with annotations and directives) as the input source to the generator packages. This will let users bring their own custom directives and annotations for their specific use cases.

Perhaps graphback is already doing a lot of these things, and perhaps the documentation just isn't in a state where all of these things are obviously understood without further inquiry. I'm really excited about this project, I hope it can be great!

Within only a couple of months Graphback has pivoted from being a CLI-based GraphQL server generator to integrating into the graphql-cli as the generator for the server and database. Our documentation is still a little bit disjointed in this regard and we have intentions to fill that void soon (#631). With our new planned plugin-based architecture it should soon become easier to do a lot of these things!

[wtrocki]

Hi

We have done initial work for supporting that use case in our plugin based architecture. We going to provide short blog post soon explaining the nature of the changes. For the field mapping we still need to connect existing implementations with the CRUD layer that is being used by resolvers.

@craicoverflow is going to deal with that.

We going to support mapping for:

• ID (Primary keys)
• Table Names
• Field Names
• Relationship field names

Mappings will be done by utilizing `@db' annotations that will give developers the ultimate flexibility with very little to no work needed outside annotating your input schema.

[wtrocki]

Update. We have made major improvements to enable this case.
We still did not release the official version but early feedback can be provided.

What we have at the moment:

• Id mapping
• Relationships mapping
• Table name mapping

To be done:

• Field level mapping

[lastmjs]

Hopefully I'll have time soon on my project to spend a bit of time seeing how this is all looking, great work!

[lastmjs]

This has been working very well for me!