Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add postgresql schemata options #2028

Closed

Conversation

jrjsmrtn
Copy link

@jrjsmrtn jrjsmrtn commented Sep 15, 2023

Add PgSchemataTable class
Add support for options.postgres_schema "*" and "**":

  • open a schemata table
  • "*" show user schemata
  • "**" show user and system schemata
  • entering a schemata row open the table

Sheet names follow the "database[.schema[.table]]" format

Fixes #2027

Add PgSchemataTable class
Add support for options.postgres_schema "*" and "**":
- open a schemata table
- "*" show user schemata
- "**" show user and system schemata
- entering a schemata row open the table
Sheet names follow the "database[.schema[.table]]" format
@CLAassistant
Copy link

CLAassistant commented Sep 15, 2023

CLA assistant check
All committers have signed the CLA.

@jrjsmrtn
Copy link
Author

  • vd postgres:///dbname still show public.* tables;
  • vd --postgres-schema=myschema postgres:///dbname still show myschema.* tables;
  • vd --postgres-schema=* postgres:///dbname will show the database schemas;
  • vd --postgres-schema=** postgres:///dbname will show both database and system schemas;

Copy link
Owner

@saulpw saulpw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jrjsmrtn, thanks for putting this together! This will be a useful addition. There are a few things that need to be adjusted, if you can make these changes, I think we're well on our way to getting it merged.

rowtype = 'schemas'

def reload(self):
if options.postgres_schema == "*":
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use self.options (here at least, also other places it's used) so that the option can be overridden on a per-sheet basis.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And let's make it so that an empty string means all-but-internal schema, instead of *. Maybe * can mean "including internal schemas" like you have for ** below. Do we think it's important to support internal schemas though?

qstr = """SELECT schema_name FROM information_schema.schemata;"""

with self.sql.cur(qstr) as cur:
self.nrowsPerTable = {}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used anywhere?

return PgTable(self.name+"."+row[0], source=row[0], sql=self.sql)
return PgTable(
self.name+"."+row[0],
source=(self.source or options.postgres_schema)+"."+row[0],
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the given name and the source should be the same here. Let's use source, since I think it's reasonable to require source to be the schema for this class, and only use options.postgres_schema in the outer openurl_postgres function.

class PgSchemataSheet(Sheet):
rowtype = 'schemas'

def reload(self):
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally we're standardizing on defining iterload(), which is almost entirely like reload, but you don't have to clear the rows, and instead of calling addRow() directly, just yield the row.

@anjakefala
Copy link
Collaborator

Ping @jrjsmrtn!

@jrjsmrtn
Copy link
Author

jrjsmrtn commented Nov 15, 2023 via email

Comment on lines -125 to +176
if self.options.postgres_schema:
source = f"{self.options.postgres_schema}.{self.source}"
else:
source = self.source
source = self.source
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will conflict with #2129, where the table name and schema were quoted to support both being something other than lowercase.

i.e.:

"public"."TableName" rather than public.TableName or "public.TableName"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for checking! As part of this PR, should we then move the #2129 fix into openurl_postgres?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will have to check that the loading of lowercase table names is preserved before merging.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we then move the #2129 fix into openurl_postgres?

I don't see how that could be done. I think my tweak belongs in these places, where PgTable.source is set:

certainly here:
https://github.com/jrjsmrtn/visidata/blob/621dd88a75c6a133ce1a79cbd795c51fccd1d743/visidata/loaders/postgres.py#L167

maybe here?:
https://github.com/jrjsmrtn/visidata/blob/621dd88a75c6a133ce1a79cbd795c51fccd1d743/visidata/loaders/postgres.py#L123

I don't understand what some of the variables in this code refers to, such as row[0], columns[0:1], etc, so I'm on shaky ground here.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

row[0] is the table_name as per the rowdef comment at the top of the class. columns[0:1] creates a list containing the first column (and an empty list if there are no columns).

I don't think the quoting should be done in the schema.table string given to the sheet; the SQL should do the quoting. So this means that the schema and the table name should be passed separately to the postgres sheet.

@anjakefala
Copy link
Collaborator

Hi @jrjsmrtn!

We're going to close this for now due to lack of activity. Please re-open when you make the changes!

@anjakefala anjakefala closed this Feb 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add option to show tables from all PgSQL schemas
5 participants