Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FIX] OWSql: enforce data download for non PostgreSQL databases #3178

Merged
merged 2 commits into from
Aug 3, 2018

Conversation

robertcv
Copy link
Collaborator

@robertcv robertcv commented Aug 1, 2018

Issue

Fixes #3043

Description of changes

SqlTable has poor support for non-PostgreSQL databases. This enforces the download of the selected table so that it works as a normal Table.

Includes
  • Code changes
  • Tests
  • Documentation

@robertcv robertcv force-pushed the fixes/sql_download branch from a8dc795 to b39f43b Compare August 1, 2018 11:55
@codecov-io
Copy link

codecov-io commented Aug 1, 2018

Codecov Report

Merging #3178 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #3178      +/-   ##
==========================================
+ Coverage   82.54%   82.54%   +<.01%     
==========================================
  Files         337      337              
  Lines       58431    58446      +15     
==========================================
+ Hits        48229    48246      +17     
+ Misses      10202    10200       -2

@lanzagar lanzagar added this to the 3.15 milestone Aug 3, 2018
@lanzagar lanzagar merged commit 2fb5af8 into biolab:master Aug 3, 2018
@Ryujjin
Copy link

Ryujjin commented Sep 14, 2018

Thank you for the patch but with the auto download always enabled the data is to large to fit in memory. No data is loaded and thus no widgets can be used to manipulate the information. What is the workaround for working with large databases?

Thanks!

@ajdapretnar
Copy link
Contributor

I thought we went with sampling here... But it could be we discarded this option for some reason.

@Ryujjin Did you try using Data Sampler immediately after SQL and sample it yourself?

@Ryujjin
Copy link

Ryujjin commented Sep 18, 2018

Thanks for the reply, I need to be able to pick date ranges or time intervals, last I looked the sampler appeared to be a random range but I could be wrong will look at it again.

Thanks!

@lanzagar
Copy link
Contributor

If you know what kind of subset of a large data set you want to explore it will probably be best to use an SQL select query to get just that data. This will also be much faster than transferring large amounts of data and sampling on the client.

@robertcv robertcv deleted the fixes/sql_download branch May 31, 2019 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants