Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IN-PROGRESS: Simply 3661 registry search #190

Open
wants to merge 13 commits into
base: develop
Choose a base branch
from

Conversation

nballenger
Copy link
Contributor

This is not ready for merge, it's a partial, in-progress view of the work I've done so far on search for the finder. In particular it's so we can discuss the geographic search portion, which is for SIMPLY-3662.

@nballenger
Copy link
Contributor Author

@leonardr - This is a big set of diffs (there are a number of formatting changes that make the overall diff pretty big), but here are the pieces to look at around geographic search:

  1. I've added a Location class in util/geo.py, which represents a point-based location with some convenience functions and input validation.
  2. Several functions from app_helpers.py have been moved to decorators.py, and now use the flask.g storage object, and store a user's location as a Location object if it is valid.
  3. The geographic search is still in the Library.nearby() class method, but has new logic
  4. The relevant tests are in tests/models/test_library.py and tests/test_util_geo.py

I'll flag individual pieces with comments.

@nballenger nballenger requested a review from leonardr May 25, 2021 21:10
return decorated


def uses_location(f):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decorator that looks for location, tries to store it in g.location as a Location instance.

# Decrease the score based on how far away the library's focus area is.
score = score * exponential_decrease(1.0 * focus_area_distance_factor * focus_min_distance)
@classmethod
def nearby(cls, db_session, target, max_radius=150, production=True):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New version of Library.nearby()

"""Raised when a Location is created with invalid input"""


class Location:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New Location class


GENERATED_SHORT_NAME_REGEX = re.compile(r'^[A-Z]{6}$')

##############################################################################
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explanation of geographic test fixtures

db_session.delete(a_library)
db_session.delete(a_place)
db_session.commit()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests of Library.nearby() start here


return f(*args, **kwargs)

return decorated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this duplicative of uses_location_factory in app_helper.py?


return response

return decorated
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm getting the impression that you've moved the app_helpers into this file and will be removing or greatly cutting app_helpers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right--I'll remove them from app_helpers once I confirm that nothing uses the older versions.

model.py Outdated
"""Constant container for library types.
This is as defined here:
https://github.com/NYPL-Simplified/Simplified/wiki/LibraryRegistryPublicAPI#the-subject-scheme-for-library-types
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be moved into constants.py?

# use the point-to-polygon-edge distance to order our result set.
query_obj = query_obj.add_columns(meters_to_near_edge)
query_obj = query_obj.group_by(Library.id)
query_obj = query_obj.order_by(meters_to_near_edge.asc())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note for later unless you fixed this, but this GROUP BY might explain why libraries sometimes show up twice in the results -- they have two different service areas that both match the criteria.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, interesting. I'll have to figure out how to consolidate that and test for it. Good catch.

[(focus_areas_subquery.c.type==Place.EVERYWHERE, literal_column(str(510000000000000)))],
else_=func.ST_Area(focus_areas_subquery.c.geometry)
)
) / 1000000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are handling supralocal libraries separately I'm OK with removing this.

goal=ExternalIntegration.DRM_GOAL
)
integration.setting(Configuration.ADOBE_VENDOR_ID).value = "VENDORID"
integration.setting(Configuration.ADOBE_VENDOR_ID_NODE_VALUE).value = "0x685b35c00f05"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just confirming this is a made-up value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far as I know, though I didn't make it up myself--I just ported it directly into a fixture from what's in here:

NODE_VALUE = "0x685b35c00f05"

* If truncating to fewer total characters than MAX_SEARCH_STRING_LEN,
the string should not end in a partial word.
"""
assert LibrarySearchQuery._normalize_search_string(search_string) == result
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've got code to correct the most common spelling of "library" ("libary") which isn't tested here; the code may not be in _normalize_search_string but I think it should be.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird, I remember adding that at some point because I'd seen it in the existing code. I'll make sure it gets back in.

return (latitude, longitude, srid)

@classmethod
def location_in_ocean(cls, location):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you plan to make use of this outside of tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to performance test it and use it if it's a cheap operation, which I think it should be. I just wanted some very simple protection against having to do the more expensive geographic operations on obviously unusable locations. Possibly a premature / unnecessary optimization, can take it out if it seems silly.

@leonardr
Copy link
Contributor

Just to set a baseline, I've reviewed the code so far. For the sake of unblocking mobile development, I think it's perfectly fine to merge this without any working search functionality -- just the proximity matching -- since the mobile apps currently don't use search.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants