feat: update vision-camera-plugin-inatvision to a version that fully supports the geomodel #2584
Conversation
| @@ -134,6 +132,10 @@ const FrameProcessorCamera = ( { | |||
|
|
|||
| const patchedOrientationAndroid = orientationPatchFrameProcessor( deviceOrientation ); | |||
| const patchedRunAsync = usePatchedRunAsync( ); | |||
| const hasUserLocation = !!userLocation && !!userLocation?.latitude && !!userLocation?.longitude; | |||
| const location = hasUserLocation | |||
There was a problem hiding this comment.
TODO: Explain this
There was a problem hiding this comment.
Why is this even necessary? You're only setting it when we already know the user location, so why would you ask InatVision for some other location when we already know the user's location?
There was a problem hiding this comment.
As far as I can tell this is working fine (I can build, AICamera and Suggestions work as I would expect both online and offline). I'm not noticing an improvement in results, but I also don't have a good example of something the vision model performed poorly on without geomodel input.
My only caveat is my confusion over why we need to re-request location info from the geomodel before, which doesn't need to block.
| @@ -134,6 +132,10 @@ const FrameProcessorCamera = ( { | |||
|
|
|||
| const patchedOrientationAndroid = orientationPatchFrameProcessor( deviceOrientation ); | |||
| const patchedRunAsync = usePatchedRunAsync( ); | |||
| const hasUserLocation = !!userLocation && !!userLocation?.latitude && !!userLocation?.longitude; | |||
| const location = hasUserLocation | |||
There was a problem hiding this comment.
Why is this even necessary? You're only setting it when we already know the user location, so why would you ask InatVision for some other location when we already know the user's location?
In this PR the vision-plugin was updated (changes see below).
In the app the
taxonomyRollupCutoffis removed. If no value is passed into the frame processor it uses a dynamic calculation of this value of 0.0001 times the top combined score. Combined score is equal to vision + geo scores or vision scores only if no geo scores obtained.Changes in the vision-plugin from 4.1.4 to 4.2.0 are:
https://www.worldclim.org/data/worldclim21.html
The location used for getting geomodel scores then consists of the average elevation as well as centroid of the h3 cell the passed in location is in.
Edit (kvangork) for tracking purposes: ML-185