Added Wit.ai document type determiner #63

Awinja-j · 2017-10-13T13:34:27Z

We are currently using keywords to determine which document type the application should query from, which limits to the application to some words. Wit.ai will identify key words in queries and determine which document type it should query.

This PR therefore introduces:

wit.ai message API which returns the extracted meaning from a sentence, based on the app data.

This task involved:

opening a wit.ai account
Training wit.ai to check for the 8 entities that health tools uses which are:
doctors, nurses, clinical-officers,health-facilities,nhif-inpatient,nhif-outpatient,nhif-outpatient-cs and query

wit.ai is suitable for the `/search` endpoint and further `SMS` function

It will enable users do searches like:
http://127.0.0.1:5000/search?q=is Doctor Jane Wairimu registered?

or send texts like:
where are civil servants outpatient services offered in kerugoya?

Relevant pivotal tracker stories

#51

RyanSept · 2017-10-14T23:59:40Z

healthtools/search/wit_ai.py

+    This checks the doc type against the doc list an determines whether it should be elastic search or nurses search.
+    If the doc type is empty,search type is None 
+    """
+    doc = ['nhif-outpatient', 'nhif-inpatient', 'nhif-outpatient-cs', 'doctors', 'health-facilities', 'clinical-officers']


A bit of a nitpick but lines 30-33 aren't pep8

Nitpick is so important in these uncertain times.. 💯

RyanSept · 2017-10-15T00:09:46Z

healthtools/views/search_api.py

+            'result': {'hits': [], 'total': 0},
+            'doc_type': doc_type,
+            'status': 'FAILED',
+            'msg': ''  # TODO: Pass run_query message here.


Why do we have this TODO here?

That's me. We should pass an actual message if this fails. Currently we don't have any message being returned so we only know it's failed but not why.

Wait, @Awinja-Andela why are these changes reflecting here? I thought I had already written this code.

@DavidLemayian i am reusing this code for wit.ai. I am using it just so that we can have two options for searching for the document name.
using wit.ai or using document.py
It is either i do this, or i remove document.py completely.

RyanSept · 2017-10-15T00:10:00Z

healthtools/views/search_api.py

+            'msg': ''  # TODO: Pass run_query message here.
+        })
+
+    # TODO: Log event here (send to Google Analytics)


This one too

This is me again. Notes in code to get back to. Showing where we should do stuff.

DavidLemayian · 2017-10-15T11:01:52Z

healthtools/views/search_api.py



 blueprint = Blueprint('search_api', __name__)


 @blueprint.route('/search', methods=['GET'], strict_slashes=False)
+def search_index(doc_type=None):


@Awinja-Andela Why are we creating a separate function for what's already below as "index"?

@DavidLemayian notice that i have introduced :
from healthtools.search.wit_ai import wit_run_query
while still retaining:
from healthtools.search import run_query

@blueprint.route('/wit_search', methods=['GET'], strict_slashes=False)
that uses:
result, doc_type = wit_run_query(query, doc_type)

and
@blueprint.route('/search', methods=['GET'], strict_slashes=False) @blueprint.route('/search/<doc_type>', methods=['GET'], strict_slashes=False)
that still uses:
result, doc_type = run_query(query, doc_type)

DavidLemayian · 2017-10-15T11:02:53Z

healthtools/settings.py

+
+#wit.ai access token
+
+access_token = os.getenv("access_token")


Formating and naming on these changes do not follow the rest of the project..

DavidLemayian · 2017-10-18T14:32:04Z

healthtools/search/wit_ai.py

+    resp = client.message(message_text)
+    query = ''.join(nested_lookup('value', resp['entities']['query']))
+    doc_type = ''.join([var for var in (resp['entities'].keys()) if var != 'query'])
+    #wit returns the entity with an underscore,this changes it to hyphen


When you see it.

DavidLemayian · 2017-10-18T14:34:41Z

healthtools/search/wit_ai.py

+            search_type = 'nurses'
+    else:
+        search_type = 'elastic'
+    return search_type


This is already defined in the DOCUMENTS. Why an entire function for it? https://github.com/CodeForAfricaLabs/HealthTools.API/blob/develop/healthtools/documents.py

DavidLemayian · 2017-10-18T14:35:31Z

healthtools/settings.py

+
+################################################################################
+
+#Wit.ai Access Token


When you see it.

DavidLemayian · 2017-10-18T14:45:03Z

@Awinja-Andela I can kind of see what's happening here now.

So as to avoid replication, let's approach this as a doc-type.

When you search using https://api.healthtools/search/wit?q=Dr+Ouma it then determines the doc type using wit.ai. Make sense?

Awinja-j · 2017-10-19T08:06:49Z

@DavidLemayian yes. i see. let me try it out.

Awinja-j · 2017-10-19T09:45:16Z

@DavidLemayian check it out!!!

DavidLemayian · 2017-10-25T13:18:09Z

This is shaping up nicely @Awinja-Andela. Now what I'd like is for determine_doc_type be the only function wit_ai runs. It should return only the query and doc_type. Makes sense?

Awinja-j · 2017-10-27T09:57:33Z

@DavidLemayian Makes Sense. I have changed it to return query and doc_type

Awinja-j · 2017-10-27T11:20:57Z

@DavidLemayian you will see a lot has changed now.
wit_ai.py is no more.
there's a new function in query.py
and run_query fucntion has changed to accomodate the new function
And search_api.py no longer checks if doc_type is 'wit'

Awinja-Andela added 6 commits October 11, 2017 08:08

wit.ai configurations

50c0b05

working search type and query

c6e5733

finall wit.ai commit

2d909a5

finall wit.ai commit

b7012a4

changed settings configurations

03969b4

added new search fucntion

a1791cc

RyanSept reviewed Oct 14, 2017

View reviewed changes

RyanSept reviewed Oct 15, 2017

View reviewed changes

DavidLemayian reviewed Oct 15, 2017

View reviewed changes

DavidLemayian suggested changes Oct 15, 2017

View reviewed changes

Awinja-j changed the title ~~Wit.ai~~ Added Wit.ai document type determiner Oct 16, 2017

Awinja-Andela added 5 commits October 16, 2017 12:20

pep8 changes

5305c9d

fixed pep 8 errors

919e5de

removed .idea file

1e58e66

new search url with wit

56c0ecc

added in line comment

8b0f526

DavidLemayian reviewed Oct 18, 2017

View reviewed changes

pulled upstream develop and changed inline comment

f8454ad

Awinja-Andela added 3 commits October 19, 2017 12:12

added search parameter url

bd9b636

removed new url, modified to use doc type instead

50e5a48

removed commented line

a348c5a

Awinja-Andela added 3 commits October 24, 2017 11:56

removed find_search_doc used doc_exist instead

8037ca9

updated read me with wit details

5944cfe

added images to gitignore

15cac71

correcting gramatical error

5c7310b

changed determine_doc_type

b873e12

intergrated wit.ai into runquery function

399a30c

Awinja-Andela and others added 2 commits October 27, 2017 14:25

removed commented line

8a87358

Update .gitignore

5f56da5

DavidLemayian approved these changes Oct 28, 2017

View reviewed changes

DavidLemayian merged commit a6aeeb9 into CodeForAfrica:develop Oct 28, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added Wit.ai document type determiner #63

Added Wit.ai document type determiner #63

Awinja-j commented Oct 13, 2017 •

edited

Loading

RyanSept Oct 14, 2017 •

edited

Loading

DavidLemayian Oct 15, 2017

RyanSept Oct 15, 2017

DavidLemayian Oct 15, 2017

DavidLemayian Oct 15, 2017

Awinja-j Oct 16, 2017

RyanSept Oct 15, 2017

DavidLemayian Oct 15, 2017

DavidLemayian Oct 15, 2017

Awinja-j Oct 16, 2017

DavidLemayian Oct 15, 2017

DavidLemayian Oct 18, 2017

DavidLemayian Oct 18, 2017

DavidLemayian Oct 18, 2017

DavidLemayian commented Oct 18, 2017

Awinja-j commented Oct 19, 2017

Awinja-j commented Oct 19, 2017

DavidLemayian commented Oct 25, 2017

Awinja-j commented Oct 27, 2017

Awinja-j commented Oct 27, 2017


		#wit.ai access token

		access_token = os.getenv("access_token")


		################################################################################

		#Wit.ai Access Token

Added Wit.ai document type determiner #63

Added Wit.ai document type determiner #63

Conversation

Awinja-j commented Oct 13, 2017 • edited Loading

This PR therefore introduces:

This task involved:

wit.ai is suitable for the /search endpoint and further SMS function

Relevant pivotal tracker stories

RyanSept Oct 14, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DavidLemayian commented Oct 18, 2017

Awinja-j commented Oct 19, 2017

Awinja-j commented Oct 19, 2017

DavidLemayian commented Oct 25, 2017

Awinja-j commented Oct 27, 2017

Awinja-j commented Oct 27, 2017

Awinja-j commented Oct 13, 2017 •

edited

Loading

wit.ai is suitable for the `/search` endpoint and further `SMS` function

RyanSept Oct 14, 2017 •

edited

Loading