Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new model for question-answer for comparison and improved code readability with comments #25

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions answer.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,25 @@
import warnings

import pandas as pd
from transformers.pipelines import pipeline

hg_comp = pipeline('question-answering', model="distilbert-base-uncased-distilled-squad", tokenizer="distilbert-base-uncased-distilled-squad")

# Defining model for given objective (question-answering)
hg_comp = pipeline('question-answering', model="distilbert-base-uncased-distilled-squad",
tokenizer="distilbert-base-uncased-distilled-squad")
hg_comp2 = pipeline('question-answering', model="bert-large-uncased-whole-word-masking-finetuned-squad",
tokenizer="bert-large-uncased-whole-word-masking-finetuned-squad")
# Import file with question and context
data = pd.read_csv('examples.csv')

# Loop to iterate over each row and find answers
for idx, row in data.iterrows():
context = row['context']
question = row['question']
# distilbert-base-uncased-distilled-squad
answer = hg_comp({'question': question, 'context': context})['answer']
# bert-large-uncased-whole-word-masking-finetuned-squad
answer2 = hg_comp2({'question': question, 'context': context})['answer']
print(question)
print(answer)
print(answer2)
print("\n")
7 changes: 7 additions & 0 deletions examples.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
question,context,answer
who won the match sri lanka or india?,"India won the Test series 1–0, after the first and third matches were drawn.[15] India won the ODI series 2–1, their eighth consecutive series win since beating Zimbabwe in June 2016.[16] India won the T20I series 3–0.[17]",India
"
what was the prison called in the green mile?","In 1935, Paul supervises officers Brutus Howell, Dean Stanton, Harry Terwilliger, and Percy Wetmore at Cold Mountain Penitentiary. Paul is suffering from a severe bladder infection and receives John Coffey, a physically imposing but mentally challenged black man, into his custody. John had been sentenced to death after being convicted of raping and murdering two white girls. One of the other inmates is a Native-American named Arlen Bitterbuck, who is charged with murder and is the first to be executed. Percy demonstrates a severe sadistic streak, but, as the nephew of Louisiana's First Lady, he is beyond reproach. He is particularly abusive with inmate Eduard Delacroix; he breaks Del's fingers with his baton, steps on a pet mouse named Mr. Jingles, which Del had adopted, repeatedly calls him by a gay slur, and ultimately sabotages his execution by failing to soak the sponge used to conduct electricity to Del's head; Del dies screaming in pain.",Cold Mountain Penitentiary
what is the widest highway in north america?,"King's Highway 401, commonly referred to as Highway 401 and also known by its official name as the Macdonald–Cartier Freeway or colloquially as the four-oh-one,[3] is a controlled-access400-series highway in the Canadian province of Ontario. It stretches 828.0 kilometres (514.5 mi) from Windsor in the west to the Ontario–Quebec border in the east. The part of Highway 401 that passes through Toronto is North America's busiest highway,[4][5] and one of the widest.[6][7] Together with Quebec Autoroute 20, it forms the road transportation backbone of the Quebec City–Windsor Corridor, along which over half of Canada's population resides and is also a Core Route in the National Highway System of Canada. The route is maintained by the Ministry of Transportation of Ontario (MTO) and patrolled by the Ontario Provincial Police. The speed limit is 100 km/h (62 mph) throughout its length, unless posted otherwise.",Highway 401
what makes the center of an atom stable?,"The electrons in an atom are attracted to the protons in the nucleus by the electromagnetic force. This force binds the electrons inside an electrostaticpotential well surrounding the smaller nucleus, which means that an external source of energy is needed for the electron to escape. The closer an electron is to the nucleus, the greater the attractive force. Hence electrons bound near the center of the potential well require more energy to escape than those at greater separations.",electromagnetic force
who did holly matthews play in waterloo rd?,"She attended the British drama school East 15 in 2005, and left after winning a high-profile role in the BBC drama Waterloo Road, playing the bully Leigh-Ann Galloway.[6] Since that role, Matthews has continued to act in BBC's Doctors, playing Connie Whitfield; in ITV's The Bill playing drug addict Josie Clarke; and she was back in the BBC soap Doctors in 2009, playing Tansy Flack.",Leigh-Ann Galloway