Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat:Speech to text added #114

Merged
merged 42 commits into from
Sep 8, 2023
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
4c6e22b
feat: mic button working
prtkjakhar May 11, 2023
6bb88c6
feat: audio to speech added
geeky-abhishek May 11, 2023
404acc1
chore: ui changes
geeky-abhishek May 11, 2023
85e6002
chore: ui changes
geeky-abhishek May 11, 2023
2d64dc5
feat: speech to text added to chat ui
geeky-abhishek May 11, 2023
69e0ccb
Merge branch 'main' into hindi-stt
geeky-abhishek May 11, 2023
632a02c
added flagsmith for model ids
prtkjakhar May 15, 2023
a05e265
Merge branch 'main' into hindi-stt
prtkjakhar May 15, 2023
7008843
feat: changed audio models
prtkjakhar May 25, 2023
c577bd4
added flagsmith cloud
prtkjakhar Jun 2, 2023
b5af3e7
feat: added new api for odia audio
prtkjakhar Jun 5, 2023
0cbd5a3
feat: added new api for odia audio
prtkjakhar Jun 5, 2023
d5ca333
feat: added voice button to chat screen
prtkjakhar Jun 8, 2023
7d94748
added ai tools odia stt
prtkjakhar Aug 21, 2023
2e94ebc
yarn issues resolved
prtkjakhar Aug 21, 2023
e66df08
Style/new UI (#159)
prtkjakhar Aug 23, 2023
e165b6c
removed yarn lock file
prtkjakhar Aug 23, 2023
78a5932
Merge branch 'main' into hindi-stt
prtkjakhar Aug 23, 2023
a2361fe
bug fix
prtkjakhar Aug 23, 2023
190e63e
bug fix
prtkjakhar Aug 23, 2023
111489f
bug fix
prtkjakhar Aug 23, 2023
ca1eff7
download share added to history page
prtkjakhar Aug 23, 2023
d4daa19
added asr id and sharing toasts
prtkjakhar Aug 24, 2023
efbe106
added toasts in sharing
prtkjakhar Aug 24, 2023
85a9168
Added error handling for no audio
prtkjakhar Aug 29, 2023
f54698e
modified recorder
prtkjakhar Aug 31, 2023
b67f17e
audio working on mobile
prtkjakhar Aug 31, 2023
6d4c006
Fix socket (#160)
prtkjakhar Sep 1, 2023
a9a5dc5
audio working
prtkjakhar Sep 1, 2023
b9a4e6b
socket connection first time issue fixed
prtkjakhar Sep 5, 2023
6bd3524
new audio package used (#161)
prtkjakhar Sep 5, 2023
a5f1164
Used web audio APIs directly (#162)
prtkjakhar Sep 5, 2023
87635ad
Style new UI (#164)
prtkjakhar Sep 6, 2023
472c693
downtime state fixed
prtkjakhar Sep 6, 2023
65d1f23
stop gif size reduced
prtkjakhar Sep 6, 2023
c2ae3aa
recorder error msg changed
prtkjakhar Sep 6, 2023
310b961
added timer1 back
prtkjakhar Sep 7, 2023
56dc49e
refresh odia translation added
prtkjakhar Sep 7, 2023
82e36dc
increased fallback values for timer1 timer2
prtkjakhar Sep 7, 2023
506cca9
removed transliteration
prtkjakhar Sep 7, 2023
44684a2
title changed
prtkjakhar Sep 7, 2023
6809a34
changed odia title spelling
prtkjakhar Sep 8, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions apps/amakrushi/lang/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,8 @@
"message.review_description": "Please write your experience's feedback.",
"message.temporarily_down": "We're temporarily down",
"message.temporarily_down_description": "We are experiencing high user volume at the moment, please try logging in after some time",
"message.recorder_wait": "Please wait while we process your request...",
"message.recorder_error": "Unable to process your request at the moment. Please try after sometime.",
"message.dialer_popup":"You may speak to an Ama Krushi expert to get a satisfactory response",
"error.fail_to_submit":"Failed to submit rating.",
"error.fail_to_submit_review": "Failed to submit review.",
Expand Down
2 changes: 2 additions & 0 deletions apps/amakrushi/lang/or.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@
"message.review_description": "ଦୟାକରି ଆପଣଙ୍କର ଅଭିଜ୍ଞତା ମତାମତ ଲେଖନ୍ତୁ |",
"message.temporarily_down": "କିଛି ସମୟ ପାଇଁ ସର୍ଭର ଡାଉନ୍ ଅଛି |",
"message.temporarily_down_description": "ଆମେ ଏହି ମୁହୂର୍ତ୍ତରେ ଉଚ୍ଚ ଉପଭୋକ୍ତା ଏହାକୁ ବ୍ୟବହାର କରୁଥିବା ର‌ ଅନୁଭବ କରୁଛୁ | ଦୟାକରି କିଛି ସମୟ ପରେ ଲଗଇନ୍ କରିବାକୁ ଚେଷ୍ଟା କରନ୍ତୁ|",
"message.recorder_wait": "ଆମେ ତୁମର ଅନୁରୋଧ ପ୍ରକ୍ରିୟା କରୁଥିବାବେଳେ ଦୟାକରି ଅପେକ୍ଷା କର...",
"message.recorder_error": "ଏହି ସମୟରେ ଆପଣଙ୍କର ଅନୁରୋଧ ପ୍ରକ୍ରିୟାକରଣ କରିବାରେ ଅସମର୍ଥ | ଦୟାକରି କିଛି ସମୟ ପରେ ଚେଷ୍ଟା କରନ୍ତୁ |",
"message.dialer_popup":"ଏକ ସନ୍ତୋଷଜନକ ପ୍ରତିକ୍ରିୟା ପାଇବା ପାଇଁ ଆପଣ ଆମା କ୍ରୁଶି ବିଶେଷଜ୍ଞଙ୍କ ସହିତ କଥା ହେଇପାରିବେ |",
"error.fail_to_submit":"ମୂଲ୍ୟାୟନ ଦାଖଲ କରିବାରେ ବିଫଳ |",
"error.fail_to_submit_review": "ସମୀକ୍ଷା ଦାଖଲ କରିବାରେ ବିଫଳ |",
Expand Down
2 changes: 2 additions & 0 deletions apps/amakrushi/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,15 @@
"@emotion/react": "^11.7.1",
"@emotion/styled": "^11.6.0",
"@magicbell/magicbell-react": "^8.5.3",
"@material-ui/core": "^4.12.4",
"@next/bundle-analyzer": "^13.3.0",
"@storybook/react": "^6.5.9",
"@testing-library/jest-dom": "^5.16.2",
"@testing-library/react": "^10.4.9",
"@testing-library/user-event": "^12.8.3",
"@types/jest": "^25.2.3",
"@types/node": "^12.20.46",
"audio-react-recorder": "^1.0.4",
"axios": "^1.3.5",
"bootstrap": "^5.1.3",
"chatui": "*",
Expand Down
1 change: 1 addition & 0 deletions apps/amakrushi/src/assets/icons/startIcon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added apps/amakrushi/src/assets/icons/stop.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions apps/amakrushi/src/assets/icons/stopIcon.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion apps/amakrushi/src/components/HistoryPage/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ const HistoryPage: NextPage = () => {
})
.then((res) => {
const sortedConversations = _.filter(
res?.data,
res?.data?.userHistory,
(conv) => conv?.conversationId !== null
).sort(
//@ts-ignore
Expand Down
3 changes: 2 additions & 1 deletion apps/amakrushi/src/components/HomePage/index.module.css
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
}

.inputBox {
height: 36px;
height: fit-content;
display: flex;
text-align: center;
position: absolute;
Expand Down Expand Up @@ -79,3 +79,4 @@
border-radius: 40px !important;
background-color: var(--secondarygreen) !important;
}

9 changes: 8 additions & 1 deletion apps/amakrushi/src/components/HomePage/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import { logEvent } from 'firebase/analytics';
import Menu from '../menu';
import { getInitialMsgs } from '../../utils/textUtility';
import { AppContext } from '../../context';

import RightIcon from '../../assets/icons/right';
import sunIcon from '../../assets/icons/sun.svg';
import reloadIcon from '../../assets/icons/reload.svg';
Expand All @@ -23,6 +24,7 @@ import Image from 'next/image';
import { Button } from '@chakra-ui/react';
import toast from 'react-hot-toast';
import { v4 as uuidv4 } from 'uuid';
import RenderVoiceRecorder from '../recorder/RenderVoiceRecorder';

const HomePage: NextPage = () => {
const context = useContext(AppContext);
Expand Down Expand Up @@ -102,6 +104,9 @@ const HomePage: NextPage = () => {
})}
<form onSubmit={(event) => event?.preventDefault()}>
<div className={styles.inputBox}>
<div>
<RenderVoiceRecorder setInputMsg={setInputMsg}/>
</div>
<input
type="text"
value={inputMsg}
Expand All @@ -111,10 +116,12 @@ const HomePage: NextPage = () => {
<button
type="submit"
onClick={() => sendMessage(inputMsg)}
className={styles.sendButton}>
className={styles.sendButton}
>
{t('label.send')}
</button>
</div>

</form>
</div>
<Menu />
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,13 @@ import { useLocalization } from '../../../hooks';
import { getMsgType } from '../../../utils/getMsgType';
import ChatMessageItem from '../../chat-message-item';
import { v4 as uuidv4 } from 'uuid';
import toast from 'react-hot-toast';
import RenderVoiceRecorder from '../../recorder/RenderVoiceRecorder';
import DownTimePage from '../../down-time-page';

const ChatUiWindow: React.FC = () => {
const t = useLocalization();
const context = useContext(AppContext);
const context = useContext(AppContext);

useEffect(() => {
const fetchData = async () => {
Expand Down Expand Up @@ -103,7 +105,7 @@ const ChatUiWindow: React.FC = () => {
})),
[context?.messages]
);

console.log("fghj:",{messages:context?.messages})
const msgToRender = useMemo(() => {
return context?.isMsgReceiving
? [
Expand All @@ -120,6 +122,7 @@ const ChatUiWindow: React.FC = () => {
console.log('debug:', { msgToRender });

const placeholder = useMemo(() => t('message.ask_ur_question'), [t]);

if (context?.isDown) {
return <DownTimePage />;
} else
Expand Down
3 changes: 2 additions & 1 deletion apps/amakrushi/src/components/down-time-page/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,5 @@ function DownTimePage() {
);
}

export default DownTimePage;
export default DownTimePage;

Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import C from '../../../constants';

const action = (updatedState) => {
return {
type: C.UPDATE_BENCHMARK_FILTER,
payload: updatedState
}
};

export const initialSearchFilter = () => {
return {
type: C.INITIAL_BENCHMARK_FILTER
}
};

export default action;
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/**
* BenchmarkDetails API
*/
import API from "../../../api";
import C from "../../../constants";
import ENDPOINTS from "../../../../../configs/apiendpoints";

export default class BenchmarkDetails extends API {
constructor(benchmarkId, timeout = 2000) {
super("GET", timeout, false);
this.endpoint = `${super.apiEndPointAuto()}${
ENDPOINTS.benchmarkDetails
}?benchmarkId=${benchmarkId}`;
this.type = C.GET_BENCHMARK_DETAILS;
}

toString() {
return `${super.toString()} email: ${this.email} token: ${
this.token
} expires: ${this.expires} userid: ${this.userid}, type: ${this.type}`;
}

processResponse(res) {
super.processResponse(res);
if (res) {
this.report = res;
}
}

apiEndPoint() {
return this.endpoint;
}

getBody() {
return {}
}

getHeaders() {
this.headers = {
headers: {
"Content-Type": "application/json",
},
};
return this.headers;
}

getPayload() {
return this.report;
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/**
* Model Search API
*/
import API from "../../../api";
import C from "../../../constants";
import CONFIGS from "../../../../../configs/configs";
import ENDPOINTS from "../../../../../configs/apiendpoints";
import md5 from 'md5';

export default class SearchModel extends API {
constructor(task = "translation", sourceLanguage = null, targetLanguage = [], domain = 'All', submitter = 'All', timeout = 2000) {
super("POST", timeout, false);
this.task = task;
this.sourceLanguage = sourceLanguage;
this.targetLanguage = targetLanguage;
this.domain = domain;
this.submitter = submitter
this.endpoint = `${super.apiEndPointAuto()}${ENDPOINTS.benchmarkModelSearch}`;
this.userDetails = JSON.parse(localStorage.getItem('userInfo'));
this.type = C.BENCHMARK_MODEL_SEARCH;
}

toString() {
return `${super.toString()} email: ${this.email} token: ${this.token} expires: ${this.expires} userid: ${this.userid}, type: ${this.type}`;
}



processResponse(res) {
super.processResponse(res);
if (res) {
this.report = res.data;
}
}

apiEndPoint() {
return this.endpoint;
}

getBody() {
let bodyData = {
task: this.task,
sourceLanguage: this.sourceLanguage,
targetLanguage: this.targetLanguage,
domain: this.domain,
submitter: this.submitter
}
bodyData.userId = localStorage.getItem('userDetails') && JSON.parse(localStorage.getItem('userDetails')).userID
return bodyData;
}


getHeaders() {
this.headers = {
headers: {
"Content-Type": "application/json",

}
};
return this.headers;
}


getPayload() {
return this.report;

}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
import C from '../../../constants';

const action = (updatedState) => {
return {
type: C.UPDATE_SEARCH_FILTER,
payload: updatedState
}
};

export const initialSearchFilter = () => {
return {
type: C.INITIAL_SEARCH_FILTER
}
};

export default action;
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
/**
* Model Search API
*/
import API from "../../../api";
import C from "../../../constants";
import CONFIGS from "../../../../../configs/configs";
import ENDPOINTS from "../../../../../configs/apiendpoints";
import md5 from 'md5';

export default class SearchModel extends API {
constructor(modelId, timeout = 2000) {
super("GET", timeout, false);
this.endpoint = `${super.apiEndPointAuto()}${ENDPOINTS.benchmarkTable}?modelId=${modelId}`;
this.type = C.BENCHMARK_TABLE;
}

toString() {
return `${super.toString()} email: ${this.email} token: ${this.token} expires: ${this.expires} userid: ${this.userid}, type: ${this.type}`;
}



processResponse(res) {
super.processResponse(res);
if (res) {
this.report = res;
}
}

apiEndPoint() {
return this.endpoint;
}

getBody() {
return {};
}


getHeaders() {
this.headers = {
headers: {
"Content-Type": "application/json",

}
};
return this.headers;
}


getPayload() {
return this.report;

}
}
Loading