Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added TTS #195

Merged
merged 98 commits into from
Oct 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
8d21714
Style: fixed odia title (#104)
prtkjakhar May 9, 2023
beec8fe
Feat chat history (#106)
geeky-abhishek May 9, 2023
69137b3
Delete .env.production
geeky-abhishek May 9, 2023
61e535f
UI changes (#112)
geeky-abhishek May 10, 2023
fb95fbc
fix: conversation id (#115)
geeky-abhishek May 11, 2023
3c16d52
Merge branch 'production' into main
geeky-abhishek May 11, 2023
f9192d4
Added down time page (#117)
prtkjakhar May 11, 2023
091ed6e
Update ContextProvider.tsx (#118)
prtkjakhar May 11, 2023
585157e
Feat dialer popup (#120)
prtkjakhar May 12, 2023
939e319
Merge branch 'production' into main
geeky-abhishek May 12, 2023
ad9f44b
Added Auth header (#122)
prtkjakhar May 12, 2023
425947c
Merge branch 'production' into main
geeky-abhishek May 15, 2023
5db5d56
chore: msgId display for testing (#127)
geeky-abhishek May 16, 2023
ecfa55c
Feat dsply msg (#128)
geeky-abhishek May 16, 2023
e8e157c
Merge branch 'production' into main
geeky-abhishek May 17, 2023
129d6eb
chore: fixed userID bug (#130)
prtkjakhar May 23, 2023
4af2c31
Feat fcm (#133)
prtkjakhar May 23, 2023
b4e863b
fix: update in conversation api (#135)
prtkjakhar May 29, 2023
5caf66f
fix: added firebase event for msg delay (#136)
prtkjakhar May 29, 2023
cd31282
fix: flagsmith init inside useeffect (#140)
geeky-abhishek Jun 1, 2023
5e0760b
Merge branch 'production' into main
geeky-abhishek Jun 1, 2023
8b9f282
Added query log and splash screen log (#143)
prtkjakhar Jun 2, 2023
3c79987
Feat trans (#134)
prtkjakhar Jun 5, 2023
6796a5b
Merge branch 'production' into main
geeky-abhishek Jun 5, 2023
8defe56
Fix: fcm logic updated (#146)
prtkjakhar Jun 7, 2023
8eb2122
Feat: Added example questions to flagsmith (#147)
prtkjakhar Jun 8, 2023
bf0929e
Feat: Feature Pop Up (#148)
prtkjakhar Jun 12, 2023
eaaf5c0
Merge branch 'production' into main
geeky-abhishek Jun 12, 2023
0dfb454
fix: empty feature details bug (#152)
prtkjakhar Jun 12, 2023
13d1d3b
Merge branch 'production' into main
geeky-abhishek Jun 12, 2023
80c4a8b
fix: handled new history api (#157)
prtkjakhar Jun 21, 2023
27cfafd
fix: no internet error for otp verify in offline (#158)
geeky-abhishek Jun 28, 2023
9ad530b
feat:Speech to text added (#114)
geeky-abhishek Sep 8, 2023
4508a91
Fix chat history (#166)
prtkjakhar Sep 13, 2023
0052a3c
modified label.ask_me in or (#167)
prtkjakhar Sep 14, 2023
f59e057
Minor UI fix (#168)
prtkjakhar Sep 14, 2023
9d15259
Minor UI fix (#169)
prtkjakhar Sep 18, 2023
a5159b1
Merge branch 'production' into main
prtkjakhar Sep 18, 2023
3898b8f
Fix odia translations (#171)
prtkjakhar Sep 18, 2023
e4be1dc
welcome text centered (#173)
prtkjakhar Sep 20, 2023
fce0f6f
chat history pdf api updated
prtkjakhar Sep 20, 2023
b4ec921
Merge branch 'production' into main
prtkjakhar Sep 20, 2023
96b75be
pdf opening
prtkjakhar Sep 21, 2023
117aab8
Merge branch 'main' of https://github.com/AmakrushAI/app
prtkjakhar Sep 21, 2023
c6e2c53
Merge branch 'production' into main
prtkjakhar Sep 21, 2023
f508a27
pdfurl defined two times fixed
prtkjakhar Sep 21, 2023
c40debe
added target in anchor
prtkjakhar Sep 21, 2023
84c1c07
added object url back
prtkjakhar Sep 21, 2023
e3f600f
Merge branch 'production' into main
prtkjakhar Sep 21, 2023
fd2a6d7
added pdfUrl back
prtkjakhar Sep 21, 2023
33e177d
merge main
prtkjakhar Sep 21, 2023
b315a6f
Merge branch 'production' into main
prtkjakhar Sep 21, 2023
6d05890
added toast message if no pdf link
prtkjakhar Sep 21, 2023
c958285
minor fixes
prtkjakhar Sep 21, 2023
48e0aa7
Merge branch 'main' of https://github.com/AmakrushAI/app
prtkjakhar Sep 21, 2023
26282ed
Merge branch 'production' into main
prtkjakhar Sep 21, 2023
1accc3f
fixed manual link
prtkjakhar Sep 21, 2023
f0cec7a
added android handshake for sharing
prtkjakhar Sep 21, 2023
cbcc2d1
android handshake function added
prtkjakhar Sep 21, 2023
7bee713
android handshake function added
prtkjakhar Sep 21, 2023
99d8b62
android handshake added
prtkjakhar Sep 21, 2023
d756020
added android handshake function
prtkjakhar Sep 21, 2023
7d2bd79
minor ui fix
prtkjakhar Sep 21, 2023
5e512d7
updated chat history url
prtkjakhar Sep 25, 2023
9eed839
downloaded pdf opening in new tab
prtkjakhar Sep 26, 2023
60d9ac5
added firebase logs for download, share clicks
prtkjakhar Sep 26, 2023
770c89d
Merge branch 'production' into main
prtkjakhar Sep 26, 2023
dc0f2cd
added error handling if shareUrl not implemented in android
prtkjakhar Sep 26, 2023
5d4494b
Merge branch 'production' into main
prtkjakhar Sep 26, 2023
cab4ad2
Updated shareUrl error message in both langs
prtkjakhar Sep 26, 2023
6e99b75
Merge branch 'production' into main
prtkjakhar Sep 26, 2023
a18507c
sending phoneNumber in asr API
prtkjakhar Sep 27, 2023
91b7c4e
ui enhancements
prtkjakhar Oct 6, 2023
e508b87
tts working
prtkjakhar Oct 6, 2023
5528887
changes for flutter
prtkjakhar Oct 9, 2023
c6ae1ab
changes for flutter
prtkjakhar Oct 9, 2023
848938f
changes for flutter
prtkjakhar Oct 9, 2023
8a35ff3
changes for flutter
prtkjakhar Oct 9, 2023
9bf45b9
added tts loader
prtkjakhar Oct 9, 2023
df95f10
added flutter handshake
prtkjakhar Oct 10, 2023
a859f0c
flutter handshake changed
prtkjakhar Oct 10, 2023
e80f123
flutter handshake changed
prtkjakhar Oct 10, 2023
ba80ae0
flutter handshake functions changed
prtkjakhar Oct 10, 2023
729d7fb
flutter handshake functions changed
prtkjakhar Oct 10, 2023
5e30de3
flutter handshake functions changed
prtkjakhar Oct 10, 2023
c0449bc
flutter handshake functions changed
prtkjakhar Oct 10, 2023
eef08bf
flutter handshake functions changed
prtkjakhar Oct 10, 2023
172bb7f
changed flutter handshake
prtkjakhar Oct 10, 2023
08c49a0
added audio playback from flagsmith
prtkjakhar Oct 10, 2023
0ee4fc1
added flutter download channel
prtkjakhar Oct 11, 2023
df101bf
added flutter download channel
prtkjakhar Oct 11, 2023
01a7c5b
removed tts
prtkjakhar Oct 11, 2023
a501ca6
Merge branch 'production' into main
prtkjakhar Oct 11, 2023
f7b2c16
removed tts
prtkjakhar Oct 11, 2023
4affe40
added tts
prtkjakhar Oct 11, 2023
f7b6a70
added pause icon to tts
prtkjakhar Oct 17, 2023
218e8ee
added incrementaudioused api
prtkjakhar Oct 17, 2023
96bf7a8
Merge branch 'production' into main
prtkjakhar Oct 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions apps/amakrushi/src/assets/icons/speakerPause.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions apps/amakrushi/src/components/NavBar/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ function NavBar() {
const newConversationId = uuidv4();
sessionStorage.setItem('conversationId', newConversationId);
if(context?.audioElement) context?.audioElement.pause();
if(context?.setAudioPlaying) context?.setAudioPlaying(false);
context?.setConversationId(newConversationId);
context?.setMessages([]);
context?.setIsMsgReceiving(false);
Expand Down
18 changes: 13 additions & 5 deletions apps/amakrushi/src/components/chat-message-item/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import { analytics } from '../../utils/firebase';
import { logEvent } from 'firebase/analytics';
import RightIcon from '../../assets/icons/right.jsx';
import SpeakerIcon from '../../assets/icons/speaker.svg';
import SpeakerPauseIcon from '../../assets/icons/speakerPause.svg';
import MsgThumbsUp from '../../assets/icons/msg-thumbs-up.jsx';
import MsgThumbsDown from '../../assets/icons/msg-thumbs-down.jsx';
import { AppContext } from '../../context';
Expand Down Expand Up @@ -158,11 +159,11 @@ const ChatMessageItem: FC<ChatMessageItemPropType> = ({

const handleAudio = (url: any) => {
// console.log(url)
if(url === ''){
if (url === '') {
toast.error('No audio');
return;
}
context?.playAudio(url);
context?.playAudio(url, content);
};

const { content, type } = message;
Expand Down Expand Up @@ -284,19 +285,26 @@ const ChatMessageItem: FC<ChatMessageItemPropType> = ({
&nbsp;
<p>{t('message.helpful')}</p>
</div>
{/* <div
<div
className={styles.msgSpeaker}
onClick={() => handleAudio(content?.data?.audio_url || '')}>
{context?.clickedAudioUrl === content?.data?.audio_url ? (
context?.ttsLoader ? (
<Loader />
) : (
<Image src={SpeakerIcon} width={15} height={15} alt="" />
<Image
src={
!context?.audioPlaying ? SpeakerIcon : SpeakerPauseIcon
}
width={!context?.audioPlaying ? 15 : 40}
height={!context?.audioPlaying ? 15 : 30}
alt=""
/>
)
) : (
<Image src={SpeakerIcon} width={15} height={15} alt="" />
)}
</div> */}
</div>
</div>
)
)}
Expand Down
1 change: 1 addition & 0 deletions apps/amakrushi/src/components/menu/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ const Menu: FC = () => {

const urlChanger = (link: string) => {
if(context?.audioElement) context?.audioElement.pause();
if(context?.setAudioPlaying) context?.setAudioPlaying(false);
if (cookies['access_token'] !== undefined) {
if(link === '/history' && context?.loading){
toast.error(`${t("error.wait_new_chat")}`);
Expand Down
136 changes: 86 additions & 50 deletions apps/amakrushi/src/context/ContextProvider.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -55,97 +55,129 @@ const ContextProvider: FC<{
);
const timer1 = flagsmith.getValue('timer1', { fallback: 30000 });
const timer2 = flagsmith.getValue('timer2', { fallback: 45000 });
const audio_playback = flagsmith.getValue('audio_playback', { fallback: 1.5 });
const audio_playback = flagsmith.getValue('audio_playback', {
fallback: 1.5,
});
const [isDown, setIsDown] = useState(false);
const [showDialerPopup, setShowDialerPopup] = useState(false);
// const [isConnected, setIsConnected] = useState(newSocket?.connected || false);
const [cookie, setCookie, removeCookie] = useCookies();
const [isOnline, setIsOnline] = useState(navigator.onLine);
const [audioElement, setAudioElement] = useState(null);
const [ttsLoader, setTtsLoader] = useState(false);
const [audioPlaying, setAudioPlaying] = useState(false);
const [clickedAudioUrl, setClickedAudioUrl] = useState<string | null>(null);

const downloadChat = useMemo(() => {
return (e: string) => {
try{
try {
//@ts-ignore
downloadHandler.postMessage(e);
}catch(err){
console.log(err)
} catch (err) {
console.log(err);
}
}
};
}, []);

const shareChat = useMemo(() => {
return (e: string) => {
try{
try {
//@ts-ignore
shareUrl.postMessage(e);
}catch(err){
console.log(err)
} catch (err) {
console.log(err);
}
}
};
}, []);

const playAudio = useMemo(() => {
return (url: string) => {
if (!url) {
console.error('Audio URL not provided.');
return;
}
if (audioElement) {
return (url: string, content: any) => {
if (!url) {
console.error('Audio URL not provided.');
return;
}
if (audioElement) {
//@ts-ignore
if (audioElement.src === url) {
// If the same URL is provided and audio is paused, resume playback
//@ts-ignore
if (audioElement.src === url) {
// If the same URL is provided and audio is paused, resume playback
//@ts-ignore
if (audioElement.paused) {
setClickedAudioUrl(url);
setTtsLoader(true);
if (audioElement.paused) {
setClickedAudioUrl(url);
setTtsLoader(true);
audioElement
//@ts-ignore
audioElement.play().then(() => {
.play()
.then(() => {
setTtsLoader(false);
setAudioPlaying(true);
console.log('Resumed audio:', url);
//@ts-ignore
}).catch((error) => {
})
//@ts-ignore
.catch((error) => {
setAudioPlaying(false);
setTtsLoader(false);
setAudioElement(null);
setClickedAudioUrl(null);
console.error('Error resuming audio:', error);
});
} else {
// Pause the current audio if it's playing
//@ts-ignore
audioElement.pause();
console.log('Paused audio:', url);
}
return;
} else {
// Pause the older audio if it's playing
// Pause the current audio if it's playing
//@ts-ignore
audioElement.pause();
setAudioPlaying(false);
console.log('Paused audio:', url);
}
return;
} else {
// Pause the older audio if it's playing
//@ts-ignore
audioElement.pause();
setAudioPlaying(false);
}
setClickedAudioUrl(url);
setTtsLoader(true);
const audio = new Audio(url);
audio.playbackRate = audio_playback;
audio.play().then(() => {
}
setClickedAudioUrl(url);
setTtsLoader(true);
const audio = new Audio(url);
audio.playbackRate = audio_playback;
audio.addEventListener('ended', () => {
setAudioElement(null);
setAudioPlaying(false);
});
axios
.get(
`${process.env.NEXT_PUBLIC_BASE_URL}/incrementaudioused/${content?.data?.messageId}`
)
.then((res) => {})
.catch((err) => {
console.log(err);
});
audio
.play()
.then(() => {
setTtsLoader(false);
setAudioPlaying(true);
console.log('Audio played:', url);
}).catch((error) => {
// Update the current audio to the new audio element
//@ts-ignore
setAudioElement(audio);
})
.catch((error) => {
setAudioPlaying(false);
setTtsLoader(false);
setAudioElement(null);
setClickedAudioUrl(null);
console.error('Error playing audio:', error);
});

// Update the current audio to the new audio element
//@ts-ignore
setAudioElement(audio);
};
}, [audioElement]);
};
}, [audioElement, audio_playback]);

useEffect(() => {
console.log("online")
console.log('online');
if (navigator.onLine) {
console.log("online")
console.log('online');
setIsOnline(true);
} else {
console.log("online")
console.log('online');
setIsOnline(false);
onMessageReceived({
content: {
Expand All @@ -159,7 +191,7 @@ const ContextProvider: FC<{
messageId: uuidv4(),
});
}
// eslint-disable-next-line react-hooks/exhaustive-deps
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [navigator.onLine]);

useEffect(() => {
Expand Down Expand Up @@ -466,7 +498,9 @@ const ContextProvider: FC<{
setTtsLoader,
shareChat,
clickedAudioUrl,
downloadChat
downloadChat,
audioPlaying,
setAudioPlaying,
}),
[
locale,
Expand All @@ -493,7 +527,9 @@ const ContextProvider: FC<{
setTtsLoader,
shareChat,
clickedAudioUrl,
downloadChat
downloadChat,
audioPlaying,
setAudioPlaying,
]
);

Expand Down