Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discussion of desktop version #1

Closed
2 tasks done
andrewtavis opened this issue Jan 14, 2022 · 48 comments
Closed
2 tasks done

Discussion of desktop version #1

andrewtavis opened this issue Jan 14, 2022 · 48 comments
Assignees
Labels
good first issue Good for newcomers help wanted Extra attention is needed question Further information is requested

Comments

@andrewtavis
Copy link
Member

andrewtavis commented Jan 14, 2022

Terms

Issue

Scribe's focus will eventually shift from Scribe-iOS to Scribe-Android, as there already is significant interest in a port to Android. With that being said, there does seem to be the potential for desktop and browser versions of Scribe. The current ideas revolve around the following:

  • The Scribe preview bar could be positioned in a corner of the desktop or browser screen and always be at the front of the in-use applications
  • The controls for it could be positioned in the desktop menu bar or extension panel of a browser
  • Annotation would be displayed to a user as it is in the mobile versions (as soon as a space is typed, then it is displayed)
  • There could be buttons for Scribe commands that the user could press to then type within the preview bar, with return then executing the command (an enter button could also be included in the interface)
  • There could also be keyboard shortcuts to then type within the preview bar, which the user could set themselves

This issue is to discuss the potential of a desktop version of Scribe. Ideas and suggestions, including technology that could be used, would be more than welcome. Ideas for browser support could also be included or shifted to a discussion in scribe-org/Organization.

Before work can begin, all language logic for Scribe-iOS should be shifted over to a common repository that Scribe-Desktop can also reference. This requirement corresponds to Scribe-iOS issue #10.

@andrewtavis andrewtavis added good first issue Good for newcomers help wanted Extra attention is needed question Further information is requested labels Jan 14, 2022
@andrewtavis
Copy link
Member Author

andrewtavis commented Jan 15, 2022

Initial impressions would be that this would be written in Python.

@andrewtavis
Copy link
Member Author

andrewtavis commented Apr 3, 2022

Tkhinter is the baseline Python GUI package, and Toga, PySimpleGUI as well as DearPyGui also seem to be useful for GUIs in Python.

@andrewtavis
Copy link
Member Author

Something to consider is how to get the finished app onto the given app stores, as it won't be written in Swift/Xcode for example where the App Store Connect process is defined. Useful resources for this are:

Getting Scribe-Desktop onto Homebrew would be worth it as well.

@andrewtavis
Copy link
Member Author

andrewtavis commented Apr 11, 2022

Tkhinter is the baseline Python GUI package, and Toga, PySimpleGUI as well as DearPyGui also seem to be useful for GUIs in Python.

Tkhinter has the most tutorials for GUIs that accept keyboard inputs as well. For annotation purposes new words could be read in after the space bar is pressed, with commands then being bound to hotkeys or to pressing the icon in the GUI.

@andrewtavis
Copy link
Member Author

Points that come to mind on the process going forward:

  • Figuring out how to get what the user is typing read into a currently running GUI app (I expect not too hard)
  • Checking what the last word was after a space has been pressed and reacting to it for noun genders and preposition cases (I expect easyish)
  • Allowing for a mouse click to then type into the GUI (I expect easy)
  • Figuring out how to get what the user has typed and executed back into the currently running app/text field (I expect hard)
  • From there we'd just be sending out dictionary values from the JSONs that are generated by Scribe-Data (I expect easyish)

@mrnninster
Copy link

Hi @andrewtavis
I would like to contribute to this
I have built 4 desktop applications with pysimpleGUI
one of which is a simple language text translator
using the google translate api I also have background in machine learning.

I have attached a copy of the app
Translator.zip
this is the code repo https://github.com/mrnninster/Translator

For the browser I would suggest using it as a plugin, similar to how the
blackbox plugin has been implemented

No matter the route you decide to take, I will be happy to work on any.

@andrewtavis
Copy link
Member Author

Hi @mrnninster! Thanks so much for reaching out! 😊 This is so great to hear, as I really do think that Scribe Desktop would be a very useful thing for people - especially if they're working in a second language!

You seem to be the very kind of person we'd need for this project. Really is a pleasure to meet you 🤝

Sending along the message to welcome you and thank you for your interest. I'll be in touch in the coming days as my focus for this weekend will be Scribe-iOS, but let's get at this later. If you have ideas, then it'd be great to hear them, and I'll definitely take a look at the GUI you've written later. We'll also talk a bit more about Scribe-Desktop in the Wikimania session tomorrow at 14:30 UTC.

@mrnninster
Copy link

Nice, I would like to attend if possible.
I also checked out the android version and it seems
you are implementing in kotlin, I am new to kotlin, less than a month
so I am not sure i can make any significant contribution with that, but
I will look around try to pick one.

@andrewtavis
Copy link
Member Author

andrewtavis commented Aug 12, 2022

Hopefully you can attend :)

For Kotlin and android, I'm also super new and really am looking for people who want to learn with me 😊 You're welcome to take a look! Wikimedia is super interested in an Android version, and people who will work on it will doubtless be presenting the progress at future events.

The most pressing issue to work on is Scribe-Android/issues/22. Scribe-Android is the combination of two open-source projects: one basic keyboard app and another one that has helper functions. The ones that has helper functions added a lot of unnecessary code, so it'd be best to just go through and remove everything that's not needed. Basically what that issue is is all the functions that are needed to run the keyboard app. We need to check what's necessary for those, and if it's not then let's remove it :) There are also some other really nice good-first-issues for it, so just take a look and know that you're very welcome to try things out to learn!

Happy to check in about the above issue or anything else during the hackathon. Just reach out 😊

@andrewtavis
Copy link
Member Author

@mrnninster, hi again :) There's another person from Wikimania who has contacted me about working on the Android app. Would you maybe want to do a call with them and we can discuss the project a bit more? I think that that would be the best place for us to focus. Happy to discuss desktop as well if you really would prefer working here.

@mrnninster
Copy link

@andrewtavis yes I'd love to jump on a call, like I said before the desktop software development is my forte, and I am completely new to kotlin, I am still working my way through the course I am taking, I can start out on the desktop and when I feel like I am knowledgeable enough join you on the android development, but if you'd prefer we all focus on the android app then I will go through the course as fast as I can and join you.

@andrewtavis
Copy link
Member Author

Ok, great! Let me check with the others really quickly and I'll be in touch :)

Hope you're well!

@andrewtavis
Copy link
Member Author

@mrnninster, if you would send an email to the one in my profile, I'll contact you in regards to planning for Android :)

@wkyoshida
Copy link
Member

Ah! 🙌 Okay - I hadn't gotten to reading through the discussion here before. I feel that it might have clarified some things I was wondering over in this convo.

Honestly, I didn't yet have a clear picture, and I was under the impression before that Scribe-Desktop would also be providing a pack of keyboards on top of providing the GUI. I guess I got used to thinking in packs due to Scribe-iOS 🤦😆 Maybe disregard some of the thoughts I had on the physical constraints with Scribe-Desktop. I'm thinking what could make sense is to simply allow the user to set whatever keyboard layout they'd like - since those would already be provided as an option by their OS. I believe that is probably what you already had in mind. The hovering GUI idea then is interesting in that it could provide the Scribe functionality while listening to just regular keyboard input. However, I'm not 100% sure if, with this approach, Scribe would have to think about if this affects policy around use of user data and system access - would be something to double-check.

I am thinking though that priority for a non-browser implementation would likely make more sense, at least to me. I feel that it's likely that users would want the Scribe functionality to stay present as they work outside the browser, e.g. in Microsoft Word and etc.


Thinking about Scribe-Desktop just suddenly came to me yesterday in that other convo, but I had really only skimmed through its repo. Things are clearer to me now 🙌😆 Scribe-Desktop will definitely be interesting as it will involve different challenges from its mobile counterparts.

@andrewtavis
Copy link
Member Author

andrewtavis commented Oct 27, 2022

I believe that is probably what you already had in mind.

Yes, exactly :) :)

However, I'm not 100% sure if, with this approach, Scribe would have to think about if this affects policy around use of user data and system access - would be something to double-check.

Yes our policy of definitely not recording any kind of user input would be challenged a bit. We just need to make sure that the way that this is written assures that we're not saving anything, but then there are doubtless further considerations for this 🤔

I am thinking though that priority for a non-browser implementation would likely make more sense, at least to me.

Exactly 😊 I think that a GUI that works in whatever app they're using is the best way to maximize the effect of the app. Maybe a browser extension some day, but that's a wholly different conversation.

Scribe-Desktop will definitely be interesting as it will involve different challenges from its mobile counterparts.

Yes, exactly :) There are interesting challenges, but then a GUI also takes away a lot of the headaches of Scribe-iOS and allows us to really just focus on getting the information to the user in an intuitive manner 😊 Plus Python 🐍😊

What are your thoughts on the status of this though, @wkyoshida? I've always been thinking that Android comes before Desktop, but I'm happy to be challenged on that.

@wkyoshida
Copy link
Member

but then a GUI also takes away a lot of the headaches of Scribe-iOS and allows us to really just focus on getting the information to the user in an intuitive manner 😊

Agreed!

What are your thoughts on the status of this though, @wkyoshida? I've always been thinking that Android comes before Desktop, but I'm happy to be challenged on that.

Hmm.. I think some points to consider could be (in favor of...):

  • Android My feeling is that there might be more need for the mobile versions in the sense that a user could more likely be in a poor-connection situation with a mobile phone than with a laptop/desktop, which might be relevant since offline use is one of Scribe's strengths. A user would more likely be on a mobile device when in-transit. Use of a laptop/desktop could also experience poor connection of course, but a user is more likely to be in a stationary location and not be as dependent on cellular coverage or Wi-Fi points.
  • Android With mobile apps, there is always that aspect of supporting the two major platforms, iOS and Android. Having the Android option would of course appeal to the currently neglected side of the mobile userbase.
  • Android I feel that the workflow of a user, for instance, referencing Google Translate or other resources for the target language is likely an easier experience in laptop/desktop. Trying to do the same while attempting to type something in a mobile device might not be as easy. In laptop/desktop, a user can have multiple tabs/windows on the screen together and use keyboard shortcuts to move around. The need for an easier workflow might be more present on mobile.
  • Desktop To the point above, however, a counter-point could also be that, on mobile, the need to 100% accurately follow all grammar rules might not be as strong as one would experience on laptop/desktop due to "texting vocabulary". Vernacular is more relaxed. Though, the translation feature could likely remain useful still.
  • Desktop As there is still some design being thought through for iOS, there could be reason to hold off on Android a bit. Mostly, I think, since Android will likely for the most part simply mirror the functionality in iOS. If Scribe decides to change something in the mobile versions, it would have to change for both. Desktop is more on an island; development could happen slightly more separately.
  • TIE I guess it also could just come down to what the community seems to be voicing more desire for. From what you might have heard from the community already, @andrewtavis, do you feel there's a clear winner between the two?

@andrewtavis
Copy link
Member Author

Great points above, @wkyoshida! As expected 😊 The pretty clear desire so far has been for Android. It’s just a matter of getting it up and running. I think that once we have SQLite and a menu for iOS, we’d then be ready to really start the conversion to Android 🤔

@andrewtavis
Copy link
Member Author

Hopefully we can get the menu and data solution fixed in the next few months. That seems reasonable to me 😊

@wkyoshida
Copy link
Member

Gotchu! Yeah - that sounds good to me.

Also - since you mentioned, I believe I remember you mentioned planning to get the opinion of someone on the implementation of the data solution. I was just curious what came from that. We can continue the conversation on scribe-org/Scribe-iOS#96 as well if it makes more sense to keep the initial discussion for the data solution there.

@andrewtavis
Copy link
Member Author

Yes, let's keep it there :) Good call to remind me to share that 😊

@wkyoshida
Copy link
Member

You know - since Rust's been on your radar, @andrewtavis.. I feel it could potentially be an option here actually 👀 - if having a project in Rust sounds like an interesting idea

I see people saying good things about tauri

@andrewtavis
Copy link
Member Author

Ya this might be a fun thing to do with Rust, @wkyoshida! Plus be the looks of it we could do a simple Vue frontend for it? @SaurabhJamadagni and I talked briefly about Scribe-Server and doing it in Go sounds good to us :)

@wkyoshida
Copy link
Member

Plus be the looks of it we could do a simple Vue frontend for it?

That's my understanding!
Great for us - as we now have some Vue experts amongst us! 👀😆

talked briefly about Scribe-Server and doing it in Go sounds good to us :)

whoop whoop!! 🥳

@SaurabhJamadagni
Copy link
Collaborator

Whoa Scribe-Desktop!! 🤯

Going through the discussion. Wow have I fallen behind haha.

@andrewtavis
Copy link
Member Author

Not until after Android 😅

@andrewtavis
Copy link
Member Author

andrewtavis commented Nov 1, 2023

Hey there @rti 👋 As mentioned in the office today, it would be great if you could spell out a bit more some of your thoughts on tauri :) Ping @RyanPaulGannon who's interested in working on this 😊

An overview of what we'd like to do here so we can figure out what tauri can and can't do easily such that we can check if it's still possible to adopt:

  • Scribe-Desktop would be a GUI that would sit above other apps
    • GUI Width is flexible and text height is directly related to the GUI height
  • It would need keyboard access and would read in the recently typed strings as Scribe does and display noun-gender annotations (der, die, das), etc to the user as they type
    • As in Scribe-iOS, we'd check what prior strings are after the user presses space
  • The ideal user experience would be that users would activate commands via keyboard shortcuts
    • We'd still need the option for selecting commands, so pressing the Scribe Key on the GUI would bring up the translate, conjugate and plural options (with shortcuts so they learn)
    • They'd then enter the command as in the iOS app's command bar, and then hitting enter would either insert the result or change the GUI UI to match the needed interface (see conjugate below)
  • It ideally would be able access to the text that is selected so that the user can do Scribe commands on selected text (not critical, but would limit what we can do)

I did some quick designs on Figma (of course I did 😅) so we can have something to look at:

Again let's talk about what limitations for tauri might be, and then if @RyanPaulGannon wants to look into checking them to see if they can be overcame then that'd be a fun project 😊 Thanks all!

@andrewtavis
Copy link
Member Author

@RyanPaulGannon, based on what @rti is saying we do have a good idea of what we'd need to check for a proof of concept. Going through the suggested steps, we might be able to assume that the hotkeys can work for Wayland at some point. Rendering relative to the cursor isn't something too important, but is interesting to think about 🤔 Getting it on top of others should be doable, but we should check this, Big thing is:

  • Reading in text from other applications
  • Entering in text into other applications

Could you do a check for whether these two things could be possible? Ultimately the desktop GUI that can be used cross platform is still the ideality so we can have it all in one place, but if you also want to check ways of implementing a browser extension in Rust we could also think on this 😊

@wkyoshida
Copy link
Member

Thank you as well for the input @rti! 🚀 Appreciate the thoughts! Good to hear some same concerns I've had coming from other folks as well

@andrewtavis, a proper full desktop version would be the dream (I think so, at least 😆), but I agree that revisiting a browser-extension version could be an easier initiative. It's always been an idea from the start really. I'd love to keep the idea of a full desktop version alive though, with the dream that folks could potentially use Scribe with things like MS Word and such.. Maybe we have to split off and have a Scribe-Browser repo? 🤣 not sure (but not opposed hehe).. We should definitely still do some investigating first into how feasible a desktop version would be.

P.S. The browser extension idea would end up being more of a JavaScript project though, no?

@andrewtavis
Copy link
Member Author

Yes certainly, @wkyoshida :) I was just opening the idea what maybe a bit of it could be Rust as I know @RyanPaulGannon has interest in it, but for web we'd be doing JS/TS for sure 😇 My question would be if we'd be able to do Scribe-Browser in one repo or if we'd need Scribe-Firefox and Scribe-Chromium. Would be great if we could deploy from one codebase, but not sure on this :)

@RyanPaulGannon
Copy link
Collaborator

Appreciate the help here all! I'll make a start on looking at the feasibility of both and will report back when I have something 😃

@andrewtavis
Copy link
Member Author

Sounds good, @RyanPaulGannon! Let us know if and when questions come up :)

@RyanPaulGannon
Copy link
Collaborator

RyanPaulGannon commented Nov 14, 2023

Just a small update, I've started something super basic using the gtk-rs toolkit to create few keyboard buttons (not currently tasking in key strokes), but just to see how it is to work with vs Tauri. Nothing remotely exciting - yet 😄

@andrewtavis
Copy link
Member Author

Thanks for the update, @RyanPaulGannon! Let us know if you have any questions or would like some feedback :)

@AnonymousCodes911
Copy link

Hi @andrewtavis,

I'm interested in contributing to this project. I'm fluent in Python and eager to learn more about it. Could you provide more information about this project and confirm if there are any opportunities to contribute?

Thank you for your time

@andrewtavis
Copy link
Member Author

andrewtavis commented Feb 11, 2024

Hi @AnonymousCodes911 :) The readme for this project is a bit out of date. The general plan right now is that we'll do the desktop version on Rust, with @RyanPaulGannon working on a proof of concept right now. If you had interest in learning some Rust for this, then we can keep you in the loop for when we're ready to start making issues for it. Aside from this on the Python end of things, you'd be welcome to look into Scribe-Data!

@AnonymousCodes911
Copy link

Thank you for the quick response,
I will surely look into Rust and Scribe-Data.

@andrewtavis
Copy link
Member Author

Thanks for you flexibility, @AnonymousCodes911! Great to have you here :)

@andrewtavis
Copy link
Member Author

Do we want to use this issue to plan out the ideas for other issues, and then we can close it when we've made those new issues? :) What issues were you thinking of making, @RyanPaulGannon?

@RyanPaulGannon
Copy link
Collaborator

RyanPaulGannon commented Sep 4, 2024

Okay, so, we need:

  • lib.rs splitting up as we've got a style struct in there, this will probably be better in it's own file
  • Link up with SQLite or add in a json to get some suggestions in
  • Add in more of the keycodes in to lib.rs, we could probably split this up too. I don't know if rdev has all the codes we need (I suspect it will, but I'll have to check the docs)

@andrewtavis
Copy link
Member Author

Do you want to make issues for these, @RyanPaulGannon, or should I make them and check in with you on them afterwards?

@RyanPaulGannon
Copy link
Collaborator

I won't pretend to be busier than any one else, so I'm happy to do them 😅 do we have a template? If not, would you mind doing one and I'll follow that with the rest. If we do, I'm cool doing them and checking with yourself 😄

@andrewtavis
Copy link
Member Author

There are templates set up for the repo already, @RyanPaulGannon :) Feel free to make them and I'll let you know if anything should be included in future ones. Big thing is, if you can add designs from Figma in the future, then that would be good, but for right now just explaining what you're looking for and under what contexts it will be "done" is good 😊

@RyanPaulGannon
Copy link
Collaborator

Would you mind creating the SQLite issue? I don't think I know enough detail to fully get across exactly what is required for that :)

@andrewtavis
Copy link
Member Author

Just made #12 :) Any other issues that we should make, or are we good to close this?

@RyanPaulGannon
Copy link
Collaborator

Thanks! I would say we're good to close this now :)

@andrewtavis
Copy link
Member Author

Issue #1 closed! Thanks so much for the hard work @RyanPaulGannon, and also for all the conversations to get to this point, all! 🚀

@github-project-automation github-project-automation bot moved this from In Progress to Done in Scribe Board Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers help wanted Extra attention is needed question Further information is requested
Projects
Archived in project
Development

No branches or pull requests

7 participants