Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data analysis of defintion #189

Open
madebygps opened this issue Jan 19, 2024 · 40 comments
Open

data analysis of defintion #189

madebygps opened this issue Jan 19, 2024 · 40 comments

Comments

@madebygps
Copy link
Collaborator

madebygps commented Jan 19, 2024

  • most popular definitons by tag by month
  • most popular month of contributions
  • and any other ones you want to add
@madebygps madebygps changed the title data analysis data analysis of defintion Jan 19, 2024
@madebygps
Copy link
Collaborator Author

@rishp66

@rishp66
Copy link

rishp66 commented Jan 19, 2024

Additional Stuff:

  • after i create the ones for definitions, I'll make one for the daily definitions as well
  • ex: what days were most contributed out of the day
  • most popular categories, etc.

@rishp66
Copy link

rishp66 commented Feb 5, 2024

completed making a sentiment analysis of each definition within the database using NTLK library

@rishp66
Copy link

rishp66 commented Feb 5, 2024

will now group by tag and then will perform contribution by authors and aggregate for # of contributions made by author and their individual sentiment analysis before moving onto time section

@rishp66
Copy link

rishp66 commented Feb 5, 2024

issue with graphing on streamlit

# Set a dark background style for Seaborn
sns.set(style='darkgrid')

plt.figure(figsize=(10, 6))
sns.barplot(x='sentiment_score', y='tag', data=sentiment_by_tag, palette='viridis')
plt.title('Sentiment Scores by Tag')
plt.show()

# setting warning message to false 
st.set_option('deprecation.showPyplotGlobalUse', False)

# plotting graph
st.pyplot()

With the following code, I need to create a horizontal bar chart comparing the sentiment of each tag. In this case, I have done it with Seaborn so I'll continue doing that but I cannot find anything for a horizontal bar chart that would work. I tried looking it up here -- let me know if you find anything

@rishp66
Copy link

rishp66 commented Feb 6, 2024

completed analysis of tags for sentiment analysis -- will perform the same for individual contributors and authors before moving onto the time section for both

@madebygps
Copy link
Collaborator Author

amazing work tracking the progress Rish!!! @rishp66

@rishp66
Copy link

rishp66 commented Feb 7, 2024

@madebygps thanks! just added some new graphs and stuff -- going to work on authors next :)

@rishp66
Copy link

rishp66 commented Feb 20, 2024

finished the first introductory page for authors, had to take a break cuz of exams going on for uni -- gonna finish sentiment analysis soon for the authors & then move onto final portion of time

@rishp66
Copy link

rishp66 commented May 18, 2024

Completed the Authors page after some time haha

@rishp66
Copy link

rishp66 commented May 18, 2024

FInished reimporting libraries & now working on Time page - will be done hopefully soon 🤞

@rishp66
Copy link

rishp66 commented Jun 17, 2024

Got some time to work on the project and all pages are now done but there is an issue.

The project can only be run on localhost.

I have had significant issues trying to deploy this utilizing many help pages and online resources from the Streamlit docs, trying out Azure documentation and refactoring the code.
It will not take the credentials necessary to deploy it on the Streamlit cloud.
Here is the link to the GitHub repo: https://github.com/rishp66/dictionary_analysis

Screenshot 2024-06-16 at 11 12 18 AM

@rishp66
Copy link

rishp66 commented Jun 17, 2024

Above, there is the error code statement.

@madebygps
Copy link
Collaborator Author

@rishabkumar7 you're a bit more familiar with streamlit+python. Have you seen this error? If not @rishp66 we can do a debugging session on l2c discord to work through this.

@rishabkumar7
Copy link
Collaborator

@rishp66 are you trying to deploy it to Azure or Streamlit cloud, or you tried both and got the error.
Also, the second screenshot, where is that from?

@rishp66
Copy link

rishp66 commented Jun 17, 2024

@rishabkumar7 I've only tried to deploy to streamlit cloud so far. There's only 1 screenshot, it's the same error page that I took the screenshot of.

@madebygps Yeah, we can do debugging session. Tbh, it's just that the st.secrets() is not taking the secrets I input that work fine within the TOML file on local

@rishabkumar7
Copy link
Collaborator

@rishp66 and you are adding it in the advanced settings on Streamlit cloud interface?
image

I think it would we easier to debug on a discord call.

@rishp66
Copy link

rishp66 commented Jun 18, 2024

Yep exactly, my Python version is a 3.12 so I'm unsure if that's causing the issue.

@rishp66
Copy link

rishp66 commented Jun 18, 2024

@rishabkumar7 i'll send you a message on discord

@rishp66
Copy link

rishp66 commented Jun 24, 2024

Deployed first container on Docker Hub and encountered this error when trying to deploy to acr:

  "code": "ContainerAppOperationError",
  "message": "Failed to provision revision for container app 'datadefinedcloud'. Error details: The following field(s) are either invalid or missing. Field 'template.containers.datadefinedcloud.image' is invalid with details: 'Invalid value: \"docker.io/rp999/dictionaryanalysis:latest\": image OS/Arc must be linux/amd64 but found linux/arm64';.."

@rishp66
Copy link

rishp66 commented Jun 24, 2024

Reconfiguring and deploying container with datadefinedcloud to Docker Hub with this command:
docker buildx build --platform linux/amd64 -t datadefinedcloud .

@rishabkumar7
Copy link
Collaborator

So. you need to make sure, your docker image is built for linux/amd64

@rishp66
Copy link

rishp66 commented Jun 24, 2024

@rishabkumar7
Copy link
Collaborator

The link doesn't work.

@rishp66
Copy link

rishp66 commented Jun 24, 2024

fixed it :)

@rishabkumar7
Copy link
Collaborator

Yep, try deploying now, it should work.

@rishp66
Copy link

rishp66 commented Jun 24, 2024

That took about 3-4 hours so probably will work on deploying it to ACR later

@rishp66
Copy link

rishp66 commented Jun 24, 2024

Completed deployment to ACR:

@madebygps
Copy link
Collaborator Author

i removed your link, it has sub id in it, be careful with that moving forward

@rishp66
Copy link

rishp66 commented Jun 25, 2024

oh i see, my bad on that one :(

@rishabkumar7
Copy link
Collaborator

@rishp66 let me know once we have the ACA deployed.
Also, if you run into issue, feel free to reach out.

@rishp66
Copy link

rishp66 commented Jun 27, 2024

@rishabkumar7 The container app is up and running and can be accessed via a URL however it encounters an error after you put in the URL

@rishp66
Copy link

rishp66 commented Jun 27, 2024

ServiceRequestError: No connection adapters were found for [application url]

@madebygps
Copy link
Collaborator Author

what troubleshooting steps have you tried?

@rishabkumar7
Copy link
Collaborator

@rishp66 My thoughts:
So by default ACA ingress listens on port 443. For your container are you using the default streamlit port 8501?
If yes, add that as the target port in ACA settings. You can find that in the Ingress tab.
Screenshot 2024-06-27 at 9 35 53 AM

@rishp66
Copy link

rishp66 commented Jul 1, 2024

what troubleshooting steps have you tried?

I was on vacation since the 28th but I believe I tried looking at some websites:
https://www.slingacademy.com/article/fixing-python-requests-error-no-connection-adapters/
https://stackoverflow.com/questions/58866979/python-no-connection-adapters-were-found

Neither of these and the ones I found were I think relevant to the issue here.

@rishp66
Copy link

rishp66 commented Jul 1, 2024

@rishabkumar7 Ahhh I see, okay, I will change that when I can and follow up with an update on how it's going.

@rishp66
Copy link

rishp66 commented Jul 13, 2024

@rishabkumar7 I checked it out, and I reconfigured the target ports has been changed to 8501 but the same error appears of
ServiceRequestError: No connection adapters were found for [application url]

@rishp66
Copy link

rishp66 commented Jul 14, 2024

Within looking at the error codes, it has something to do within the Azure Core library within the pipeline file. I did not find any relevant articles explaining how to perform anything regarding this error so I used Claude. It had said that it had something with the SDK being out of date so I performed the following command:

pip3 install --upgrade azure-cosmos

@rishp66
Copy link

rishp66 commented Jul 14, 2024

This stated that all requirements were already satisfied. However, I am unsure as what to do further since the only other options it gives are importing the requests library which I do not believe is necessary or changing network/firewall settings on the CosmosDB resource.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

No branches or pull requests

3 participants