Skip to content

Commit

Permalink
Minor fixes and readme update + todo edit
Browse files Browse the repository at this point in the history
  • Loading branch information
0AwsD0 committed May 13, 2024
1 parent 313430c commit 7afe6ba
Show file tree
Hide file tree
Showing 3 changed files with 33 additions and 27 deletions.
31 changes: 9 additions & 22 deletions download.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,25 +10,9 @@ def download(**data):

#below code checks if the basic option is selected - if yes -> the universal function will be invoked | if no -> the advanced functions will be invoked - tailored to the data that is for set game type
if data["mode"] == "basic":
try:
basic_download(data["url"], data["name"], data["tags"])
except:
print("basic_download() - failed")
print("Is website down?")
print("Did you input proper values?")
print("Exiting in 5 seconds.")
time.sleep(5)
exit(1)
basic_download(data["url"], data["name"], data["tags"])
else:
try:
advanced_download(**data)
except:
print("advanced_download() - failed")
print("Is website down?")
print("Did you input proper values?")
print("Exiting in 5 seconds.")
time.sleep(5)
exit(1)
advanced_download(**data)


def basic_download(url, name, tags):
Expand Down Expand Up @@ -119,8 +103,9 @@ def basic_download(url, name, tags):
exit(0)
print("Getting url...")
url2 = url
print("Got url - adding page number...")
url2 += "&page=" + str(i)
print("Got url - adding page number...")
print("Url: " + url2)
print("Waiting 5 seconds for page to load...")
driver.get(url2)
time.sleep(5)
Expand Down Expand Up @@ -155,7 +140,8 @@ def advanced_download(**data):
#I need to let user config wait time and provide some values or idk explain it in documentation and readme.md
#OR
#I may try using watchdog or directory scan to check if file exists -> the card and waiting until it does so there is no need for download interval - it downloads only if previous card is downloaded
print("WARNING: Some cards weigh a lot, even over 25MB - if your intewrnet is slow CHANGE WAIT TIME between card downloads to even 10 seconds!")
print("WARNING: Some cards weigh a lot, even over 25MB - if your internet is slow CHANGE WAIT TIME between card downloads to even 10 seconds!")
print("The corresponding SLEEP functions in code have comments above, surrounded by #.")

url = data["url"]
name = data["name"]
Expand Down Expand Up @@ -193,8 +179,8 @@ def advanced_download(**data):
gender_input = Select(driver.find_element(By.ID, "gender"))
if (gender != ""):
gender_input.select_by_value(gender)
else:#for test
print("gender > empty")#for test
#else:#for test
#print("gender > empty")#for test
except:
print("gender not found in datased - skipping")

Expand Down Expand Up @@ -286,6 +272,7 @@ def advanced_download(**data):
url2 = url
print("Got url - adding page number...")
url2 += "&page=" + str(i)
print("Url: "+url2)
print("Waiting 5 seconds for page to load...")
driver.get(url2)
time.sleep(5)
Expand Down
19 changes: 17 additions & 2 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,22 @@ I'm creating this application to learn some python and have it as a backup means
## How it works
- The page for selected game is loaded and form is filled by the selenium.
- After search button clicked, program waits 5 seconds for website to load.
- It will download card every 4 seconds, to prevent multiple downloads, since the website doesn't like multiple downloads. (Some cards can be over 25MB that's why it's better to give it more than less time) //I may make function that check if the card download is finished than proceed to next card in the future.
- It will download card every 5 seconds, to prevent multiple downloads, since the website doesn't like multiple downloads. (Some cards can be over 25MB that's why it's better to give it more than less time) //I may make function that check if the card download is finished than proceed to next card in the future.
- If the page had less than 24 cards on it, it's the last page (or only one) -> Download Finished
- If page have 24 cards but the "Next" button is disabled -> Download Finished
- If the button is active -> change url and load page -> start downloading cards, untill there is less than 24 cards on page or the "Next" button is disabled.
- If the button is active -> change url and load page -> start downloading cards, untill there is less than 24 cards on page or the "Next" button is disabled.

## Info for setup
- Some cards weigh a lot, even over 25MB - if your internet is slow - CHANGE WAIT TIME - between card downloads to even 20 seconds!
- The corresponding SLEEP functions in code have comments, surrounded by #. Just like shown below:
```
######################################
#BELOW TIME FOR CARD TO BE DOWNLOADED#
######################################
```
+The known issue is -> malwarebytes will kill the program if you run it from PyCharm, and possibly in any other form. Disable it and try using it again. //It likes to kill python scripts.

## How to ensure all cards ware downloaded
- to be sure that program downloaded all the cards you can use this formula: ((number of pages) - 1) * 24 + (number of cards on last page)
- for 7 pages, when last contains 4 cards it would loo like: 6*24+4 = 148 cards

10 changes: 7 additions & 3 deletions todo.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ It will be able to select the download mode too!
The plan is to even add auto download - you create file with all parameters and don't need to type it into console!
The program will do it for you!

~Thats the plan will see when or if I'm gonna implement that
~That's the plan will see when or if I'm gonna implement that or any other functionality mentioned in this file, since for me - the current state is enough.

+ better card download -> checking if card is downloaded, and only then downloading next card
+ packing app into exe
+ better card download -> checking if card is downloaded if it's not or server went down for a second - retry (i = i), and only then downloading next card (i += 1)
+ if server went down for a second and first card couldn't be downloaded - retry loading page at least 1 time
+ packing app into exe
+ selecting download location

+START AT PAGE - when website goes down or the download is stopped // also before download check if card exists in download folder to skip already downloaded cards

0 comments on commit 7afe6ba

Please sign in to comment.