An app to simplify web scraping for books and other 'written' media
Add configurations to the configuration screen to inform the app what to scrape.
The app will target using the domain (example.com) rather than an entire url (https://example.com), so please use the domain of the website instead of full urls to the website.
| Path type | Usage |
|---|---|
| Title path | Is optional (first title element by default). A selector to get the title of a page, whether that is a chapter title or article title. |
| Content Path | A selector for the actual chapter / page contents that you want to read. It can handle both divs and individual p tags if needs be. Other text requires the usage of $text or similar. |
| Previous path | The path to the link for the previous chapter on a page. This can handle <a> tag selection as well if the href attribute is not explicitly selected. |
| Next Path | The path to the link for the next chapter on a page. This can handle <a> tag selection as well if the href attribute is not explicitly selected. |
Identical to the normal configurations, except instead of a domain, there is a name for the configuration as well as the match path.
| Path type | Usage |
|---|---|
| Config Name | A name to uniquely identify the general configuration. No restrictions on characters or format. |
| Match path | A path that matches any element on the page. If the element exists, the scraper assumes that the general configuration for the page is valid. Please make the match path select a unique element of the page (not present in websites that don't match the template). |
| Configuration | Usage |
|---|---|
| Use web view for viewing this website | A toggleable option for using the built in android webview in order to view a website in its native format, with css and all. |
| The separator between p tags | Configure the separator between paragraph tags. Use \n in place of line breaks and \t in place of tabs. You can even use normal characters as separators if you truly want to. |
| Animated Autoscroll | A toggleable option to remove the animation for scrolling down to the reading position of the chapter. With this removed, scrolling down to the saved scroll location is instant. |
| Enable headless mode | Turn on headless mode. This uses a builtin browser to obtain html from a website. This approach increases compatibility with all sorts of websites in exchange for a bit of speed and the ability to preemptively load content. |
If you add a $ after a normal css path, it will target that attribute.
a.chapter $href will target the href attribute of an <a> tag with the class 'chapter'.
For paths, the href attribute is prioritized, so if you want to get the text content of the <a> tag, you can use the modifier $text.
Xpaths also have the ability to use $ syntax. This is generally not needed, however, because they allow you to target individual attributes by default.
Note that books require a booklist to be created first in order to function.
| Domain Name | Title Path (Not required) | Content Path | Previous Path | Next Path | Headless Mode | Image Url Path |
|---|---|---|---|---|---|---|
| scribblehub.com | div.chapter-title | #chp-raw | a.btn-prev | a.btn-next | Yes | .s_novel_img > img $src |
| m.fanfiction.net | N/A | div#storycontent | //a[text()[contains(., 'Prev')]] $href | //a[text()[contains(., 'Next')]] $href | Yes | N/A |
| Config Name | Match Path | Title Path (Not required) | Content Path | Previous Path | Next Path | Headless Mode | Image Url Path |
|---|---|---|---|---|---|---|---|
| Madara WP Theme | .wp-manga-template-default | N/A | div.text-left | a.prev_page | a.next_page | Yes | N/A |
| Base WordPress Theme | body.post-template-default | N/A | div.entry-content | div.nav-previous > a | giv.nav-next > a | Either | N/A |