Can no longer view archived pages

[jweber714]

Ever since the 01-28-25 update, I get a "can't find page" error when I try to open an archived page. I backed up my Firefox profile. I noticed my backend was not the latest version, so I downloaded and installed the latest version following the instructions exactly. However, when I checked the Backend tab in Scrapyard, it still shows I have the older version installed. Now all of my archived folders and pages are gone. HELP !

[GChristensen]

Hi, what operating system and browser you are using?
Please, check if the path in the settings still points to the correct directory and the files are still there.

[jweber714]

I distinctly remember going through the Transition process a few versions ago when it first became available. When this issue occurred a few days ago, the Transition tab showed up. I checked my current version of the backend. Current version was 2.0. Latest version was 2.1.2. In Settings, the Content folder box was blank and the Content location showed browser internal storage. I checked the folder I had set up the last time I did the transition; it is empty! So, I can only assume I have been unknowingly using the browser's internal storage up to this point and everything was working fine. I installed the 2.1.2 and the installer confirmed the installation was successful. When I went back to look at the Backend tab, it showed the old 2.0 was still my current version. So, all my data must be in my Firefox profile, but only you can tell me how to actually confirm its exact location. Very puzzling to me!

[GChristensen]

The path to the profile in Firefox could be found on the about:support page. When installing the backend application it is necessary to exit Firefox. If Firefox was running during the installation, it may be worth to try again.
The situation when you have your pages in the sidebar, using browser internal storage, and the pages are unavailable, is of nearly impossible probability, so more context is needed. What operating system you are using?

[jweber714]

I am already aware of the path to my profile. As I said earlier, I backed up my profile prior to doing anything. And yes, I followed the instruction exactly; Firefox was closed before installing. I also have tried installing several times. Each time, the installer confirms the installation was successful. Yet, when I check the Setting tab in Scrapyard, it still shows I am using the old version of the backend (2.0). I am using Windows 11. So I can at least confirm I still have my pages, exactly where are they located within my profile??

[GChristensen]

To check if pages are there it is necessary to get the id of the add-on. The easiest way is from the URL of the settings page: moz-extension:///ui/options.html. In the profile there should be the folder storage/default/moz-extension+++. The pages are stored in the internal database and they are still there if the size of the folder is large enough.

The new version of the extension may be not installed if Firefox still resides in the memory after it was closed which is not uncommon, so it worth to check this before the installation or install just after reboot (the new Firefox may also have habit to start just after boot).
It is also worth to completely uninstall the previous backend manually before installing the new one and check if the extension itself is of the last version.
One more thing, what exactly is the URL of the unfound pagge?

[jweber714]

Ok, I deleted the 2.0 version of the backend. Then I checked the Settings tab in Scrapyard and it was gone. I rebooted my PC and made sure Firefox was not running. I then installed the 2.1.2 version backend. I started Firefox and checked the Settings tab and it now shows I have 2.1.2 installed!

To check if pages are there it is necessary to get the id of the add-on. The easiest way is from the URL of the settings page: moz-extension:///ui/options.html.

I don't understand what you are saying here. What is the ID of the add-on? The settings page in Scrapyard??? I tried entering the URL in the Firefox address bar, but it went nowhere.

> In the profile there should be the folder storage/default/moz-extension+++

I did find several "moz-extension+++" folders in my profile. Screenshot attached. Are these folders what you are talking about?

One more thing, what exactly is the URL of the unfound page?

I can't provide the URL for any of the lost pages; they are ALL gone.

Assuming the data is in the "moz-extension+++" folders, can I just click on the "Rebuild Item Index" link in the Advanced tab in Scrapyard???? Or, isn't it that simple?

[GChristensen]

It seems that the tag was eaten by the GitHub markup. I've meant extension://<extension id>/ui/options.html. As I understand, you have no pages in the sidebar. So the problem is that there is no pages in the sidebar or the pages in the sidebar are inaccessible?

[GChristensen]

Either way, I'd restore the backup of the profile that worked in a portable Firefox on or using Firefox profile manager, then export the "Everything" shelf as a json file, then in the correct/recent Firefox profile i'd switch to filesystem storage and import the file.

It is only possible to say is there a hope, if the extension folder has large enough size, e.g. several hundreds megabytes.

[jweber714]

It seems that the tag was eaten by the GitHub markup. I've meant extension:///ui/options.html. As I understand, you have no pages in the sidebar. So the problem is that there is no pages in the sidebar or the pages in the sidebar are inaccessible?

At first, the pages were there, just inaccessible after the 01-28-25 update. Later, after trying to update the backend, no pages displayed in the sidebar. It is now empty.

[jweber714]

Either way, I'd restore the backup of the profile that worked in a portable Firefox on or using Firefox profile manager, then export the "Everything" shelf as a json file, then in the correct/recent Firefox profile i'd switch to filesystem storage and import the file.

It is only possible to say is there a hope, if the extension folder has large enough size, e.g. several hundreds megabytes.

I only have a backup of my current profile that I backed up several days ago to my desktop. Scrapyard had already stopped working BEFORE I made the backup. There are no other profile backups.

So, If I follow your instructions; I would copy the backed up profile on my desktop, and paste it into my profile folder (effectively overwriting my current profile that now contains new/additional data from the last several days activity in Firefox. Then, open Scrapyard and export the Everything shelf as a json file. Then, while still in Scrapyard, go to the Settings tab, leave the Content folder box blank (or should I specify a location first?), change the Content location to Filesystem and import the json file. Is this correct???

Would it also be wise to backup the current profile before I do this?

[GChristensen]

Firefox could be launched with the -profilemanager parameter. You may create a new profile, specify your old profile folder, launch with it, export Scrapyard data if any, and switch to the new profile.

[jweber714]

OK, I have finally got Scrapyard up and running with all of my archived page accessible again. My Content is stored in the browser internal storage for now.

Here is how I did it. I restored the profile I was using when the problem began. When I opened Scrapyard for the first time, the folder tree was there, but when I clicked on a saved page, it displayed a message “Transition required” or something to that effect. I clicked the Transition “shortcut/link” and it opened the Transition tab in the Settings page. I followed the instructions all the way through, but when I checked the Content folder location I had set, to confirm the transfer worked, it was empty! I went ahead and went to the Advanced tab and Reset internal storage, but it still didn’t work. Can you tell me why my content didn't transfer to my specified folder???

So then I deleted the profile and copied the backed up copy back into the profile folder a second time and started over. This time, when the Transition Required page appeared, I clicked on the link to keep the data in the browser internal storage; and not use the filesystem location. Now Scrapyard is working as it should. Once the browser internal storage is selected, the Transition tab disappears. How can I get the Transition tab back if I later want to transition to Filesystem storage???

Then I tried to backup Scrapyard. I went to the Backup tab, specified a backup directory, and clicked the Backup button. The progress bar appeared and finished. I went to the directory I specified and it was also empty! I tried the backup with and without the zip box checked; same result; the folder location was still empty.

[GChristensen]

Without seeing the logs, I cant' say why the directory is empty. It worth to try with the process described above - export the Everything shelf to a jsbk file, then switch to the filesystem storage, then import the jsbk file. Import and export entries are available at the shelf menu.

[GChristensen]

In the recent versions of the extension there is no backup tab. The backup option is obsolete now and is located at the Advanced tab. It is also worth to check if autoupdate is disabled for the extension.

[jweber714]

Where can I find the logs?

My mistake; I accessed the backup option on the Advanced tab.

[GChristensen]

The debug logs of the client part is at the about:devtools-toolbox?id=scrapyard@firefox&type=extension page, on the console tab. They are volatile and reset each time the Add-on or Frefox restart. The backend logs should be enabled manually at /ui/options.html#debug page (the link to it is available at the advanced tab, the debugging should be enabled). But I wouldn't bother with it, just export and import first.