unable to use standalone Firefox profile with Ruby bindings

[chrismcmahon]

What I am trying to accomplish is to create a Firefox Profile with two non-default aspects: a personal certificate installed (the "cert9.db" file in the Profile), and the preference "security.default_personal_cert" set to "Select Automatically" (the "prefs.js" file in the Profile). Having created this Profile in MacOS, I want have a different Firefox instance running in a docker-selenium container on a Linux machine use this Profile.

According to the geckodriver documentation http://firefox-source-docs.mozilla.org/testing/geckodriver/Profiles.htmlCustom profiles can be provided to Firefox two different ways:

1. by appending --profile /some/location to the args capability, which will instruct geckodriver to use the profile in-place;
2. or by setting the profile capability to a Base64-encoded ZIP of the profile directory.

If this document is in fact true, I would prefer to do that second one. I have not been able to do that myself, nor have I found any evidence that anyone has done this in Ruby since approximately 2016. There is some evidence that other languages have done this, although evidence after the advent of marionette and geckodriver circa 2016 is sparse.

Examples are using watir, but I traced all the errors to the selenium bindings. This is what I understand about launching Firefox with Profiles with watir currently (watir 6.16.5, selenium-webdriver 3.142.6): we can choose among existing profiles by doing

@browser = Watir::Browser.new :firefox, marionette: true, profile: "demoprofile"
where "demoprofile" has been created in advance. Note that @browser = Watir::Browser.new :firefox, marionette: true, profile: "bogus" yields an error "unable to find profile: "bogus""

@browser = Watir::Browser.new :firefox, marionette: true, profile: "/full/path/to/ifkw017g.default-release.zip.b64"
yields the same error "unable to find profile: "full/path/to/ifkw017g.default-release.zip.b64""

I've traced this error to here:

selenium/rb/lib/selenium/webdriver/firefox/profile.rb

Line 48 in 2fd646c

raise Error::WebDriverError, "unable to find profile named: #{name.inspect}"

@browser = Watir::Browser.new :firefox, marionette: true, args: ["-profile", "/full/path/to/ifkw017g.default-release.zip.b64"]
gets us a running browser but the arg is ignored

After that, I edited the Firefox profiles.ini file directly to read

[Profile0]
Name=default-release
IsRelative=0
Path=/path/to/ifkw017g.default-release.zip.b64

This yields the error "Not a directory..." I have traced this error to the procedure somewhere near here in profile-helper.rb which is interestingly marked "#TODO: must be a better way"

selenium/rb/lib/selenium/webdriver/common/profile_helper.rb

Line 55 in 2fd646c

# TODO: must be a better way..

I have been unable to discover any other place in the Ruby bindings for Firefox where Profiles are addressed. I may have missed something though.

A "Zipper" function exists but it is not clear how we could use this in Watir. https://github.com/SeleniumHQ/selenium/blob/2abd80f236d1a7459ef638e96af8c4efd86b4abd/rb/lib/selenium/webdriver/common/zipper.rb It does seem to have been used historically: #2933. Colleagues on the watir Slack channel who tried to use it with Selenium ended up with one of the ::Browser.new errors noted above.

I don't see any other approaches using Watir::Browser.new that allow reading in a .zip.b64 Profile. If it is possible, I'm hoping someone will know how to do it. If it's not possible, I'm hoping that it can be implemented. It is a useful feature if it can be made to work.

And while the following approach is not optimal because it does not use the .zip.b64 file for the Profile, I can get Firefox to address a copy of a good profile correctly if I edit the default (and only the default) profiles.ini file to point to a directory where I have copied the profile contents:

[Profile0]
Name=default-release
IsRelative=0
Path=/path/to/copy/of/profile/directory

But this still does not get me a Firefox running my profile in a docker-selenium container. The documentation for docker-selenium suggests that while using a custom standalone Profile is possible, there is a current related bug in geckodriver that may or may not have a workaround: https://github.com/SeleniumHQ/docker-selenium/wiki/How-to-use-a-Firefox-custom-profile Even if we could work around the geckodriver bug, the docker-selenium issue report above requires "firefoxOptions.addArguments("-profile", "/tmp/firefox_profiles/hcqx24zh.selenium");" and we've demonstrated that there is no way to point Watir::Browser.new to an actual directory and have Firefox use it.

The only straightforward answer I have found is to build a localized custom docker-selenium image with the OS-specific Firefox Profile changes in place, but this is both inconvenient and also abandons features that worked circa 2016. I'm tracking progress on that here: elgalu/docker-selenium#344

To summarize, here are the points where I found problems trying to make this work:

1. Geckodriver should document exactly what it will accept for a Profile, especially with changes since Firefox version 67.
2. The Ruby Selenium bindings should provide a mechanism to supply modern Profile arrangements to geckodriver.
3. Watir (which I use) should wrap the Selenium bindings properly with Watir::Browser.new
4. docker-selenium should implement a way to access a Firefox profile on the local filesystem for the Ruby bindings to use.
5. Related projects like Zalenium and the elgalu docker-selenium fork (which I use) should pick up the changes to docker-selenium.

I have not filed issues with other projects yet. I am hoping for constructive comments here before I do that.

[titusfortner]

Hey Chris, I've seen similar issues and I started digging into the updates from the switch to geckodriver with #7376

The last line was there because I'm not confident it is doing the right things:

I'm not sure the best way to go about testing this in an automated fashion. I think someone needs to sit down with extensions and preferences and certificates to make sure everything does what we expect for all the different methods outlined above, and I'm not currently volunteering for that. :)

[titusfortner]

Oh, and I added an issue for it at that time hoping someone would snap it up: #7404

[titusfortner]

According to the docs (https://firefox-source-docs.mozilla.org/testing/geckodriver/Profiles.html), we have two options.

First is passing in the directory location as an options args value:
I got this working after I found mozilla/geckodriver#1058

profile = "/Users/titusfortner/Library/Application Support/Firefox/Profiles/x6muv71k.default-1540548528938"
service = Selenium::WebDriver::Service.firefox(args: {marionette_port: 2828})
options = Selenium::WebDriver::Firefox::Options.new(args: ["-profile", profile])
Selenium::WebDriver.for(:firefox, service: service, options: options).tap(&:quit)

Don't zip anything, don't edit profiles.ini, just point to the directory

As for the other approach to zip the directory and send it in the capabilities as a profile... I also could not get it to work. Zipper is a private API class that is used in Profile#encoded method, which gets called by the Capabilities class when the provided Options instance has a profile value. This code correctly uses it:

profile = "/Users/titusfortner/Library/Application Support/Firefox/Profiles/x6muv71k.default-1540548528938"
ff_profile = Selenium::WebDriver::Firefox::Profile.new(profile)
options = Selenium::WebDriver::Firefox::Options.new(profile: ff_profile)
Selenium::WebDriver.for(:firefox, options: options).tap(&:quit)

but I'm not seeing the profile I expect to see when I do that code.

So, I'll file a bug with geckodriver for the second one, but since I have gotten the first one to work with Selenium, I'm closing this issue. If you need help translating the Selenium to Watir we can discuss it more on Slack, or raise an issue in Watir.

[chrismcmahon]

That is all fine. The value of a zipped/encoded profile is to have that file in source control instead of 50+ disparate files in the profile directory. Thank you very much for doing this work.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.