Selenuim 2 (WebDriver 2) and The Trouble of Automatic Firefox Updates

One day all my Selenium-based tests suddenly broke with error message: "org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host localhost on port 7055 after 45000 ms." That day my Firefox was automatically updated from 10 to 11. All tests were fine before the update. I guessed that it was the update that broke all the tests. The full error message is below (with a little modification removing security sensitive information):

[TestNG] Running:
  C:\Documents and Settings\ted-gao\Local Settings\Temp\testng-eclipse--811630090\testng-customsuite.xml

org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host localhost on port 7055 after 45000 ms. Firefox console output:
*** LOG addons.xpi: startup
*** LOG addons.xpi: Skipping unavailable install location app-system-local
*** LOG addons.xpi: Skipping unavailable install location app-system-share
*** LOG addons.xpi: Ignoring file entry whose name is not a valid add-on ID: C:\DOCUME~1\TED-GAO\LOCALS~1\Temp\anonymous4693713743778448909webdriver-profile\extensions\webdriver-staging
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi: No changes found

    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:95)
    at org.openqa.selenium.firefox.FirefoxDriver.startClient(FirefoxDriver.java:147)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:75)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:127)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:82)
   (a few lines removed here)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
    at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:543)
    at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:212)
    at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:140)
    at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:175)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:107)
    at org.testng.TestRunner.privateRun(TestRunner.java:753)
    at org.testng.TestRunner.run(TestRunner.java:613)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:337)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:330)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:292)
    at org.testng.SuiteRunner.run(SuiteRunner.java:241)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:64)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1170)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1094)
    at org.testng.TestNG.run(TestNG.java:1006)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:107)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:210)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:170)
FAILED CONFIGURATION: @BeforeClass setup
org.openqa.selenium.WebDriverException: Failed to connect to binary FirefoxBinary(D:\Program Files\Mozilla Firefox\firefox.exe) on port 7055; process output follows:
*** LOG addons.xpi: startup
*** LOG addons.xpi: Skipping unavailable install location app-system-local
*** LOG addons.xpi: Skipping unavailable install location app-system-share
*** LOG addons.xpi: Ignoring file entry whose name is not a valid add-on ID: C:\DOCUME~1\TED-GAO\LOCALS~1\Temp\anonymous4693713743778448909webdriver-profile\extensions\webdriver-staging
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi: No changes found

Build info: version: '2.14.0', revision: 'unknown', time: '2011-11-29 13:13:16'
System info: os.name: 'Windows XP', os.arch: 'x86', os.version: '5.1', java.version: '1.6.0_17'
Driver info: driver.version: FirefoxDriver
    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:107)
    at org.openqa.selenium.firefox.FirefoxDriver.startClient(FirefoxDriver.java:147)
    at org.openqa.selenium.remote.RemoteWebDriver.<init>(RemoteWebDriver.java:75)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:127)
    at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:82)
    (a few lines removed here)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
    at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:543)
    at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:212)
    at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:140)
    at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:175)
    at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:107)
    at org.testng.TestRunner.privateRun(TestRunner.java:753)
    at org.testng.TestRunner.run(TestRunner.java:613)
    at org.testng.SuiteRunner.runTest(SuiteRunner.java:337)
    at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:330)
    at org.testng.SuiteRunner.privateRun(SuiteRunner.java:292)
    at org.testng.SuiteRunner.run(SuiteRunner.java:241)
    at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:64)
    at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:87)
    at org.testng.TestNG.runSuitesSequentially(TestNG.java:1170)
    at org.testng.TestNG.runSuitesLocally(TestNG.java:1094)
    at org.testng.TestNG.run(TestNG.java:1006)
    at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:107)
    at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:210)
    at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:170)
Caused by: org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host localhost on port 7055 after 45000 ms. Firefox console output:
*** LOG addons.xpi: startup
*** LOG addons.xpi: Skipping unavailable install location app-system-local
*** LOG addons.xpi: Skipping unavailable install location app-system-share
*** LOG addons.xpi: Ignoring file entry whose name is not a valid add-on ID: C:\DOCUME~1\TED-GAO\LOCALS~1\Temp\anonymous4693713743778448909webdriver-profile\extensions\webdriver-staging
*** LOG addons.xpi: checkForChanges
*** LOG addons.xpi: No changes found

    at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:95)
    ... 33 more

SKIPPED CONFIGURATION: @AfterMethod teardown
SKIPPED: testHomePage
SKIPPED: testBenefitPage
SKIPPED: testProviderLocatorPageFromSideMenu
SKIPPED: testProviderSearchByZip
SKIPPED: testProviderLocatorPageFromTopMenu
SKIPPED: testWellness

===============================================
    Default test
    Tests run: 6, Failures: 0, Skips: 6
    Configuration Failures: 1, Skips: 1
===============================================


===============================================
Default suite
Total tests run: 6, Failures: 0, Skips: 6
Configuration Failures: 1, Skips: 1
===============================================

[TestNG] Time taken by org.testng.reporters.SuiteHTMLReporter@2079451: 141 ms
[TestNG] Time taken by org.testng.reporters.EmailableReporter@207a59c: 31 ms
[TestNG] Time taken by [TestListenerAdapter] Passed:0 Failed:0 Skipped:0]: 16 ms
[TestNG] Time taken by org.testng.reporters.JUnitReportReporter@207a66b: 15 ms
[TestNG] Time taken by org.testng.reporters.XMLReporter@20795a3: 32 ms

Even though the error message might seem suggesting something else. The really reason was that the version of WebDriver that I was using (2.11.0) was not compatible with Firefox 11 (at that point of time, there was not any version of WebDriver that supported Firefox 11).

It was too much trouble to uninstall Firefox 11 and to reinstall Firefox 10. My solution is to copy the base directory of a Firefox 10 installation (and all sub-directories, etc) on another computer to my computer, and use the following code fragment to point to the specific Firefox binary:

FirefoxBinary binary = new FirefoxBinary(new File(firefoxBinaryPath));
WebDriver webDriver = new FirefoxDriver(binary, profile);

Where firefoxBinaryPath is a String representing the path to the Firefox binary executable; profile is a FirefoxProfile object. (My another post is about choosing Firefox profile for WebDriver.)

It works. All my Selenium-based tests work again.

To disable automatic Firefox updates, follow this article.

Selenium 2 (WebDriver), Firefox Profiles, and Invalid Certificates

 (Last update on March 25, 2012)

When use Selenium 2 (WebDriver) and Firefox to automate web testing, one may need to select a Firefox profile for the test runs.  There are two ways to do it with the Java API for Selenium 2. One way is:

System.setProperty(“webdriver.firefox.profile”, profileName);
WebDriver webDriver = new FirefoxDriver();

Another way is:

ProfilesIni profilesIni = new ProfilesIni();
// Clone the named profile               
FirefoxProfile profile = profilesIni.getProfile(profileName);
WebDriver webDriver = new FirefoxDriver(profile);


Both of them do the same thing: starting Firefox with a clone of the specified profile. When the FirefoxDriver constructor is called without any argument, WebDriver will use the value of the webdriver.firefox.profile system property to identify the profile to clone. If a FirefoxProfile object is passed as an argument to the constructor, the webdriver.firefox.profile system property will simply be ignored. The second approach has its advantage as providing a reference to the FirefoxProfile object for further manipulation. 

In one case I need to manipulate the FirefoxProfile object to get my tests working. The site that I was testing used an invalid site certificate. I manually accessed the site with Firefox. When Firefox complained about the invalid site certificate, I told Firefox to accept it. After that, Firefox did not mention anything about the site certificate when I manually browsed the site. At that point, when I clicked the Site Identity Button of Firefox, I saw the dialog box like in Figure 1. (Identity of the site being tested is removed from the figures in this article)

Figure 1

However, when Firefox was started by WebDriver, it kept showing error message like “Secure Connection Failed” … so and so “uses an invalid security certificate” … “The certificate is only valid for “ … as in Figure 2.

Figure 2

The automated testing could not progress further. At that point, when I clicked the Site Identity Button, I saw the dialog box that reads “This website does not supply identify information. Your connection to this website is not encrypted.” as in Figure 3. There was no way at that point for me to tell Firefox to accept the certificate via Firefox's GUI.

Figure 3

Setting the AssumUntrustedCertificateIssuer of the FirefoxProfile object to false solved the problem, as illustrated in the code fragment below:

ProfilesIni profilesIni = new ProfilesIni();
// Clone the named profile               
FirefoxProfile profile = profilesIni.getProfile(profileName);

profile.setAssumeUntrustedCertificateIssuer(false);
WebDriver webDriver = new FirefoxDriver(profile);

In another case, the site that I was testing used a valid but self-signed certificate. As usual when I manually browse it with Firefox, Firefox warned me that "This Connection is Untrusted", like in Figure 4 below. I could click the "I Understand the Risks" link to pass it.

Figure 4

 However, when WebDriver started Firefox to browse the site, the same warning came out again. At this point when I click the Site Identity Button, I again saw the dialog box that reads “This website does not supply identify information. Your connection to this website is not encrypted.” as in Figure 3.

If one does not want the warning come out during test, one has to make  the AssumUntrustedCertificateIssuer of the FirefoxProfile object to be true. AssumUntrustedCertificateIssuer is true by default. One can either omit
  

profile.setAssumeUntrustedCertificateIssuer(false);

or change it to

profile.setAssumeUntrustedCertificateIssuer(true);

In short, if the certificate is valid but self-signed, we have to make AssumUntrustedCertificateIssuer true (which is the default); if the certificate is invalid, we have to make AssumUntrustedCertificateIssuer false.

By the way, to find which profile is really used when Firefox is running, do the following. From Firefox click Help -> Troubleshooting Information as shown in Figure 5 below.

Figure 5

On the new tab with the title "Troubleshooting Information", click the button "Open Containing Folder" as shown in Figure 5 below. It will open the base directory of the profile in use.

Figure 6