Can Selenium detect if the page has JavaScript errors?

We usually have common requirement for Selenium: Check JS error in step.

Use case like:
1. beginJsErrorChecker
2. open | http://containJSError.jsp
3. open | http://noissueJS.jsp
4. endJsErrorChecker
It can report url2 has jserror in Selenium:

[error] JsErrorChecker:[JavaScript Error: "foo is not defined" {file: "http://containJSError.jsp" line: 9}]

It's hard to provide a generic solution for all browsers but my team implements above scenario for Selenium IDE only by Firefox chrome Error Console XPCOM APIs.

user-extensions.js like:

 
if (browserVersion.isChrome) {
  var theConsoleListener = {
     stepIdx:0, //beginJsErrorChecker command index
     observe:function( aMessage ){//async!
        dump("Log : " + aMessage.message);
        //Error, Warning, Message too noise. We only report "[JavaScript Error:..."
        if(aMessage.message != null && aMessage.message.indexOf("[JavaScript Error:") == 0){ 
          LOG.error("JsErrorChecker:" + aMessage.message);
                  
          //throw new SeleniumError("Catch JS Error: " + aMessage.message);
          //can't throw exception here: pollution to target JSError
          //so we manually markFailed
          
          //IDE only
          var olddebugIndex = testCase.debugContext.debugIndex;
          testCase.debugContext.debugIndex = this.stepIdx;//set actual index
                  
          testCase.debugContext.failed = true;
          testCase.debugContext.currentCommand().result = 'failed';
          editor.view.rowUpdated(testCase.debugContext.debugIndex);
                  
          testCase.debugContext.debugIndex = olddebugIndex;//recover
        } 
   },
   QueryInterface: function (iid) {
        if (!iid.equals(Components.interfaces.nsIConsoleListener) &&
              !iid.equals(Components.interfaces.nsISupports)) {
            throw Components.results.NS_ERROR_NO_INTERFACE;
            }
        return this;
   }
  }; 
}
 
Selenium.prototype.doBeginJsErrorChecker = function(){
  try {
    if (browserVersion.isChrome) {// firefox 
          theConsoleListener.stepIdx=testCase.debugContext.debugIndex;//set current step idx for async call
          var aConsoleService = Components.classes["@mozilla.org/consoleservice;1"]
                          .getService(Components.interfaces.nsIConsoleService);
          aConsoleService.registerListener(theConsoleListener);
          aConsoleService.reset();
    }else{
      throw new SeleniumError("TODO: Non-FF browser...");
    }
    } catch (e) {
    throw new SeleniumError("Threw an exception: " + e.message);
    }
};

Selenium.prototype.doEndJsErrorChecker = function(){
  try {
    if (browserVersion.isChrome) {// firefox  
          var aConsoleService = Components.classes["@mozilla.org/consoleservice;1"]
                          .getService(Components.interfaces.nsIConsoleService);
          aConsoleService.unregisterListener(theConsoleListener);
          aConsoleService.reset();
    }else{
      throw new SeleniumError("TODO: Non-FF browser...");
    }
    } catch (e) {
    throw new SeleniumError("Threw an exception: " + e.message);
    }
};

I just add a Selenim listener to Firefox Error Console to make jserror checker work for Selenium-IDE.
Because it is async mode to check JSError, we treat beginJsErrorChecker failed if any step contains JSError in begin...end block.
We also filter all Warning and Message in Error Console, treat Error only as failed here.

I pasted solution on http://clearspace.openqa.org/message/52135
http://jira.openqa.org/browse/SEL-613

Selenium1.0b buglist for Firefox 3 Chrome TestRunner

There are a lot of bugs in Selenium1.0b to run on Firefox 3 with chrome TestRunner.html.

I list them here:
1. CSS/Resource issue: http://jira.openqa.org/browse/SIDE-222
2. Permission Denied issue: http://jira.openqa.org/browse/SEL-612
3. waitForPopUp issue: http://jira.openqa.org/browse/SIDE-228
4. click link issue: http://jira.openqa.org/browse/SEL-614
5. UI-Element not work issue: http://sejq.blogspot.com/2008/12/fix-ui-element-cant-run-bug-in-chrome.html

Two Javascript synchronous sleep functions which won't freeze browser

Javascript didn't provide Thread.sleep(ms) method like Java which is useful for synchronization.

Most of time we can solve Javascript synchronization by use setTimeout or setInteval. However, using setTimeout requires you to split your function up into discrete processes, which is hard to do for some Javascript logic. What we really need is a free-standing function that could be added inline to any block of code to share the processor on the fly.

I encounter a scenario in Selenium today: I want Javascript to sleep to wait for an async call. It's much harder than I thought to implement it. Think about below code:

function sleep(delay) {
   var startTime = new Date();
   var endTime = null;
   do {
       endTime = new Date();
   } while ((endTime - startTime);
} 

It looks fine but if you try it, you will find CPU is very busy executing the while loop and freeze browser. This is not what we wanted.

I got 2 workaround methods. The keypoint is JS NOT freeze CPU/browser during sleep:
1)javascript sleep 1: use XMLHttpRequest to do sync call for sleep.jsp

function wbsleep(ms) { 
  var   startDate = new Date();   
  while((new Date()-startDate) < ms){   
      var xmlHttpReq; 
      try {
         // for IE/ActiveX
         if(window.ActiveXObject) {
             try {
                 xmlHttpReq = new ActiveXObject("Msxml2.XMLHTTP");
             }
             catch(e) {
                 xmlHttpReq = new ActiveXObject("Microsoft.XMLHTTP");
             }
         }
         // Native XMLHttp
         else if(window.XMLHttpRequest) {
             xmlHttpReq = new XMLHttpRequest();
         } 
                tempurl = "http://localhost/sleeputil.jsp?sleepms="+ms;
         xmlHttpReq.open("GET", tempurl, false); // synchron mode  
         xmlHttpReq.send(null);
      }
      catch(e) {  
      } 
  }
};

The sleeputil.jsp is very simple, get request parameter sleepms and then do a Java Thread.sleep(sleepms) in doGet. This method is suitable for all browsers but need you provide a server side sleeputil.jsp.

2)javascript sleep 2: XPCOM on Firefox chrome only

/**
 * Netscape compatible WaitForDelay function.
 * You can use it as an alternative to Thread.Sleep() in any major programming language
 * that support it while JavaScript it self doesn't have any built-in function to do such a thing.
 * parameters:
 *  (Number) delay in millisecond
*/
function nsWaitForDelay(delay) {
  try{
    /**
    * Just uncomment this code if you're building an extention for Firefox.
    * Since FF3, we'll have to ask for user permission to execute XPCOM objects.
    */
    netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");

    // Get the current thread.
    var thread = Components.classes["@mozilla.org/thread-manager;1"].getService(Components.interfaces.nsIThreadManager).currentThread;

    // Create an inner property to be used later as a notifier.
    this.delayed = true;

    /* Call JavaScript setTimeout function
     * to execute this.delayed = false
     * after it finish.
     */
    setTimeout("this.delayed = false;", delay);

    /**
     * Keep looping until this.delayed = false
    */
    while (this.delayed) {
        /**
         * This code will not freeze your browser as it's documented in here:
         * https://developer.mozilla.org/en/Code_snippets/Threads#Waiting_for_a_background_task_to_complete
        */
        thread.processNextEvent(true);
    }
  }catch(e) {  
  } 
}

This is especially suitable for Selenium CORE*ffchrome.

Automate HttpFox(HttpWatch) test case for http traffic manipulation in Selenium

Selenium can verify and store values by DOM on current page easily. However, it's hard for Selenium to do manipulation on http traffic like POST Data, http headers, Query String and Content sniffer.

I know many QA did http traffic testing manually by 2 famous software:
1) HTTPWatch: commercial. support both IE and Foxfox.
2) HTTPFox: free, opensource. a Firefox extension.

One day, a guy came to my cube and raised a question: Can I stop my tons of manual functional testing with HttpFox, but use Selenium to automate them? Is it possible that we create a bundle of Selenium APIs to manuplate Http Traffic Sniffer operations?

Yes, we should add this important feature to Selenium. I finished a beta version to support HttpFox APIs in Selenium:
1) merge httpfox.xpi into Selenium-IDE.xpi
2) leverage XPCOM in httpfoxservice.js HTTPFox provided
3) create Selenium APIs and print http traffic info to Selenium LOG:
startHttpFox4Sel
stopHttpFox4Sel
clearHttpFox4Sel
verifyTextInHttpFoxURLs(Not)Present | textpattern
verifyTextInHttpFoxContent(Not)Present | urlpattern | textpattern
storeContentInHttpFoxByURL | urlpattern | varname
verifyParamInHttpFoxQueryString(Not)Present | urlpattern | param1=value1,param2=value2 list
storeParamInHttpFoxQueryString | param@urlpattern | varname
verifyParamInHttpFoxPostData(Not)Present | urlpattern | param1=value1,param2=value2 list
storeParamInHttpFoxPostData | param@urlpattern | varname
...

Source draft:

Selenium.prototype.doStartHttpFox4Sel = function() {
    try {
     if(browserVersion.isChrome){
      HttpFox.cmd_hf_clear();
         HttpFox.cmd_hf_startWatching(); 
        }
    } catch (e) {
     throw new SeleniumError("Threw an exception: " + e.message);
    }
}; 

Selenium.prototype.doStopHttpFox4Sel = function() {
    try {
        if(browserVersion.isChrome){ 
         HttpFox.cmd_hf_stopWatching();
         HttpFox.cmd_hf_clear();
        }
    } catch (e) {
     throw new SeleniumError("Threw an exception: " + e.message);
    }
}; 

Selenium.prototype.doClearHttpFox4Sel = function() {
    try {
        if(browserVersion.isChrome){  
         HttpFox.cmd_hf_clear();
        }
    } catch (e) {
     throw new SeleniumError("Threw an exception: " + e.message);
    }
}; 
       
Selenium.prototype.isTextInHttpFoxURLsPresent = function(pattern) {
   try {
     if(browserVersion.isChrome){  
   for (var i = 0; i < HttpFox.HttpFoxService.Requests.length; i++){ 
        var request = HttpFox.HttpFoxService.Requests[i];
        //1.Time:
        var time = formatTimeDifference(request.StartTimestamp, request.EndTimestamp) ;
        //2.Sent:
        var sent = "";
        if (request.IsSending){
          sent = humanizeSize(request.getBytesSent(), 6) + "/" + humanizeSize(request.getBytesSentTotal(), 6);
        }
        else {
         sent = humanizeSize(request.getBytesSentTotal(), 6); 
        } 
        //3.Received:     
         var received = "";
         if (request.IsSending){
          received = "*";
       }
       if (!request.IsFinished){
          // show loading body progress
         received = humanizeSize(request.getBytesLoaded(), 6) + "/" + humanizeSize(request.getBytesLoadedTotal(), 6);
       }else{
         received = humanizeSize(request.getBytesLoaded(), 6); 
       }
       if (request.IsFromCache || request.ResponseStatus == 304){
         received = "(" + received + ")";
       } 
        //4.Method:
       var method = request.RequestMethod; 
        //5.Result: 
        var result = "";
       if (request.IsAborted){
         result =  "(Aborted)";
       }else if (request.isError()){
          result = "(Error)";
       }else if (request.IsFromCache && (request.ResponseStatus != 304)) {
          result =  "(Cache)";
       }else if (!request.HasReceivedResponseHeaders && !request.IsFinal){
          result = "*";
       }else{
           result =  request.ResponseStatus;
       } 
        //6.Type:
              var type = "";
              if (request.hasErrorCode()){
               if (request.ContentType){
               type =  request.ContentType + " (" + request.Status.toString(16) + ")";//we omit nsResultErrors translate
              }else{
               type =  request.Status.toString(16);
              }
             }else if (!request.HasReceivedResponseHeaders && !request.IsFromCache && !request.IsFinal){
               type =  "*";
             }else if (request.isRedirect()){
              if (request.ResponseHeaders && request.ResponseHeaders["Location"]){
               type =  "Redirect to: " + request.ResponseHeaders["Location"]; 
              }else{
               type =  "Redirect (cached)";
              }
             }else{
              type = request.ContentType;     
             }  
       //7.URL
       var urlstring = request.Url ;
       //TODO: LOG.info(http sniffer info);
       //TODO: verify as you like by PatternMatcher
   } //end for
    }//end chrome
    } catch (e) {
     throw new SeleniumError("Threw an exception: " + e.message);
    }
}; 

Now, QA can automate HttpFox steps in Selenium. They don't need to do http traffic testing manually all days.

Dedicated to developing Selenium Core Firefox chrome branch

Since my team has chosen Selenium Core-Firefox chrome with Selenese table test case as primary Selenium testing method, my task will focus on Selenium Core Firefox chrome branch development(A brand new Selenium-IDE). I will also contribute a bundle of Java APIs to play Selenium Core*ffchrome. Moreover, I should provide a GRID framework for huge test case number run.

"Simply the best". No proxy server, no Remote Control, no IE/HTA, no Java Test Case, no Same Origin Policy, I really found this mode is efficient but powerful as long as we provide an enterprise framework.

For QA, they only need learn Selenese table knowledge without Java programming skills. They don't need to install any complex framework but only setup our Selenium-IDE version on their desktop.

For framework developer, then needn't build a heavyweight infrastructure like proxy server, client driver. What they need do are only:
1) Let Selenium-Core*ffchrome save detailed enough result capture/snapshot with well formed structure to local file system.
2) Extend chrome URL parameters to meet all configuration requirements. We can benifit a lot from Mozilla APIs and do some fantasitic jobs for Selenium core(That why we choose Firefox as Selenium browser).
3) A bundle of Java APIs to play Selenium-Core*ffchrome under isolated profile and parse run results to Java Objects. We can define an atomic unit(1-3) as one Selenium Agent.
4) A GRID Java framework to distribute huge test case number run on Agents with scalability. DB support is necessary to save both test case info and run result info.
5) Then a GUI web based test case maintenance and execution portal is possible with above back end framework supporting.

Comparing to official RC+GRID solution, our solution is more suitable for acceptance test, integrating test & regression test(black box).

Selenese flowControl extensions - advanced support for include template

As my previous post, Selenese flowControl extensions work well on IDE and Core now. However, when guys used another powerful extension 'include', they found flowControl not work in include template steps.

I provide a quick workaround here. Please revise include extension javascript as below:

Selenium.prototype.doInclude = function(locator, paramString) {
    LOG.debug(IncludeCommand.LOG_PREFIX + " Version " + IncludeCommand.VERSION);
    var includeCommand = new IncludeCommand();
    includeCommand.doInclude(locator, paramString);
  
     //Jerry Qian: add flowControl support for include template steps
     try{
       htmlTestRunner.currentTest.gotoLabels  = {};
       htmlTestRunner.currentTest.whileLabels = { ends: {}, whiles: {} };
       htmlTestRunner.currentTest.initialiseLabels();
     }catch(e){}
     //End flowControl
};

Include template steps can use flowControl commands now.

Selenese flowControl extensions

Usually, we don't need to do conditional logic in Selenese table. Selenium RC programming language is more suitable for complex logic. But sometimes we still want to add some conditional steps in Selenese, like if...else, while.

Fortunately, there are already flow control extensions on openqa's wiki.
http://wiki.openqa.org/display/SEL/flowControl

It can meet most of flow control requirements for Selenese. We still recommend use RC to implement more complicated logic.

Also some people work on both Core and IDE. They found this extensions not work on IDE. It's a CORE version.

One guy contributed an IDE version: http://51elliot.blogspot.com/2008/02/selenium-ide-goto.html. IDE user can also use it.

So I combined them to a unified flowControl extensions: support both IDE and CORE.

The simplest code:

var sel_locstr = window.location.href;  
if(sel_locstr.indexOf("selenium-ide.xul") != -1){ //IDE mode      
   // paste IDE extension here
}else{//Core mode
   // paste Core extension here
} 

Enjoy!

Firefox3 new security rule: Selenium no CSS on Chrome TestRunner

Firefox 3 introduced more strict security rules than older version. Many Selenium weird issues on FF3 are related to it, while most of these issues didn't happen on FF2.

Today I encountered one SeleniumIDE1.0b2 bug: [SIDE-222]Test execution is not possible in chrome mode in Firefox 3.

FF3 Error Console reported:

Security Error: Content at (local test suite html address like file:///c:/seltest/testsuite1.html) may not load or link to "chrome://selenium-ide-testrunner/content/selenium/selenium-test.css" 

It seems FF3's default security settings not allow external file access chrome:// resource.
I read Mozilla docs: https://developer.mozilla.org/en/Chrome_Registration#contentaccessible

It said: (New in FF3) Chrome resources can no longer be referenced from within <img>, <script>, or other elements contained in, or added to, content that was loaded from an untrusted source. This restriction applies to both elements defined by the untrusted source and to elements added by trusted extensions. If such references need to be explicitly allowed, set the contentaccessible flag to yes to obtain the behavior found in older versions of Firefox.

So, I try to modify Selenium-IDE1.0b2's chrome.manifest file.

content selenium-ide-testrunner jar:chrome/selenium-ide.jar!/content/ xpcnativewrappers=no 
content selenium-ide-testrunner jar:chrome/selenium-ide.jar!/content/ xpcnativewrappers=no contentaccessible=yes

SeleniumIDE1.0b2 displayed CSS correctly this time on FF3 Chrome TestRunner.

Test Flex/Flash in Selenium

There are many automation tools for testing Flex & Flash RIA web page, like TestComplete, QTP. However, they are all not free. It's good news if Selenium supports such kind of RIA testing.

We found there are some interesting projects on Selenium for Flex/Flash testing.

Here is a topic: http://www.agimatec.de/blog/2008/11/selenium-flex-tests-with-maven/

Introduction:
SeleniumFlexApi is in one part a Flex library which needs to be included in your app and on the other part an extension to the Selenium IDE. I like the Flex part, because it makes it easy to dive over Javascript into the details of your Flex app. What I don’t really like is the Javascript extension. Yes, it is not only a Selenium IDE part and you can include it into the start of the Selenium Server, but it is far away from Java and Maven.

FlashSelenium is a piece of code which let’s you talk from Java over Selenium with your Flex app. The Java part is very handy, but you need to code into Flex, which methods you could call over your bridge. So I need to write special code, which opens the bridge. The SeleniumFlexAPI is much better in this part.

Fix ui-element can't run bug in Chrome TestRunner mode

Selenium-IDE1.0b2 introduced a new feature: UI-Element. It is very useful feature for annoyed XPATH EL.

We try demo in Selenium IDE panel successfully. However, when we try UI-Element test case in Chrome TestRunner, even if we set &userExtensionsURL=chrome://selenium-ide/content/ui-element.js,file:///c:/ui/my-ui-map.js correctly in chrome URL, it failed to load any UI-Element feature.

Weird, we have discussed it on openqa forum. Finally, I fix this bug in chrome://selenium-ide/content/ui-element.js

function is_IDE()
{      
 //[Begin Fix]
 var locstr = window.location.href;  
 if(locstr.indexOf("selenium-ide.xul") != -1){
     //IDE mode   
     return true;
 }else{
     return false; 
 }
 //[End]
 return (typeof(SeleniumIDE) != 'undefined');
}

It seems

typeof(SeleniumIDE) != 'undefined'

not work in Chrome TestRunner mode.

Finally, with above fix, UI-Element works in both IDE and Chrome TestRunner.

More discussion: http://clearspace.openqa.org/message/53306