Tuesday, January 13, 2009

Selenium IDE include command for template

There is a useful extension for Selenium-Core: include, which can add the content of another test to the current test. Then it is possible to reuse test scripts for Selenese. See OpenQA wiki: http://wiki.openqa.org/display/SEL/include

However, it supported Selenium-Core only. Many Selenium-IDE users also want to use this feature. So I've ported the existing include extension to work in Selenium IDE. I added it to wiki: http://wiki.openqa.org/pages/viewpageattachments.action?pageId=283

  • target receives the page address (relative path from base url or use absolute path)
  • text receives vars names and their values for this test as a comma separated list of var_name=value


  • include step run in Selenium-IDE looks like:

    Please set user-extension.js path for SeleniumIDE->options->Selenium Core Extensions

    Saturday, January 3, 2009

    Remember Certificate Exception extension for Firefox3

    I submit a new add-on for Firefox3: Remember Certificate Exception.
    https://addons.mozilla.org/en-US/firefox/addon/10246



    RCE.xpi features:
    Auto complete Firefox3 SSL certificate exception override on exceptionDialog.
    Firefox3 introduced new 'Add Exception/BadCert Override' mandatory operations to bypass bad certificate exception.
    It is much safer but inconvenient to do mass bypassing while doing development with self-signed certificates.
    User wants a tool to auto complete add exception/badcert override operations.
    RCE extension is a such kind robot which can:
    - auto-click 'Or you can add an exception' link
    - auto-click 'Add Exception' button
    - auto-click 'Get Certificate' button
    - auto-click 'Confirm Security Exception' button
    - auto-reload ssl page
    The idea of RCE.xpi came from solving my Selenium web automation testing ssl page blocking issue.
    Please try badcert demo URLs inside http://kuix.de/mozilla/certwarndiscussion/proposal20061016/ with RCE extension.
    Please read firefox3 security topic: http://www.smop.co.uk/blog/index.php/2008/03/25/firefox-3-security-madness/
    WARNING: please disable RCE when you are going to browse untrusted websites to avoid phishing and mal-ware.

    Screenshot:

    Friday, January 2, 2009

    Selenium1.0beta for Firefox3 is ready

    We have fixed most of blocking bugs on Firefox3. It's time to upgrade our Selenium lab machines to Firefox3.0.5. Mozilla has announced Firefox2.0.0.20 would be the last version and stopped Firefox2 maintenance & supports. The Firefox3 market share is increasing to 17% at the end of 2008.


    The interesting new issue on Firefox3 is SSL cert exception page blocking Selenium automation. On Firefox2, we invoke RMD.xpi XPCOM in Selenium to intercept all bad cert exception for our self-signed certificates. However, it doesn't work on Firefox3 now that RMD.xpi's author said it only supported Firefox2. Firefox3 introduced a new SSL certificate exception rule. We must find a new workaround.

    Block step:


    Try below selenium ssl test case:
     
    <tr>
     <td>open</td>
     <td>http://kuix.de/mozilla/certwarndiscussion/proposal20061016/</td>
     <td></td>
    </tr>
    <tr>
     <td>open</td>
     <td>https://www.cacert.org</td>
     <td></td>
    </tr>
    <tr>
     <td>open</td>
     <td>https://www.kuix.de</td>
     <td></td>
    </tr>
    <tr>
     <td>open</td>
     <td>https://mur.at</td>
     <td></td>
    </tr>
    <tr>
     <td>open</td>
     <td>https://kuix.de:9443</td>
     <td></td>
    </tr>
    <tr>
     <td>open</td>
     <td>https://www.kuix.de:9443</td>
     <td></td>
    </tr>
    
    I plan to create a new project 'Remember Certificate Exception'(RCE) on AMO today. The RCE.xpi's purpose is auto-complete Firefox3 certificate exception override.

    Sunday, December 28, 2008

    Can Selenium detect if the page has JavaScript errors?

    We usually have common requirement for Selenium: Check JS error in step.

    Use case like:
    1. beginJsErrorChecker
    2. open | http://containJSError.jsp
    3. open | http://noissueJS.jsp
    4. endJsErrorChecker
    It can report url2 has jserror in Selenium:
    [error] JsErrorChecker:[JavaScript Error: "foo is not defined" {file: "http://containJSError.jsp" line: 9}]
    
    It's hard to provide a generic solution for all browsers but my team implements above scenario for Selenium IDE only by Firefox chrome Error Console XPCOM APIs.

    user-extensions.js like:
     
    if (browserVersion.isChrome) {
      var theConsoleListener = {
         stepIdx:0, //beginJsErrorChecker command index
         observe:function( aMessage ){//async!
            dump("Log : " + aMessage.message);
            //Error, Warning, Message too noise. We only report "[JavaScript Error:..."
            if(aMessage.message != null && aMessage.message.indexOf("[JavaScript Error:") == 0){ 
              LOG.error("JsErrorChecker:" + aMessage.message);
                      
              //throw new SeleniumError("Catch JS Error: " + aMessage.message);
              //can't throw exception here: pollution to target JSError
              //so we manually markFailed
              
              //IDE only
              var olddebugIndex = testCase.debugContext.debugIndex;
              testCase.debugContext.debugIndex = this.stepIdx;//set actual index
                      
              testCase.debugContext.failed = true;
              testCase.debugContext.currentCommand().result = 'failed';
              editor.view.rowUpdated(testCase.debugContext.debugIndex);
                      
              testCase.debugContext.debugIndex = olddebugIndex;//recover
            } 
       },
       QueryInterface: function (iid) {
            if (!iid.equals(Components.interfaces.nsIConsoleListener) &&
                  !iid.equals(Components.interfaces.nsISupports)) {
                throw Components.results.NS_ERROR_NO_INTERFACE;
                }
            return this;
       }
      }; 
    }
     
    Selenium.prototype.doBeginJsErrorChecker = function(){
      try {
        if (browserVersion.isChrome) {// firefox 
              theConsoleListener.stepIdx=testCase.debugContext.debugIndex;//set current step idx for async call
              var aConsoleService = Components.classes["@mozilla.org/consoleservice;1"]
                              .getService(Components.interfaces.nsIConsoleService);
              aConsoleService.registerListener(theConsoleListener);
              aConsoleService.reset();
        }else{
          throw new SeleniumError("TODO: Non-FF browser...");
        }
        } catch (e) {
        throw new SeleniumError("Threw an exception: " + e.message);
        }
    };
    
    Selenium.prototype.doEndJsErrorChecker = function(){
      try {
        if (browserVersion.isChrome) {// firefox  
              var aConsoleService = Components.classes["@mozilla.org/consoleservice;1"]
                              .getService(Components.interfaces.nsIConsoleService);
              aConsoleService.unregisterListener(theConsoleListener);
              aConsoleService.reset();
        }else{
          throw new SeleniumError("TODO: Non-FF browser...");
        }
        } catch (e) {
        throw new SeleniumError("Threw an exception: " + e.message);
        }
    };
    
    I just add a Selenim listener to Firefox Error Console to make jserror checker work for Selenium-IDE.
    Because it is async mode to check JSError, we treat beginJsErrorChecker failed if any step contains JSError in begin...end block.
    We also filter all Warning and Message in Error Console, treat Error only as failed here.

    I pasted solution on http://clearspace.openqa.org/message/52135
    http://jira.openqa.org/browse/SEL-613

    Saturday, December 27, 2008

    Selenium1.0b buglist for Firefox 3 Chrome TestRunner

    There are a lot of bugs in Selenium1.0b to run on Firefox 3 with chrome TestRunner.html.

    I list them here:
    1. CSS/Resource issue: http://jira.openqa.org/browse/SIDE-222
    2. Permission Denied issue: http://jira.openqa.org/browse/SEL-612
    3. waitForPopUp issue: http://jira.openqa.org/browse/SIDE-228
    4. click link issue: http://jira.openqa.org/browse/SEL-614
    5. UI-Element not work issue: http://sejq.blogspot.com/2008/12/fix-ui-element-cant-run-bug-in-chrome.html

    Sunday, December 21, 2008

    Two Javascript synchronous sleep functions which won't freeze browser

    Javascript didn't provide Thread.sleep(ms) method like Java which is useful for synchronization.

    Most of time we can solve Javascript synchronization by use setTimeout or setInteval. However, using setTimeout requires you to split your function up into discrete processes, which is hard to do for some Javascript logic. What we really need is a free-standing function that could be added inline to any block of code to share the processor on the fly.

    I encounter a scenario in Selenium today: I want Javascript to sleep to wait for an async call. It's much harder than I thought to implement it. Think about below code:
    function sleep(delay) {
       var startTime = new Date();
       var endTime = null;
       do {
           endTime = new Date();
       } while ((endTime - startTime);
    } 
    
    It looks fine but if you try it, you will find CPU is very busy executing the while loop and freeze browser. This is not what we wanted.

    I got 2 workaround methods. The keypoint is JS NOT freeze CPU/browser during sleep:
    1)javascript sleep 1: use XMLHttpRequest to do sync call for sleep.jsp
    function wbsleep(ms) { 
      var   startDate = new Date();   
      while((new Date()-startDate) < ms){   
          var xmlHttpReq; 
          try {
             // for IE/ActiveX
             if(window.ActiveXObject) {
                 try {
                     xmlHttpReq = new ActiveXObject("Msxml2.XMLHTTP");
                 }
                 catch(e) {
                     xmlHttpReq = new ActiveXObject("Microsoft.XMLHTTP");
                 }
             }
             // Native XMLHttp
             else if(window.XMLHttpRequest) {
                 xmlHttpReq = new XMLHttpRequest();
             } 
                    tempurl = "http://localhost/sleeputil.jsp?sleepms="+ms;
             xmlHttpReq.open("GET", tempurl, false); // synchron mode  
             xmlHttpReq.send(null);
          }
          catch(e) {  
          } 
      }
    };
    The sleeputil.jsp is very simple, get request parameter sleepms and then do a Java Thread.sleep(sleepms) in doGet. This method is suitable for all browsers but need you provide a server side sleeputil.jsp.

    2)javascript sleep 2: XPCOM on Firefox chrome only
    /**
     * Netscape compatible WaitForDelay function.
     * You can use it as an alternative to Thread.Sleep() in any major programming language
     * that support it while JavaScript it self doesn't have any built-in function to do such a thing.
     * parameters:
     *  (Number) delay in millisecond
    */
    function nsWaitForDelay(delay) {
      try{
        /**
        * Just uncomment this code if you're building an extention for Firefox.
        * Since FF3, we'll have to ask for user permission to execute XPCOM objects.
        */
        netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
    
        // Get the current thread.
        var thread = Components.classes["@mozilla.org/thread-manager;1"].getService(Components.interfaces.nsIThreadManager).currentThread;
    
        // Create an inner property to be used later as a notifier.
        this.delayed = true;
    
        /* Call JavaScript setTimeout function
         * to execute this.delayed = false
         * after it finish.
         */
        setTimeout("this.delayed = false;", delay);
    
        /**
         * Keep looping until this.delayed = false
        */
        while (this.delayed) {
            /**
             * This code will not freeze your browser as it's documented in here:
             * https://developer.mozilla.org/en/Code_snippets/Threads#Waiting_for_a_background_task_to_complete
            */
            thread.processNextEvent(true);
        }
      }catch(e) {  
      } 
    }
    This is especially suitable for Selenium CORE*ffchrome.

    Saturday, December 20, 2008

    Automate HttpFox(HttpWatch) test case for http traffic manipulation in Selenium

    Selenium can verify and store values by DOM on current page easily. However, it's hard for Selenium to do manipulation on http traffic like POST Data, http headers, Query String and Content sniffer.

    I know many QA did http traffic testing manually by 2 famous software:
    1) HTTPWatch: commercial. support both IE and Foxfox.
    2) HTTPFox: free, opensource. a Firefox extension.

    One day, a guy came to my cube and raised a question: Can I stop my tons of manual functional testing with HttpFox, but use Selenium to automate them? Is it possible that we create a bundle of Selenium APIs to manuplate Http Traffic Sniffer operations?

    Yes, we should add this important feature to Selenium. I finished a beta version to support HttpFox APIs in Selenium:
    1) merge httpfox.xpi into Selenium-IDE.xpi
    2) leverage XPCOM in httpfoxservice.js HTTPFox provided
    3) create Selenium APIs and print http traffic info to Selenium LOG:
    startHttpFox4Sel
    stopHttpFox4Sel
    clearHttpFox4Sel
    verifyTextInHttpFoxURLs(Not)Present | textpattern
    verifyTextInHttpFoxContent(Not)Present | urlpattern | textpattern
    storeContentInHttpFoxByURL | urlpattern | varname
    verifyParamInHttpFoxQueryString(Not)Present | urlpattern | param1=value1,param2=value2 list
    storeParamInHttpFoxQueryString | param@urlpattern | varname
    verifyParamInHttpFoxPostData(Not)Present | urlpattern | param1=value1,param2=value2 list
    storeParamInHttpFoxPostData | param@urlpattern | varname
    ...

    Source draft:
    Selenium.prototype.doStartHttpFox4Sel = function() {
        try {
         if(browserVersion.isChrome){
          HttpFox.cmd_hf_clear();
             HttpFox.cmd_hf_startWatching(); 
            }
        } catch (e) {
         throw new SeleniumError("Threw an exception: " + e.message);
        }
    }; 
    
    Selenium.prototype.doStopHttpFox4Sel = function() {
        try {
            if(browserVersion.isChrome){ 
             HttpFox.cmd_hf_stopWatching();
             HttpFox.cmd_hf_clear();
            }
        } catch (e) {
         throw new SeleniumError("Threw an exception: " + e.message);
        }
    }; 
    
    Selenium.prototype.doClearHttpFox4Sel = function() {
        try {
            if(browserVersion.isChrome){  
             HttpFox.cmd_hf_clear();
            }
        } catch (e) {
         throw new SeleniumError("Threw an exception: " + e.message);
        }
    }; 
           
    Selenium.prototype.isTextInHttpFoxURLsPresent = function(pattern) {
       try {
         if(browserVersion.isChrome){  
       for (var i = 0; i < HttpFox.HttpFoxService.Requests.length; i++){ 
            var request = HttpFox.HttpFoxService.Requests[i];
            //1.Time:
            var time = formatTimeDifference(request.StartTimestamp, request.EndTimestamp) ;
            //2.Sent:
            var sent = "";
            if (request.IsSending){
              sent = humanizeSize(request.getBytesSent(), 6) + "/" + humanizeSize(request.getBytesSentTotal(), 6);
            }
            else {
             sent = humanizeSize(request.getBytesSentTotal(), 6); 
            } 
            //3.Received:     
             var received = "";
             if (request.IsSending){
              received = "*";
           }
           if (!request.IsFinished){
              // show loading body progress
             received = humanizeSize(request.getBytesLoaded(), 6) + "/" + humanizeSize(request.getBytesLoadedTotal(), 6);
           }else{
             received = humanizeSize(request.getBytesLoaded(), 6); 
           }
           if (request.IsFromCache || request.ResponseStatus == 304){
             received = "(" + received + ")";
           } 
            //4.Method:
           var method = request.RequestMethod; 
            //5.Result: 
            var result = "";
           if (request.IsAborted){
             result =  "(Aborted)";
           }else if (request.isError()){
              result = "(Error)";
           }else if (request.IsFromCache && (request.ResponseStatus != 304)) {
              result =  "(Cache)";
           }else if (!request.HasReceivedResponseHeaders && !request.IsFinal){
              result = "*";
           }else{
               result =  request.ResponseStatus;
           } 
            //6.Type:
                  var type = "";
                  if (request.hasErrorCode()){
                   if (request.ContentType){
                   type =  request.ContentType + " (" + request.Status.toString(16) + ")";//we omit nsResultErrors translate
                  }else{
                   type =  request.Status.toString(16);
                  }
                 }else if (!request.HasReceivedResponseHeaders && !request.IsFromCache && !request.IsFinal){
                   type =  "*";
                 }else if (request.isRedirect()){
                  if (request.ResponseHeaders && request.ResponseHeaders["Location"]){
                   type =  "Redirect to: " + request.ResponseHeaders["Location"]; 
                  }else{
                   type =  "Redirect (cached)";
                  }
                 }else{
                  type = request.ContentType;     
                 }  
           //7.URL
           var urlstring = request.Url ;
           //TODO: LOG.info(http sniffer info);
           //TODO: verify as you like by PatternMatcher
       } //end for
        }//end chrome
        } catch (e) {
         throw new SeleniumError("Threw an exception: " + e.message);
        }
    }; 
    

    Now, QA can automate HttpFox steps in Selenium. They don't need to do http traffic testing manually all days.