Showing posts with label Selenium-IDE. Show all posts
Showing posts with label Selenium-IDE. Show all posts

Tuesday, January 13, 2009

Selenium IDE include command for template

There is a useful extension for Selenium-Core: include, which can add the content of another test to the current test. Then it is possible to reuse test scripts for Selenese. See OpenQA wiki: http://wiki.openqa.org/display/SEL/include

However, it supported Selenium-Core only. Many Selenium-IDE users also want to use this feature. So I've ported the existing include extension to work in Selenium IDE. I added it to wiki: http://wiki.openqa.org/pages/viewpageattachments.action?pageId=283

  • target receives the page address (relative path from base url or use absolute path)
  • text receives vars names and their values for this test as a comma separated list of var_name=value


  • include step run in Selenium-IDE looks like:

    Please set user-extension.js path for SeleniumIDE->options->Selenium Core Extensions

    Saturday, December 20, 2008

    Automate HttpFox(HttpWatch) test case for http traffic manipulation in Selenium

    Selenium can verify and store values by DOM on current page easily. However, it's hard for Selenium to do manipulation on http traffic like POST Data, http headers, Query String and Content sniffer.

    I know many QA did http traffic testing manually by 2 famous software:
    1) HTTPWatch: commercial. support both IE and Foxfox.
    2) HTTPFox: free, opensource. a Firefox extension.

    One day, a guy came to my cube and raised a question: Can I stop my tons of manual functional testing with HttpFox, but use Selenium to automate them? Is it possible that we create a bundle of Selenium APIs to manuplate Http Traffic Sniffer operations?

    Yes, we should add this important feature to Selenium. I finished a beta version to support HttpFox APIs in Selenium:
    1) merge httpfox.xpi into Selenium-IDE.xpi
    2) leverage XPCOM in httpfoxservice.js HTTPFox provided
    3) create Selenium APIs and print http traffic info to Selenium LOG:
    startHttpFox4Sel
    stopHttpFox4Sel
    clearHttpFox4Sel
    verifyTextInHttpFoxURLs(Not)Present | textpattern
    verifyTextInHttpFoxContent(Not)Present | urlpattern | textpattern
    storeContentInHttpFoxByURL | urlpattern | varname
    verifyParamInHttpFoxQueryString(Not)Present | urlpattern | param1=value1,param2=value2 list
    storeParamInHttpFoxQueryString | param@urlpattern | varname
    verifyParamInHttpFoxPostData(Not)Present | urlpattern | param1=value1,param2=value2 list
    storeParamInHttpFoxPostData | param@urlpattern | varname
    ...

    Source draft:
    Selenium.prototype.doStartHttpFox4Sel = function() {
        try {
         if(browserVersion.isChrome){
          HttpFox.cmd_hf_clear();
             HttpFox.cmd_hf_startWatching(); 
            }
        } catch (e) {
         throw new SeleniumError("Threw an exception: " + e.message);
        }
    }; 
    
    Selenium.prototype.doStopHttpFox4Sel = function() {
        try {
            if(browserVersion.isChrome){ 
             HttpFox.cmd_hf_stopWatching();
             HttpFox.cmd_hf_clear();
            }
        } catch (e) {
         throw new SeleniumError("Threw an exception: " + e.message);
        }
    }; 
    
    Selenium.prototype.doClearHttpFox4Sel = function() {
        try {
            if(browserVersion.isChrome){  
             HttpFox.cmd_hf_clear();
            }
        } catch (e) {
         throw new SeleniumError("Threw an exception: " + e.message);
        }
    }; 
           
    Selenium.prototype.isTextInHttpFoxURLsPresent = function(pattern) {
       try {
         if(browserVersion.isChrome){  
       for (var i = 0; i < HttpFox.HttpFoxService.Requests.length; i++){ 
            var request = HttpFox.HttpFoxService.Requests[i];
            //1.Time:
            var time = formatTimeDifference(request.StartTimestamp, request.EndTimestamp) ;
            //2.Sent:
            var sent = "";
            if (request.IsSending){
              sent = humanizeSize(request.getBytesSent(), 6) + "/" + humanizeSize(request.getBytesSentTotal(), 6);
            }
            else {
             sent = humanizeSize(request.getBytesSentTotal(), 6); 
            } 
            //3.Received:     
             var received = "";
             if (request.IsSending){
              received = "*";
           }
           if (!request.IsFinished){
              // show loading body progress
             received = humanizeSize(request.getBytesLoaded(), 6) + "/" + humanizeSize(request.getBytesLoadedTotal(), 6);
           }else{
             received = humanizeSize(request.getBytesLoaded(), 6); 
           }
           if (request.IsFromCache || request.ResponseStatus == 304){
             received = "(" + received + ")";
           } 
            //4.Method:
           var method = request.RequestMethod; 
            //5.Result: 
            var result = "";
           if (request.IsAborted){
             result =  "(Aborted)";
           }else if (request.isError()){
              result = "(Error)";
           }else if (request.IsFromCache && (request.ResponseStatus != 304)) {
              result =  "(Cache)";
           }else if (!request.HasReceivedResponseHeaders && !request.IsFinal){
              result = "*";
           }else{
               result =  request.ResponseStatus;
           } 
            //6.Type:
                  var type = "";
                  if (request.hasErrorCode()){
                   if (request.ContentType){
                   type =  request.ContentType + " (" + request.Status.toString(16) + ")";//we omit nsResultErrors translate
                  }else{
                   type =  request.Status.toString(16);
                  }
                 }else if (!request.HasReceivedResponseHeaders && !request.IsFromCache && !request.IsFinal){
                   type =  "*";
                 }else if (request.isRedirect()){
                  if (request.ResponseHeaders && request.ResponseHeaders["Location"]){
                   type =  "Redirect to: " + request.ResponseHeaders["Location"]; 
                  }else{
                   type =  "Redirect (cached)";
                  }
                 }else{
                  type = request.ContentType;     
                 }  
           //7.URL
           var urlstring = request.Url ;
           //TODO: LOG.info(http sniffer info);
           //TODO: verify as you like by PatternMatcher
       } //end for
        }//end chrome
        } catch (e) {
         throw new SeleniumError("Threw an exception: " + e.message);
        }
    }; 
    

    Now, QA can automate HttpFox steps in Selenium. They don't need to do http traffic testing manually all days.

    Tuesday, November 18, 2008

    Write user-extensions.js working for both Selenium-IDE and Selenium-Core

    Sometimes we found our user-extensions.js(like include cmd)work on Selenium-Core but error on Selenium-IDE, or vise versa.

    The reason is IDE and Core used 30% different js objects.

    We can write below style user-extension to support both IDE and Core.
    Sample:
    if(IDE_MODE){
       //IDE manner
    }else{//CORE_MODE
       //Core manner
    }
    The key function here is how to determine current mode type by javascript?
    Simple workaround code:
    function isIDE(){
      var locstr = window.location.href;  
      if(locstr.indexOf("selenium-ide.xul") != -1){
          //IDE mode   
          return true;
      }else{
          return false; //Core mode
      }
    }

    Friday, November 14, 2008

    Include JS src file in Firefox Chrome Javascript file

    To include a Javascript src file in HTML, it's very simple:
    <script src="scripts/b.js" type="text/javascript"></script>
    However, how to implement above logic in a Chrome js file(XUL missed including b.js).
    There is a XPCOM API @mozilla.org/moz/jssubscript-loader;1
    Sample:
    const subScriptLoader = Components.classes["@mozilla.org/moz/jssubscript-loader;1"].getService(Components.interfaces.mozIJSSubScriptLoader);
     //file:/// path for XPCOM. http:// not support
    subScriptLoader.loadSubScript('file:///c:/scripts/b.js', this);
    In normal js file(non-chrome), it's also simple.
    Sample:
    document.write('<script src="scripts/b.js" type="text/javascript"></script>');
    This tip is useful for Selenium-IDE enhancement.

    Thursday, November 13, 2008

    Security Alerts Interceptor for Selenium-IDE

    There are 2 kinds of HTTPs Certification Security Alerts in Firefox.
    1) Domain name mismatch
    2) Certificate expired
    Read http://kb.mozillazine.org/Security_Error:_Domain_Name_Mismatch_or_Server_Certificate_Expired
    Unfortunately, Firefox 2 didn't provide a preferences entry to disable or hidden alerts.
    These annoyed alerts slow down selenium speed and block selenium automation if we don't click out alerts.

    There is a Firefox extension Remember Mismatch Domains(RMD.xpi) to add a "Don't warn me again about this certificate for this domain" checkbox to the Domain Mismatch and Expired Certificate warning windows. When selected the domain name and security certificate domain pair (or certificate and expiration date pair) is stored in a Firefox preference and the security error dialogue will be bypassed on subsequent visits.

    The author provided a XPCOM to destroy Security Alerts Window before it displays in Firefox. We can integrate this XPCOM with Selenium-IDE and provide Security Alerts Interceptor feature.

    Here are steps and code:
    1) copy /platform folder from RMD.xpi to selenium-ide.xpi
    2) copy rmdBadCertHandler.js and rmdIBadCertHandler.xpt to /components folder
    3) edit rmdBadCertHandler.js
    _isRemembered: function (handler_type, target_url, cert) {
       return true;
       //intercept and kill all security alerts
       //or add a preference entry here and return true or false
       ...
    }
    NOTE: it is used for Firefox 2 only. Firefox 3 has a new security exception handler.

    Sunday, November 9, 2008

    PseudoTestRunner.html to run Selenium Chrome by cmdline

    Launch Selenium Chrome TestRunner by cmdline: firefox.exe -chrome "chrome://selenium-ide-testrunner/content/selenium/TestRunner.html..." will cause many weird issues on Firefox.
    I'm not sure the root cause which maybe related to Firefox Chrome Security Strategy for -chrome parameter.

    Here is a workaround:
    Hijack selenium-ide.xpi. Under selenium-ide\content\selenium\ add a file PseudoTestRunner.html
    Source:
    <html>
    <head><title>Pseudo Test Runner - Used for launch Chrome by Cmdline</title></head>
    <body>
      <script language="javascript"> 
      var strHref;
      var intParamsStartIndex;
      var strParams;
      strHref = window.location.href;
      intParamsStartIndex = strHref.indexOf("?");
      strParams = strHref.substring(intParamsStartIndex);
      window.open("chrome://selenium-ide-testrunner/content/selenium/TestRunner.html" + strParams);
      window.close();
      </script>
    </body>
    </html>
    
    Then please always use PseudoTestRunner.html instead of TestRunner.html in cmdline, keep the same parameter list in URL.

    Refer to topic: http://clearspace.openqa.org/message/51801#51801

    Saturday, November 8, 2008

    How to let JAVA play with Selenium Firefox Chrome TestRunner

    Selenium Firefox Chrome TestRunner is written by JS/DHTML. When we want to embed Firefox Chrome TestRunner engine into our testing framework. We are asking for a way to let JAVA play with Firefox Chrome TestRunner. You can also refer to Selenium-RC source code to launch a browser. Here is a workaround solution with pseudo code.

    1. In & Output of 'Selenium Firefox Chrome TestRunner' component for JAVA IN: it is usually URL trigger for running. Chrome URL with parameters looks like:
    chrome://selenium-ide-testrunner/content/selenium/TestRunner.html?test=file:///c:/testing/MyTestSuite.html&auto=true&multiWindow=true&userExtensionsURL=http://localhost/user-extensions.js,http://localhost/ui/my-ui-map.js&close=true&[other self defined parms]
    We can see Selenium passed parameters by chrome URL for input. We can add more self defined params by modify Selenium-IDE code.
    If we can convert all parameters in JAVA to chrome URL parameters, it's possible to let JAVA invoke Selenium-IDE.

    OUTPUT: one output choice provided by S-CORE is using POST data when Selenium closed. It required you create a server side receiver.
    One better solution is S-IDE save run snapshot/result/log runtimely to file system directly. We treat these important data as output.

    For offical docs about IN&OUTPUT, refer to http://seleniumhq.org/projects/core/usage.html

    2. 'IN' Wrapper in JAVA
    We create a Java Object as a chrome URL paramters wrapper, simple like:
    public class SeleniumRequest implements Serializable {
     private String test; 
     private String auto;
            private String multiWindow;
    ...
            public String getAuto() {
      return auto;
     } 
     public void setAuto(String auto) {
      this.auto = auto;
     }
    ...
    }
    3. 'OUTPUT' Wrapper in JAVA
    We also create a Java Object as atomic run result wrapper, simple like:
    public class SeleniumResult implements Serializable {
     private int testRuns; 
     private int testFailures; 
            private String resultSummary;
     private String logFilePath;  
    ...
            public String getResultSummary() {
      return resultSummary;
     } 
     public void setResultSummary(String resultSummary) {
      this.resultSummary= resultSummary;
     }
    ...
    }
    Detailed run results are saved on file system by S-IDE. logFilePath is the path for their index file.

    4. SeleniumJRunner in Java
    Now, we can define a very simple & generic interface for run S-IDE in Java.
    public interface SeleniumJRunner {
          public SeleniumResult run(SeleniumRequest request); 
    }
    We assume this run API is sync mode, which means run() will be blocked in Java thread until S-IDE finished run and generate OUTPUT.

    5. SeleniumResultParser.java
    S-IDE saved all results in file system, we need a Java class to parse file content and set to SeleniumResult object. This is part of SeleniumJRunner implementation logic. To do html file parser in Java, there are many opensource lib. You can use Nekohtml.

    6. SyncExecutor.java
    Since S-IDE is an external process for Java. We need launch it with parameters from Java in sync mode. Usually, JAVA API Runtime.getRuntime().exec() meets basic requirement. However, we need a timedout mechanism and safely execution which Runtime.getRuntime() doesn't provide. We create a powerful but simple SyncExecutor util. For related topic, refer to how to safely execute process from java

    7. Launch Selenium with profile isolation on the same machine
    It is super Firefox can run multiple instances with session/cache isolation concurrently on the same machine by Firefox Profiles mechanism.
    We need to add -p ProfileName -no-remote parameters to Firefox cmdline.
    For multiple Firefox profiles run together, refer to http://clan.yo2.cn/run-multiple-firefox-profiles-simultaneously.html

    8. There is critical issue for launch Chrome TestRunner in cmdline mode
    User may encounter weird issue if launch Chrome TestRunner in cmdline like firefox.exe -chrome "chrome://selenium-ide-testrunner/content/selenium/TestRunner.html..." . It's hard to resolve. In JRunner, we must choose a workaround substitution.
    For solution, read topic: http://clearspace.openqa.org/message/51801#51801

    Thursday, November 6, 2008

    FIT mode best choice: running Selenium with Firefox Chrome TestRunner

    I don't think Selenium-RC is the best choice for QA who does feature testing & regression testing with blackbox style, esp. those who doesn't have programming background.

    S-RC(JAVA) disadvantages for blackbox QA/regression testing:
    1) programming skills trainning cost for QA
    2) test code complexity for maintenance
    3) bad HTTPS/SSL page support
    4) popup/alerts make automation stuck, require an autoclick alerts mechanism
    5) not stable and with weird issues caused by Proxy Server
    6) not lightweight tools, need setup environment
    7) still need do 2nd around development to integrate with regression distributed system(even if S-GRID).

    FIT mode is familiared by QA. There has 3 ways:
    1) deploy Selenium CORE/TEST on testing server
    2) MS IEHTA with test case repository anywhere
    3) Firefox Chrome by Selenium-IDE with test case repository anywhere

    For 1), some company has restriction to deny to do deployment
    For 2), IEHTA is unstable and has critial issues such as multiwindow popup,xpath slowness etc. It's also hard to provide a session/cache isolation solution for multiple instances running simultaniously on one machine. It's also hard to create a recorder for IE.

    I think running Selenium with Firefox Chrome TestRunner is the best choice for FIT mode.
    1) Selenium-IDE is a good tool with recorder and still can do huge improvements.
    2) Stable and well support HTTPS. Well support multiwindow.
    3) multiple selenium running simultaniously on same machine with session/cache isolation by Firefox profiles mechanism
    4) set FF preferences to let Selenium not stuck(click out popup/alerts)
    5) save run snapshot/results/log to local file system other than postUrl solution
    6) No need any proxy server. No need any additional installation and configuration.
    7) Opensource Firefox platform is developer friendly. Firefox Chrome's XPCOM APIs can support us to implement all kinds of selenium features.

    However, the bottleneck for openqa's FIT mode(Esp. Selenium-IDE) not become enterprise usage:
    1) Selenium-IDE doesn't save down detailed enough snapshot/results/log to local file system which is the significant requirement for enterprise tools. Many other significant requirements also haven't been implemented.
    2) FIT mode doesn't provide neccessary development SDK. We can build our enterprise test case maintenance system with GUI/webflow, parallel execution system with distributed infrastructure, result reporting system with DB/file system. However, we need SDK APIs to integrate Selenium FIT engine into these systems.

    Most of developers are contributing for S-RC product on community. I hope openqa can also push Selenium-IDE FIT product line for enterprise usage.

    Monday, November 3, 2008

    Selenium FIT mode vs. Driven mode

    Selenium has two usage modes:
    1)FIT mode: table style simple script which usually run on Selenium CORE/IDE in standalone IE/FF. We also call it as 'Selenese'. It's simple and easy to install/use for QA with no programming skills.
    This is more for "black box" testing and is "asynchronous" in design.
    You send the whole test script over to the browser in a "REST" fashion, it generally only has the same access to click and type and change things as a regular user would.

    2)Driven mode: a language specific driver for Selenium. It used a proxy server to bypass Same Origin Policy for Selenium. Selenium RC is the product for this mode.
    This enables "white box" testing and also "synchronous" in design.
    With white box testing, you can send one command to the browser in an "RPC" fashion, get a result, then check the back-end database to confirm like Unit Test. You can benifit from programming lauguage like Java for Selenium.

    Read Jason Huggins' topic: Choosing between "Driven Mode" and "FIT or Table style"