Selenium vs Browserbite – what bugs can you find?

Selenium vs browserbiteMany of our users have asked us some questions regarding how Browserbite compares to Selenium (or any other test automation tool). In this post we try to explain this.

The basics

Selenium is a tool for test automation. We will not get into details here, but various components that make up Selenium are meant to be used by programmers to write automated tests. Most tests written in Selenium are functional ie. meant to test features.

Browserbite can be categorized as a test assist tool in most usage scenarios. It is meant to be a simple tool for the end-user to detect layout issues on web pages.

What do you need to do to write a test?

Selenium requires you to create a test in one of the supported programming languages. Java seems to be the most popular one, but there is also support for others like C#, Python, Ruby, PHP and Perl. In short – you need to code. There’s also a tool called Selenium IDE that helps you to get going and tries to record the test for you. In order to run the test, you should establish a pass criteria – for instance the script needs to check whether a text is present on a web page.

Browserbite requires just the website address to be tested and the selection of the baseline browser. Baseline browser is the oracle – the one that the user has manually validated to be correct. All other browser renderings are compared to the baseline. For example, if you are using Chrome as a baseline, layout differences compared to Chrome will be highlighted by cross browser testing algorithm.

What Browserbite finds and Selenium does not?

In a nutshell, Selenium finds functional issues (in case the tests have scripted correctly). If a script  has been written to open up the Google webpage and search for “Browserbite” a results page should be displayed with a link to our landing page www.browserbite.com. So a programmer would want to assert whether a link containing “www.browserbite.com” is present on the web page. Since you can run Selenium on many browsers, you have the opportunity to validate whether the text is present on all of the browsers.

Browserbite finds layout issues (does the page look similar to the baseline). So if you would open up the same page – you would see highlighted layout differences on different browsers. Selenium will only care whether the text is present on the page (even if it is not visible to the user). Browserbite will highlight this as an issue if something is not shown correctly.

An example would be appropriate. Look at the test that has been run on www.scandesigns.dk website. It clearly shows that none of the Internet Explorers in the test show the height correctly for the country flags. This is a perfect example of what Browserbite can find.

Scandicdesigns

Chrome on the left and Internet Explorer 9 on the right shows that the flags have gone missing in the Microsoft browser. An excerpt from the www.scandicdesigns.dk site. Click here to see the full report!

The interesting part here is that layout issues lead to functional issues in real life. For instance, if a button is not visible in Firefox, this leads to a functional loss for the user. The user cannot click the button. We have seen real-life examples where the “Buy now” button has not been visible for some end-users. This normally results in lost revenues.

What Selenium finds and Browserbite does not?

Twitter whale would not be detected as an issue in a classic cross browser test.

Twitter whale would not be detected as an issue in a classic cross browser test.

There are scenarios where layout testing won’t find issues in a web application, but a functional testing tool will. An example:  if you have a feature behind a login-password protection and the newest build has broken it. In this case Browserbite (at its current setting) would simply identify that all browsers display “Error 500”. Usually browsers display that in a different way – so a human follow up identifies the issue. However, in the case of the world famous “Twitter whale” Selenium would detect a missing text. Browserbite would not identify an issue due to the error page being cross browser compatible!

Conclusion

Comparing Selenium and Browserbite is like comparing apples to oranges. The tools are for different audiences and for different purposes. As usual in tool world – they actually compliment each other in a good way! If you are forced to choose which one to start with then we’ll give you a simple checklist.

Choose Selenium when:

  • The application under test is stable (a simple rule of thumb is that you’ll be able to run the test script at least 8-10 times without changing a line)
  • You have coding skills
  • You have environments to test on (or are willing to rent) – Sauce Labs is a good starting place
  • You’re looking for functional bugs

Choose Browserbite when:

  • You are looking for layout issues
  • The application changes often
  • You have no people around who can code
  • You don’t want to set up your own test lab
Posted by

Comments are closed.