SELENIUM BOX - Frequently asked questions


GENERAL

Is Selenium Box a cloud service like Saucelabs or Browserstack?

No, Selenium Box is an on premise Selenium Grid solution making it extremely secure and scalable. No external access to the application under test is required.

 

Can I install the selenium box software in my own environment? 

Yes. That is the intended use of the Selenium Box. While we do also offer a hardware solution we highly recommend to install the Selenium Box into your own environment. That way it is very easy for you to scale and integrate with your other services and tools.   

 

What do I need to run the Selenium Box in my environment?

Selenium Box runs on any Linux system with Docker installed. It can run on bare metal as well as in a virtual machine. For both a 200+ GB fast disk (i.e. SSD) and 16 core / 16 GB RAM machine is required. We can provide benchmarking scripts for your infrastructure in order to ensure that you can get the maximum performance out of the Selenium Box.

 

Can I also purchase a hardware box that contains the Selenium Box software?

Yes. You can also purchase a ready made Selenium Box including the hardware from us. Or else you can provide the hardware to us and we install Selenium Box on it.

 

How does the license model work?

Selenium Box is based on an annual subscription for a master license. Upon activation, customers have the right to use a Selenium Box master license. Each Selenium Box Hub requires a master license. An unlimited number of nodes can be attached to a master hub. For most installations, the subscription to a single master license is sufficient.

 

Do you provide AN evaluation license?

Yes. Clients can request an evaluation license if they want to try out Selenium Box. We usually prepare a virtual machine which the customer can run in their environment.

 

Can I download a standard Selenium Box and quickly install it if I want to try it out?

No. This is because each customer is usually slightly different. Therefore we want to make sure that we provide a virtual machine specifically tailored to your needs. That way you get the most out of the evaluation period.

 

Can I do a proof of concept in my environment?

Yes. This is our recommended approach. Especially larger companies have specific requirements in terms of infrastructure, network etc. We strongly recommend a proof of concept.

 

Is there a limit on running tests in parallel?

No. You can run as many tests in parallel as your hardware resources are capable of. If you want to run more tests, you simply need to add more CPU / RAM.

 

Do I have to pay more when I want to run more tests in parallel?

No. A master license allows users to run as many parallel tests as their system resources allow. Customers can therefore scale virtually unlimited in terms of parallel test slots.

 

Is there a limitation on testing minutes that I can use?

No. The testing minutes are unlimited due to the on premise nature of Selenium Box.

 

is there any logging / monitoring capability built into Selenium Box?

Yes. Selenium Box has an advanced logging and monitoring mechanism which can easily be customized and integrated into the customer environment. Selenium Box logs pretty much everything. Those logs can be accessed and then aggregated and visualized based on your needs. 


OPERATING THE SELENIUM BOX

Can I run the Selenium Box in my company infrastructure?

Yes. Selenium Box can be installed in any Linux system that supports Docker.

 

Does Selenium Box automatically update when y new browser is released? 

Yes. There is an auto-update option which automatically downloads and installs new browser, driver or Selenium versions after they are tested and released by Element34. This makes the Selenium Box completely maintenance free.

 

How does the update process work?

After new browsers, Selenium or driver versions are released we thoroughly test them and release them. Selenium Box immediately (if auto-update is enabled) picks up the newly released versions and installs them. From a user perspective this process is completely transparent and seamless.

 

Does the Selenium Box support HTTPS? 

Yes, you can of course use https with the Selenium Box. 

 

Can I install my own Windows image in the Selenium Box in order to run my company Windows image and browser?

Yes. Corporate Windows images can be installed into the Selenium Box system.

 

How much effort is it to operate the Selenium Box?

Selenium Box is a fully self contained system. There is no effort required to manage the Selenium Box when running in auto-update mode. The host where the Selenium Box is running on is usually provisioned by the corporate operations team.

 

If I run out of capacity on my main Selenium Box, can I scale by adding more boxes/nodes?

Yes. You can simply add additional nodes if you wish you increase capacity for running more tests in parallel.

 

Do I need to do anything when a new browser or Selenium version releases?

No. Selenium Box can automatically update itself when we release a new binary. Customers however have the option to manually upgrade if that's what they want. Selenium Box issues an alert that there is a new version available. The upgrade is done by simply selecting the new version the the maintenance console of the Selenium Box.

 

How long does it take to have the latest browser and Selenium Versions available on the Selenium Box after they are released by the browser manufacturers?

We monitor the release of new browsers 24/7. Each time a new browser, Selenium or driver version is released we automatically build the corresponding images, run them through thorough tests and if everything works well we release them to the Selenium Box. All of this procedure is automated. Most of the time the new images are released to the Selenium Box within 24-48 hours of the initial release.

 

Do I need to check the compatibility of the new browsers / Selenium releases?

No, as described above, we do this for you. When we release new images to the Selenium Box, our customers can be sure that the versions are compatible with each other.


RUNNING TESTS ON THE SELENIUM BOX

Do I need to write my tests in a special way to execute them on the Selenium Box?

No. Selenium Box uses the standard Selenium protocol and Desired Capabilities. 

 

Do tests share browser resources? 

No. Each test is run in an isolated environment (Container or VM) and does not share any resources with other tests. A new browser is started for each test. After the test finishes, the browser is closed and the environment is destroyed. That way each test is completely isolated and there is no chance of side effects between browsers / environments. 

 

How can I direct my tests to the Selenium Box?

Running your tests on the Selenium Box is simply a matter of pointing the URL of your remote webdriver to the Selenium Box entry point. This is usually a one line change in a configuration script and therefore can be implemented really easily.

 

What browsers are supported?

All browsers supported by Selenium are supported: Chrome, Firefox, Internet Explorer, EDGE, MacOS Safari.

 

HOW do I run my tests on MacOS Safari?

In order to run tests on MacOS Safari, you need Apple hardware i.e. a Mac Pro. This is a legal requirements by Apple. The Apple hardware is then configured to be part of the Selenium Box system and seamlessly integrates into it. If a test with a desired capability Safari is sent to the Selenium Box Hub, it is dispatched to the Safari Node(s) and run there.

 

Does the Selenium Box support mobile?

Yes. We currently have support for mobile (iOS + Android). In the next release around Mid-2017, we will fully support mobile via Appium in the same fashion as currently for the web.

 

Does the Selenium Box support Windows IE/EDGE?

Yes. You can run Windows IE and EDGE browsers seamlessly on the Selenium Box. You can use a plain vanilla Windows image or provide your own corporate image which is then installed on the Selenium Box.

 

Can I use self signed SSL certificates?

Yes, Selenium Box also supports self signed SSL certificates.  

 

How can I scale if I need more parallel tests?

Scaling can very easily be done by either adding more CPU / RAM to the main Selenium Box. Alternatively additional nodes can be added to the main Selenium Box in order to increase capacity.

 

Can I take videos of my tests?

Yes. A video can be taken on a per test basis. The videos can then be easily integrated into the reporting system or CI system.

 

Can I watch my test running on the Selenium Box in real time?

Yes. Selenium Box provides a live view feature which allows to watch tests in real time while running.

 

Can I test older browser versions as well?

Yes. After a new browser or Selenium image is released to the Selenium Box, older browser versions are still available. Thus tests can be executed on the latest as well as older browser versions.