TestCafe
3 minutes read

Creating automated tests for your website, web application or mobile application was never an easy task.

Have a look how to get started with Cypress in less than 30 minutes.

Most of the companies started using Selenium WebDriver for test automation, which was a game changer 5-7 years ago when it first started getting traction, but it came with few challenges like:

  • selectors flakiness
  • missing retry mechanism
  • multi-tab browser testing
  • limited reports and report integration
  • lacking mobile testing

The new testing frameworks are trying to improve one or more of the above issues, while offering a more integrated testing frameworks.

Today, we are focusing on Cypress.io and TestCafe.

More details on how to create your testing framework with TestCafe is available in our course: Build an End to End Software Testing Framework with TestCafe Course

TestCafe Course

TestCafe

Prerequisites

  • Chrome, Firefox or Safari browser needs to be installed on the machine where you run the tests

How to install TestCafe

npm install testcafe –save-dev

How it works

TestCafe works by serving the test site via a proxy server, the server injects scripts into the page which can inspect and control elements onto the page.

It works in any web browser or mobile devices and cloud services like BrowserStack and SauceLabs.

TestCafe runs the test code in Node which enables the possibility to call out to parts of your Node server application directly from the tests.

TestCafe uses standard CSS selectors to locate elements.

TestCafe has framework specific extensions for React, Angular or Vue and allows the use of component names as selectors.

TestCafe has nice console output for test failures (similar to Jest) that shows which assertion failed and mitigates the potential issues of having many assertions in a single test.

TestCafe groups tests as a fixture.

Advantages

  • good TestCafe framework documentation
  • big contributor community
  • fast and reliable
  • no Selenium WebDriver needed to run your tests
  • standard CSS selectors
  • feels like a mature project (coming from a commercial project)
  • React selector extension
  • parallel execution supported in the framework
  • synchronisation handled by the framework
  • live reload/retest
  • allow interacting with native alerts
  • debug command for easier test debugging
  • debug made easy by the TestCafe UI (step over, resume)
  • Screenshots generated on fail
  • JavaScript errors are caught by the framework
  • cross browser support: Firefox, Chrome, Safari
  • Jenkins error reporting integrated
  • Headless browser Chrome without the need to install it by using: “testcafe-browser-provider-puppeteer
  • integrates easily with SauceLabs and BrowserStack
  • TestCafe Studio – record and playback tool (TestCafe alternative to Selenium IDE)

Disadvantages

  • Still under development

Cypress

Prerequisites
Browsers needs to be installed locally to run tests on Chrome.

How to install
npm install cypress –save-dev

How it works

Cypress works by controlling the browser via proprietary automation APIs and needs a new driver for every supported browser.

Cypress supports Chrome, Firefox and Edge browsers, check the full list of supported browsers by Cypress.

Cypress runs your actual test code in the browser process and communicates with the app via HTTP or executing shell commands.

Cypress uses jQuery selectors, also can use the useful extra capabilities like :parent and :first.

Cypress goes one further by having a dedicated Electron app that shows your tests side by side with the site under test.

Hovering over a test step shows a snapshot of the DOM for that step which makes debugging easier.

Cypress uses (a fork of) Mocha as its test runner with Chai for assertions and Sinon for mocking.

Advantages

  • good documentation
  • big community
  • fast & reliable
  • jQuery selectors (extra capabilities like :parent and :first)
  • synchronisation done by the framework
  • Live reload/retest
  • Easy debugging in the CypressIO UI
  • JS errors caught by the framework
  • Screenshots & Video recording out of the box
  • possibility to go back to previous states, visually
  • parallel execution
  • cross browser support
  • open source (MIT licensed)

Disadvantages

  • no integration with BrowserStack or SauceLabs
  • error reporting needs improvement

TestCafe pros

  • good documentation
  • big community
  • fast and reliable
  • standard CSS selectors
  • mature project
  • parallel execution
  • live reload/retest
  • React selector extension
  • native interaction
  • test debugging
  • screenshot generation
  • cross browser support
  • headless testing
  • SauceLabs integration
  • BrowserStack integration

Cypress pros

  • good documentation
  • big community
  • fast and reliable
  • jQuery selectors
  • mature project
  • parallel execution
  • live reload/retest
  • Easy debugging
  • Screenshots & Video
  • debug previous states
  • cross browser support
  • headless testing
  • user oriented test design
  • open source

Conclusions

TestCafe offers a more similar approach to pure JavaScript, where you get the values from a page and then you assert that those values are correct, works seamlessly with async/awaits.

More details on how to publish TestCafe reports in Jenkins.

Cypress, on the other hand, offers a more user oriented approach, when you select the element you want to interact with and you have to do the assertion on the spot, this approach can be a little cumbersome at the beginning.