Runner Class

An object that configures and launches test tasks.

Created using the testCafe.createRunner function.

Example

const createTestCafe = require('testcafe');
let testcafe         = null;

createTestCafe('localhost', 1337, 1338)
    .then(tc => {
        testcafe     = tc;
        const runner = testcafe.createRunner();

        return runner
            .src(['tests/fixture1.js', 'tests/func/fixture3.js'])
            .browsers(['chrome', 'safari'])
            .run();
    })
    .then(failedCount => {
        console.log('Tests failed: ' + failedCount);
        testcafe.close();
    });

Methods

src

Configures the test runner to run tests from the specified files.

src(source) → this
Parameter Type Description
source String | Array The relative or absolute path to a test fixture file, or several such paths. You can use glob patterns to include (or exclude) multiple files.

If you call the method several times, all the specified sources are added to the test runner.

Examples

runner.src(['/home/user/tests/fixture1.js', 'fixture5.js']);
runner.src(['/home/user/tests/**/*.js', '!/home/user/tests/foo.js']);

filter

Allows you to select which tests should be run.

filter(callback) → this
Parameter Type Description
callback function(testName, fixtureName, fixturePath, testMeta, fixtureMeta) The callback that determines if a particular test should be run.

The callback function is called for each test in the files specified using the src method.

Return true from the callback to include the current test or false to exclude it.

The callback function accepts the following arguments:

Parameter Type Description
testName String The name of the test.
fixtureName String The name of the test fixture.
fixturePath String The path to the test fixture file.
testMeta Object<String, String> The test metadata.
fixtureMeta Object<String, String> The fixture metadata.

Example

runner.filter((testName, fixtureName, fixturePath, testMeta, fixtureMeta) => {
    return fixturePath.startsWith('D') &&
        testName.match(someRe) &&
        fixtureName.match(anotherRe) &&
        testMeta.mobile === 'true' &&
        fixtureMeta.env === 'staging';
});

browsers

Configures the test runner to run tests in the specified browsers.

browsers(browser) → this

The browser parameter can be any of the following objects or an Array of them:

Parameter Type Description Browser Type
String A different browser alias for each browser type. See Browser Support for more details. Local browsers, cloud browsers, and browsers accessed through browser provider plugins.
{path: String, cmd: String} The path to the browser's executable (path) and command line parameters (cmd). The cmd property is optional. Local and portable browsers
BrowserConnection The remote browser connection. Remote browsers

You can use different object types in one function call. If you call the method several times, all the specified browsers are added to the test runner.

Using Browser Aliases

  • running local browsers
runner.browsers(['safari', 'chrome']);
  • running browsers accessed through browser provider plugins
runner.browsers('saucelabs:Chrome@52.0:Windows 8.1');
runner.browsers('chrome:headless');

Specifying the Path to the Browser Executable

Use the path: prefix. Enclose the path in backticks if it contains spaces.

runner.browsers('path:`C:\\Program Files\\Internet Explorer\\iexplore.exe`');

Specifying the Path with Command Line Parameters

runner.browsers({
    path: 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
    cmd: '--new-window'
});

Passing a Remote Browser Connection

const createTestCafe = require('testcafe');
let runner           = null;
let testcafe         = null;

createTestCafe('localhost', 1337, 1338)
    .then(tc => {
        testcafe = tc;
        runner   = testcafe.createRunner();

        return testcafe.createBrowserConnection();
    })
    .then(remoteConnection => {

        // Outputs remoteConnection.url so that it can be visited from the remote browser.
        console.log(remoteConnection.url);

        remoteConnection.once('ready', () => {
            runner
                .src('test.js')
                .browsers(remoteConnection)
                .run()
                .then(failedCount => {
                    console.log(failedCount);
                    testcafe.close();
                 });
        });
    });

screenshots

Enables TestCafe to take screenshots of the tested webpages.

screenshots(path [, takeOnFails, pathPattern]) → this
Parameter Type Description Default
path String The base path where the screenshots are saved. Note that to construct a complete path to these screenshots, TestCafe uses default path patterns. You can override these patterns using the method's screenshotPathPattern parameter.
takeOnFails (optional) Boolean Specifies if screenshots should be taken automatically when a test fails. false
sceenshotPathPattern (optional) String The pattern to compose screenshot files' relative path and name. See --screenshot-path-pattern for information about the available placeholders.

The screenshots function should be called to allow TestCafe to take screenshots when the t.takeScreenshot action is called from test code.

Set the takeOnFails parameter to true to take a screenshot when a test fails.

TestCafe does not take screenshots if the screenshots function is not called.

Example

runner.screenshots('reports/screenshots/', true, '${DATE}_${TIME}/test-${TEST_INDEX}/${USERAGENT}/${FILE_INDEX}.png');

reporter

Configures TestCafe's reporting feature.

reporter(name [, outStream]) → this
Parameter Type Description Default
name String The name of the reporter to use.
outStream (optional) Writable Stream implementer The stream to which the report is written. stdout

To use multiple reporters, call this method several times with different reporter names. Note that only one reporter can write to stdout.

Specifying the Reporter

runner.reporter('minimal');

Saving the Report to a File

const stream = fs.createWriteStream('report.xml');

runner
    .src('tests/sample-fixture.js')
    .browsers('chrome')
    .reporter('xunit', stream)
    .run()
    .then(failedCount => {
        stream.end();
    });

Using Multiple Reporters

const stream = fs.createWriteStream('report.json');

runner
    .src('tests/sample-fixture.js')
    .browsers('chrome')
    .reporter('json', stream)
    .reporter('list')
    .run()
    .then(failedCount => {
        stream.end();
    });

Implementing a Custom Stream

class MyStream extends stream.Writable {
    _write(chunk, encoding, next) {
        doSomething(chunk);
        next();
    }
}

const stream = new MyStream();
runner.reporter('json', stream);

You can also build your own reporter. Use a dedicated Yeoman generator to scaffold out a reporter plugin.

concurrency

Specifies that tests should run concurrently.

concurrency(n) → this

TestCafe opens n instances of the same browser and creates a pool of browser instances. Tests are run concurrently against this pool, that is, each test is run in the first available instance.

The concurrency function takes the following parameters:

Parameter Type Description
n Number The number of browser instances that are invoked.

See Concurrent Test Execution to learn more about concurrent test execution.

The following example shows how to run tests in three Chrome instances:

runner
    .browsers('chrome')
    .concurrency(3);

startApp

Specifies a shell command that is executed before running tests. Use it to launch or deploy the application that is tested.

async startApp(command, initDelay) → this

After the testing is finished, the application is automatically terminated.

The startApp function takes the following parameters:

Parameter Type Description Default
command String The shell command to be executed.
initDelay (optional) Number The amount of time (in milliseconds) allowed for the command to initialize the tested application.

TestCafe adds node_modules/.bin to PATH so that you can use binaries the locally installed dependencies provide without prefixes.

Example

runner.startApp('node server.js', 4000);

useProxy

Specifies the proxy server used in your local network to access the Internet. Allows you to bypass the proxy when accessing specific resources.

async useProxy(host [, bypassRules]) → this
Parameter Type Description
host String The proxy server host.
bypassRules (optional) String | Array A set of rules that specify which resources are accessed bypassing the proxy.

If you access the Internet through a proxy server, use the useProxy method to specify its host.

When using a proxy server, you may still need to access local or external resources directly. In this instance, provide their URLs in the bypassRules option.

The bypassRules parameter takes one or several URLs that require direct access. You can replace parts of the URL with the * wildcard that corresponds to a string of any length. Wildcards at the beginning and end of the rules can be omitted (*.mycompany.com and .mycompany.com have the same effect).

Examples

The following example shows how to use the proxy server at proxy.corp.mycompany.com:

runner.useProxy('proxy.corp.mycompany.com');

In the example below, the proxy server address is 172.0.10.10:8080 and two resources at localhost:8080 and internal-resource.corp.mycompany.com are accessed directly.

runner.useProxy('172.0.10.10:8080', ['localhost:8080', 'internal-resource.corp.mycompany.com']);

The *.mycompany.com proxy bypass rule means that all URLs in mycompany.com subdomains are accessed directly.

runner.useProxy('proxy.corp.mycompany.com', '*.mycompany.com');

You can also use the proxy host to specify authentication credentials.

runner.useProxy('username:password@proxy.mycorp.com');

run

Runs tests according to the current configuration. Returns the number of failed tests.

async run(options) → Promise<Number>

Make sure to keep the browser tab that is running tests active. Do not minimize the browser window. Inactive tabs and minimized browser windows switch to a lower resource consumption mode where tests are not guaranteed to execute correctly.

You can pass the following options to the runner.run function.

Parameter Type Description Default
skipJsErrors Boolean Defines whether to continue running a test after a JavaScript error occurs on a page (true), or consider such a test failed (false). false
skipUncaughtErrors Boolean Defines whether to continue running a test after an uncaught error or unhandled promise rejection occurs on the server (true), or consider such a test failed (false). false
quarantineMode Boolean Defines whether to enable the quarantine mode. false
debugMode Boolean Specifies if tests run in the debug mode. If this option is enabled, test execution is paused before the first action or assertion allowing you to invoke the developer tools and debug. In the debug mode, you can execute the test step-by-step to reproduce its incorrect behavior. You can also use the Unlock page switch in the footer to unlock the tested page and interact with its elements. false
debugOnFail Boolean Specifies whether to enter the debug mode when a test fails. If enabled, the test is paused at the moment it fails, so that you can explore the tested page to determine what caused the failure. false
selectorTimeout Number Specifies the time (in milliseconds) within which selectors make attempts to obtain a node to be returned. See Selector Timeout. 10000
assertionTimeout Number Specifies the time (in milliseconds) within which TestCafe makes attempts to successfully execute an assertion if a selector property or a client function was passed as an actual value. See Smart Assertion Query Mechanism. 3000
pageLoadTimeout Number Specifies the time (in milliseconds) passed after the DOMContentLoaded event, within which TestCafe waits for the window.load event to fire. After the timeout passes or the window.load event is raised (whichever happens first), TestCafe starts the test. You can set this timeout to 0 to skip waiting for window.load. 3000
speed Number Specifies the test execution speed. Should be a number between 1 (the fastest) and 0.01 (the slowest). If speed is also specified for an individual action, the action speed setting overrides test speed. 1
stopOnFirstFail Boolean Defines whether to stop an entire test run if any test fails. This allows you not to wait for all the tests included in the test task to finish and allows focusing on the first error. false
disableTestSyntaxValidation Boolean Defines whether to disable checks for test and fixture directives in test files. Use this option to run dynamically loaded tests. See details in the --disable-test-syntax-validation command line option description. false

After all tests are finished, call the testcafe.close function to stop the TestCafe server.

Example

const createTestCafe = require('testcafe');
let testcafe         = null;

createTestCafe('localhost', 1337, 1338)
    .then(tc => {
        testcafe     = tc;
        const runner = testcafe.createRunner();

        return runner.run({
            skipJsErrors: true,
            quarantineMode: true,
            selectorTimeout: 50000,
            assertionTimeout: 7000,
            pageLoadTimeout: 8000,
            speed: 0.1,
            stopOnFirstFail: true
        });
    })
    .then(failed => {
        console.log('Tests failed: ' + failed);
        testcafe.close();
    })
    .catch(error => { /* ... */ });

If a browser stops responding while it executes tests, TestCafe restarts the browser and reruns the current test in a new browser instance. If the same problem occurs with this test two more times, the test run finishes and an error is thrown.

Cancelling Test Tasks

You can stop an individual test task at any moment by cancelling the corresponding promise.

const taskPromise = runner
    .src('tests/fixture1.js')
    .browsers([remoteConnection, 'chrome'])
    .reporter('json')
    .run();

taskPromise.cancel();

You can also cancel all pending tasks at once using the runner.stop function.

Quarantine Mode

The quarantine mode is designed to isolate non-deterministic tests (that is, tests that pass and fail without any apparent reason) from the other tests.

When the quarantine mode is enabled, tests run according to the following logic:

  1. A test runs at the first time. If it passes, TestCafe proceeds to the next test.
  2. If the test fails, it runs again until it passes or fails three times.
  3. The most frequent outcome is recorded as the test result.
  4. If the test result differs between test runs, the test is marked as unstable.

Note that it increases the test task's duration if you enable quarantine mode on your test machine because failed tests are executed three to five times.

See Martin Fowler's Eradicating Non-Determinism in Tests article for more information about non-deterministic tests.

stop

Stops all the pending test tasks.

async stop()

You can also stop an individual pending task by cancelling the corresponding promise.