Runner Class

An object that configures and launches test tasks.

Created using the testCafe.createRunner function.

Example

const createTestCafe = require('testcafe');
let testcafe         = null;

createTestCafe('localhost', 1337, 1338)
    .then(tc => {
        testcafe     = tc;
        const runner = testcafe.createRunner();

        return runner
            .src(['tests/fixture1.js', 'tests/func/fixture3.js'])
            .browsers(['chrome', 'safari'])
            .run();
    })
    .then(failedCount => {
        console.log('Tests failed: ' + failedCount);
        testcafe.close();
    });

Methods

src

Configures the test runner to run tests from the specified files.

src(source) → this
Parameter Type Description
source String | Array The relative or absolute path to a test fixture file, or several such paths. You can use glob patterns to include (or exclude) multiple files.

You do not need to call this function if you specify the src property in the configuration file.

Related configuration file property: src

Examples

runner.src(['/home/user/tests/fixture1.js', 'fixture5.js']);
runner.src(['/home/user/tests/**/*.js', '!/home/user/tests/foo.js']);

filter

Allows you to select which tests should be run.

filter(callback) → this
Parameter Type Description
callback function(testName, fixtureName, fixturePath, testMeta, fixtureMeta) The callback that determines if a particular test should be run.

The callback function is called for each test in the files the src method specifies.

Return true from the callback to include the current test or false to exclude it.

The callback function accepts the following arguments:

Parameter Type Description
testName String The name of the test.
fixtureName String The name of the test fixture.
fixturePath String The path to the test fixture file.
testMeta Object<String, String> The test metadata.
fixtureMeta Object<String, String> The fixture metadata.

Related configuration file property: filter

Example

runner.filter((testName, fixtureName, fixturePath, testMeta, fixtureMeta) => {
    return fixturePath.startsWith('D') &&
        testName.match(someRe) &&
        fixtureName.match(anotherRe) &&
        testMeta.mobile === 'true' &&
        fixtureMeta.env === 'staging';
});

browsers

Configures the test runner to run tests in the specified browsers.

browsers(browser) → this

The browser parameter can be any of the following objects or an Array of them:

Parameter Type Description Browser Type
String | Array A different browser alias for each browser type. See Browser Support for more details. Local browsers, cloud browsers, and browsers accessed through browser provider plugins.
{path: String, cmd: String} The path to the browser's executable (path) and command line parameters (cmd). The cmd property is optional. Local and portable browsers
BrowserConnection The remote browser connection. Remote browsers

You do not need to call this function if you specify the browsers property in the configuration file.

Related configuration file property: browsers

Using Browser Aliases

  • running local browsers
runner.browsers(['safari', 'chrome']);
  • running browsers accessed through browser provider plugins
runner.browsers('saucelabs:Chrome@52.0:Windows 8.1');

Specifying the Path to the Browser Executable

Use the path: prefix. Enclose the path in backticks if it contains spaces.

runner.browsers('path:`C:\\Program Files\\Internet Explorer\\iexplore.exe`');

Specifying the Path with Command Line Parameters

runner.browsers({
    path: 'C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe',
    cmd: '--new-window'
});

Headless Mode, Device Emulation and User Profiles

You can add postfixes to browser aliases to run tests in the headless mode, use Chrome device emulation or user profiles.

runner.browsers('chrome:headless');

For portable browsers, use the browser alias followed by the path to an executable.

runner.browsers('firefox:/home/user/apps/firefox.app:userProfile');

The path: prefix does not support postfixes.

Passing a Remote Browser Connection

const createTestCafe = require('testcafe');
let runner           = null;
let testcafe         = null;

createTestCafe('localhost', 1337, 1338)
    .then(tc => {
        testcafe = tc;
        runner   = testcafe.createRunner();

        return testcafe.createBrowserConnection();
    })
    .then(remoteConnection => {

        // Outputs remoteConnection.url so that it can be visited from the remote browser.
        console.log(remoteConnection.url);

        remoteConnection.once('ready', () => {
            runner
                .src('test.js')
                .browsers(remoteConnection)
                .run()
                .then(failedCount => {
                    console.log(failedCount);
                    testcafe.close();
                 });
        });
    });

screenshots

Enables TestCafe to take screenshots of the tested webpages.

screenshots(path [, takeOnFails, pathPattern]) → this
Parameter Type Description Default
path String The base path where the screenshots are saved. Note that to construct a complete path to these screenshots, TestCafe uses the default path patterns. You can override these patterns using the method's screenshotPathPattern parameter.
takeOnFails (optional) Boolean Specifies if screenshots should be taken automatically when a test fails. false
sceenshotPathPattern (optional) String The pattern to compose screenshot files' relative path and name. See Path Pattern Placeholders for information about the available placeholders.

TestCafe does not take screenshots if the screenshots function is not called.

See Screenshots for details.

Related configuration file properties:

Example

runner.screenshots('reports/screenshots/', true, '${DATE}_${TIME}/test-${TEST_INDEX}/${USERAGENT}/${FILE_INDEX}.png');

video

Enables TestCafe to record videos of test runs.

video(path [, options, encodingOptions]) → this
Parameter Type Description
path String The base directory where videos are saved. Relative paths to video files are composed according to path patterns. You can also use the options.pathPattern property to specify a custom pattern.
options (optional) Object Options that define how videos are recorded. See Basic Video Options for a list of options.
encodingOptions (optional) Object Options that specify video encoding. You can pass all the options supported by the FFmpeg library. Refer to the FFmpeg documentation for information about the available options.

See Record Videos for details.

Related configuration file properties:

Example

runner.video('reports/videos/', {
    singleFile: true,
    failedOnly: true,
    pathPattern: '${TEST_INDEX}/${USERAGENT}/${FILE_INDEX}.mp4'
}, {
    r: 20,
    aspect: '4:3'
});

reporter

Configures TestCafe's reporting feature.

reporter(name, output) → this
reporter([ name | { name, output }]) → this
Parameter Type Description Default
name String The name of the reporter to use.
output (optional) String | Writable Stream implementer The file path where the report is written or the output stream. stdout

To use a single reporter, specify a reporter name and, optionally, an output target as the second parameter.

To use multiple reporters, pass an array to this method. This array can include both strings (the reporter name) and { name, output } objects (if you wish to specify the output target). See examples below.

Note that if you use multiple reporters, only one can write to stdout.

Related configuration file property: reporter

Specifying the Reporter

runner.reporter('minimal');

Saving the Report to a File

runner.reporter('xunit', 'reports/report.xml');

Using Multiple Reporters

runner.reporter(['spec', {
    name: 'json',
    output: 'reports/report.json'
}]);

Implementing a Custom Stream

class MyStream extends stream.Writable {
    _write(chunk, encoding, next) {
        doSomething(chunk);
        next();
    }
}

const stream = new MyStream();
runner.reporter('json', stream);

You can also build your own reporter. Use a dedicated Yeoman generator to scaffold out a reporter plugin.

concurrency

Specifies that tests should run concurrently.

concurrency(n) → this

TestCafe opens n instances of the same browser and creates a pool of browser instances. Tests are run concurrently against this pool, that is, each test is run in the first available instance.

The concurrency function takes the following parameters:

Parameter Type Description
n Number The number of browser instances that are invoked.

See Concurrent Test Execution to learn more about concurrent test execution.

Related configuration file property: concurrency

The following example shows how to run tests in three Chrome instances:

runner
    .browsers('chrome')
    .concurrency(3);

startApp

Specifies a shell command that is executed before running tests. Use it to launch or deploy the application that is tested.

async startApp(command, initDelay) → this

After the testing is finished, the application is automatically terminated.

The startApp function takes the following parameters:

Parameter Type Description Default
command String The shell command to be executed.
initDelay (optional) Number The amount of time (in milliseconds) allowed for the command to initialize the tested application.

TestCafe adds node_modules/.bin to PATH so that you can use binaries the locally installed dependencies provide without prefixes.

Related configuration file properties:

Example

runner.startApp('node server.js', 4000);

useProxy

Specifies the proxy server used in your local network to access the Internet. Allows you to bypass the proxy when accessing specific resources.

async useProxy(host [, bypassRules]) → this
Parameter Type Description
host String The proxy server host.
bypassRules (optional) String | Array A set of rules that specify which resources are accessed bypassing the proxy.

If you access the Internet through a proxy server, use the useProxy method to specify its host.

When using a proxy server, you may still need to access local or external resources directly. In this instance, provide their URLs in the bypassRules option.

The bypassRules parameter takes one or several URLs that require direct access. You can replace parts of the URL with the * wildcard that corresponds to a string of any length. Wildcards at the beginning and end of the rules can be omitted (*.mycompany.com and .mycompany.com have the same effect).

Related configuration file properties:

Examples

The following example shows how to use the proxy server at proxy.corp.mycompany.com:

runner.useProxy('proxy.corp.mycompany.com');

In the example below, the proxy server address is 172.0.10.10:8080 and two resources at localhost:8080 and internal-resource.corp.mycompany.com are accessed directly.

runner.useProxy('172.0.10.10:8080', ['localhost:8080', 'internal-resource.corp.mycompany.com']);

The *.mycompany.com proxy bypass rule means that all URLs in mycompany.com subdomains are accessed directly.

runner.useProxy('proxy.corp.mycompany.com', '*.mycompany.com');

You can also use the proxy host to specify authentication credentials.

runner.useProxy('username:password@proxy.mycorp.com');

run

Runs tests according to the current configuration. Returns the number of failed tests.

async run(options) → Promise<Number>

Before TestCafe runs tests, it reads settings from the .testcaferc.json configuration file if this file exists. Then it applies settings specified in the programming API. API settings override values from the configuration file in case they differ. TestCafe prints information about every overridden property in the console.

Make sure to keep the browser tab that is running tests active. Do not minimize the browser window. Inactive tabs and minimized browser windows switch to a lower resource consumption mode where tests are not guaranteed to execute correctly.

You can pass the following options to the runner.run function.

Parameter Type Description Default
skipJsErrors Boolean Defines whether to continue running a test after a JavaScript error occurs on a page (true), or consider such a test failed (false). false
skipUncaughtErrors Boolean Defines whether to continue running a test after an uncaught error or unhandled promise rejection occurs on the server (true), or consider such a test failed (false). false
quarantineMode Boolean Defines whether to enable the quarantine mode. false
debugMode Boolean Specifies if tests run in the debug mode. If this option is enabled, test execution is paused before the first action or assertion allowing you to invoke the developer tools and debug. In the debug mode, you can execute the test step-by-step to reproduce its incorrect behavior. You can also use the Unlock page switch in the footer to unlock the tested page and interact with its elements. false
debugOnFail Boolean Specifies whether to enter the debug mode when a test fails. If enabled, the test is paused at the moment it fails, so that you can explore the tested page to determine what caused the failure. false
selectorTimeout Number Specifies the time (in milliseconds) within which selectors make attempts to obtain a node to be returned. See Selector Timeout. 10000
assertionTimeout Number Specifies the time (in milliseconds) within which TestCafe makes attempts to successfully execute an assertion if a selector property or a client function was passed as an actual value. See Smart Assertion Query Mechanism. 3000
pageLoadTimeout Number Specifies the time (in milliseconds) TestCafe waits for the window.load event to fire after the DOMContentLoaded event. After the timeout passes or the window.load event is raised (whichever happens first), TestCafe starts the test. You can set this timeout to 0 to skip waiting for window.load. 3000
speed Number Specifies the test execution speed. A number between 1 (fastest) and 0.01 (slowest). If an individual action's speed is also specified, the action speed setting overrides the test speed. 1
stopOnFirstFail Boolean Defines whether to stop a test run if a test fails. You do not need to wait for all the tests to finish to focus on the first error. false

After all tests are finished, call the testcafe.close function to stop the TestCafe server.

Related configuration file properties:

Example

const createTestCafe = require('testcafe');
let testcafe         = null;

createTestCafe('localhost', 1337, 1338)
    .then(tc => {
        testcafe     = tc;
        const runner = testcafe.createRunner();

        return runner.run({
            skipJsErrors: true,
            quarantineMode: true,
            selectorTimeout: 50000,
            assertionTimeout: 7000,
            pageLoadTimeout: 8000,
            speed: 0.1,
            stopOnFirstFail: true
        });
    })
    .then(failed => {
        console.log('Tests failed: ' + failed);
        testcafe.close();
    })
    .catch(error => { /* ... */ });

If a browser stops responding while it executes tests, TestCafe restarts the browser and reruns the current test in a new browser instance. If the same problem occurs with this test two more times, the test run finishes and an error is thrown.

When you use a LiveModeRunner, you can call the runner.run method only once. In rare cases when you need multiple live mode sessions running in parallel, you can create several TestCafe server instances.

Cancelling Test Tasks

You can stop an individual test task at any moment by cancelling the corresponding promise.

const taskPromise = runner
    .src('tests/fixture1.js')
    .browsers([remoteConnection, 'chrome'])
    .reporter('json')
    .run();

taskPromise.cancel();

You can also cancel all pending tasks at once using the runner.stop function.

Quarantine Mode

The quarantine mode is designed to isolate non-deterministic tests (that is, tests that pass and fail without any apparent reason) from the other tests.

When the quarantine mode is enabled, tests run according to the following logic:

  1. A test runs at the first time. If it passes, TestCafe proceeds to the next test.
  2. If the test fails, it runs again until it passes or fails three times.
  3. The most frequent outcome is recorded as the test result.
  4. If the test result differs between test runs, the test is marked as unstable.

Note that it increases the test task's duration if you enable quarantine mode on your test machine because failed tests are executed three to five times.

See Martin Fowler's Eradicating Non-Determinism in Tests article for more information about non-deterministic tests.

stop

Stops all the pending test tasks.

async stop()

You can also stop an individual pending task by cancelling the corresponding promise.