TestRail is a test management tool which can be integrated with top applications like Jira, Cucumber, Selenium, and Bugzilla etc. and is used by big & small companies all over the world. Here, I am going to share a detailed review of this tool with all the features an overview of its functionality. TestRail is available as both a Cloud service as well as an on-premises server version. The purpose of this review I tested the Cloud version.

TestRail is a web-based software which means you need not to install anything on your machine in order to efficiently manage, track and organize your test cases, test plans, and test runs. It is easy to use and you can see real-time progress of your team efforts. It has a beautiful dashboard with color notations to specify the number of passed, failed, skipped and blocked test cases.

TestRail Core Features:

The following is the list of TestRail core features which attracts a wide range of Quality Analysts, and other stakeholders for Test Management tasks.

Test Methodologies Support: TestRail tool is capable of supporting different types of testing and associated methodologies. Given below is the brief description of the types of testing supported by TestRail test management tool.

Agile Testing: Agile testing is based on agile model (AM) which is a methodology to document the best practices of software system testing and software delivery is in iterations.

Black box Testing: It is behavioral testing which is focussed on the functionality of the application without the knowledge of the actual underlying code. This can be executed manually or can be automated if required.

Exploratory Testing: It is an ad hoc testing which believe in hitting the area of the software which is prone to defects and helps in disclosing the defects early with minimal testing effort. It is mostly experience based testing. Test scenarios are documented but with the minimal details.

Functional/Manual Testing: The tester creates a test case based on the requirements and then manually executes the test case on the application under test and records the results.

Automation Testing: It is the practice to automate the testing without the involvement of the tester during the test execution. Various tools can be used to automate test cases. TestRail has a variety of add-ins for supporting different automation tools.

Centralize, organize and manage Test Cases: TestRail is capable of centralizing, organizing and managing test cases. It is very efficient and effective for test case organization. It has the provision to import test cases from excel as well as another management tool easily. It allows creating multiple test cases versions which can be used for comparison and references. Also, the test case repository can be shared across different projects, sprints, and releases.

Execution Tracking and Management: TestRail supports building as well as re-running of the test execution suites. It can trigger automation suite and helps to capture the test execution history step by step with a thorough explanation. Also, we can configure our customized test run specific configuration and parameters in TestRail. Test data and screenshots can be attached to the test case for faster execution.

Email Notification Support: TestRail supports email notification. Users can subscribe to the email for a particular task by using voting and watching features in order to keep an eye on the task progress. We just need to use @mention in order to get the attention of a specific team member or stakeholder in Comments and Description fields. For example, if a developer needs more information about a test case then he can mention the name of the tester by using his @name and tester will be notified.

Power Search Support: TestRail has the Hotkeys feature which supports the powerful search functionality from the testing data. It also supports free text search as well as syntax search.

Reporting Support: TestRail supports various kinds of reports which include Test Case and Test Execution Reports. Also, it can help to generate a detailed summary report related to projects, milestones, plan, cross-project report, and Runs. It is capable to track the workload of the entire testing team.

Scale with Team Growth: TestRail can support any size business team or project. Whether your team has a few members to 1000s, TestRail shows its best performance every time.

Integration: I mentioned this in the beginning, TestRail supports a large number of add-ins which helps to connect with different software across the globe such as Bamboo, Cucumber, Egg Plant, JIRA, Jenkins, HipChat, Selenium etc. It also provides support for Rally and Version One integration. So when you use TestRail, you need not change your current framework, just get the add-ins and integrate your existing tools with TestRail. Some of them are shown in the figure below:

Common Helpful Features: TestRail has sticky sidebars which support rich text formatting (RTF), optimize printing, forecasting, and scheduling.

Data Recovery: TestRail helps to generate and download full system backups. It can also help to migrate and import existing test cases from excel, legacy data tool, and other management tools. It can also export data into various supported methods such as CSV, XML and Excel files.

Centralized tool: TestRail can be used by multiple teams at the same time, the repository can be shared among different teams when required and test cases can be imported from excel and other test management tool easily. It keeps all the file version for references. If anything breaks in the latest file version then you can revert it to the more stable version of the history.

Test executions: TestRail supports “Build” function and re-run of test suites if needed. You can kick off automation test suites and see a detailed explanation of the failed test cases which is useful when you want to debug your scripts.

Attractive user interface: As mentioned earlier, this tool has color notations to display the number of a test case executed and how many of them needed to be retested. It looks beautiful and is highly productive since it displays data hub, upcoming milestones, activity, actions, to-dos and latest test runs.

Multiple projects organization: TestRail displays all the required information about all the on-going test projects such as upcoming milestone details, current test runs, and previous test results. It also saves completed projects and archives them. If more than one team is working on the same project, this tool supports all of them in parallel and teams can track their progress on TestRail user-friendly interface.

TestRail Subscription Plan:

TestRail provides 30-days free trial for new users and then you can choose from their different plans according to your project needs. You can find their pricing details here: http://www.gurock.com/testrail/pricing/

TestRail Free Trial Setup:

Free trial setup for TestRail can be found at http//www.gurock.com/testrail/. You just need to click on “Try TestRail for Free” button as shown below. Fill out the details and activate your free trial for a month.

Latest Features: TestRail FastTrack

In TestRail 5.4. they introduced a new feature called TestRail FastTrack. It is a dashboard with 3 pane view layout which produces faster results and provides test results with just 1 click. You can add results to the test cases and jump to next test case with just one click. You can also add more details in a test case run, for example, for a failed test case, error details can be added with a screenshot.

Test Case Management:

A test case can be created, edited and deleted under “Test Suites & Cases” tab as shown in the figure above. Here, test cases are very detailed. Fields include Title, Pre-conditions, Steps, Expected Results, Section, Priority, and Type. You can add estimates and milestone details if needed. Fields can be customized by selecting “Add Field” option. Test data or reference documents can be added using add attachment link. I really liked the structured way of managing the test cases and test suites.

Test suites can be sectioned and test cases can be added to them accordingly. The section is a field under “Test Suites & Cases” tab where you can specify the test suite it belongs to. TestRail also saves all the test case history to ensure transparency among the teams and track all the changes made to that test case. Similarly, all the versions are kept as baselines for multiple branches. No data is lost and teams know where to find all the details.

The most current release of TestRail is 5.5 (May 2018) and for this dot release, they continued the theme of making improvements to the administration area of TestRail, in particular, so that large teams can benefit from easier user administration and increased security. Additional UI enhancements and some quite major scalability changes in the TestRail database which will improve performance for teams with large numbers of test cases and test runs. The complete list of enhancements are:

Ranorex Studio Integration
Add User shortcut keys
Last Active User Session Column and Timestamps
Force Password Reset feature
User Session Timeout Feature
Send Report as PDF
Add and Next button in Test Case Editor
Elapsed Time now supports milliseconds
New add case field API endpoint
Database scalability improvements
Cloud infrastructure hardening, resilience and performance improvements

Pros and Cons of TestRail over other Tools:

Here, we are going compare TestRail with other web-based management tools. They are SpiraTest and TestLodge.

Licensing: The commercial version of all these test management tools offer 30 days’ free trial version to get familiarized with these tools. The licensing agreement of SpiraTest and TestRail is based on the number of users. On the other hand, the licensing rules for TestLodge tool are based on the number of activities which include test cases, test plans, and test runs but not on the number of users.
Access control: TestRail access control is very fine-grained where we can edit existing or pre-set roles, add new roles, and prepare user groups. Each role is associated with a set of permissions for milestones, test cases, test results, reports, configurations, and attachments. SpiraTest access control is very similar to that of TestRail. In TestLodge, you can edit pre-set roles and create custom roles but it also allows you to assign admin role to a user which can define the set of permissions to other users for test cases, reports, test runs, test plans, and test suites.
Working interface: Both TestRail and TestLodge test management tools have simple UI and they are very well organized in terms of ease of use and capturing the idea of the project under test. Such working interface can be compared with SpiraTest where intuitive UI is offered by the SpiraTest tool that makes it capable to filter down the number of shells on the workspace. Such feature also makes SpiraTest a tool difficult to work on due to the multiplicity of the “edit” and “update” buttons which confuses the user and make the user make the redundant “clicks” and triggers edit and update operation which may cause problems.
Requirement management: TestRail does not support built-in Requirement Management (RM) service which is present in both TestLodge and SpiraTest test management tools. But TestRail can be integrated with equivalent external services such as Jira at no extra cost to contribute to requirement management where we can perform CRUD operations, attach document files, and assign test cases to the requirements. Therefore, it can also help in the preparation of traceability matrix.
Test design: TestRail and TestLodge test management tools are capable to skip the typical process to perform CRUD operations on the test-cases and allows you to add a test case requiring bare minimum details such as title, steps, expected result and description.
Test execution: The test execution result in both TestRail and SpiraTest tools are displayed as a tree whereas TestLodge presents test results in the row mode as one by one on the main page. The tree structure is very simple to understand as it displays the values and the results more clearly than row mode result display. Also, the TestRail tool allows tracing the test launches history which is not very clear in SpiraTest tool.

TestRail provides a detailed user guide for reference. It is pretty easy to understand. They have explained it really well by using detailed screenshots. You can find the guide, training courses, and other valuable documents here:

http://www.gurock.com/testrail/support/

They have got strong community support, so if you have any doubts you can post your question and get a solution from the fellow users. Overall, this is one of the best test management tool available in the market which has so many useful features and user-friendly interface. I recommend you try out this tool and then analyze it according to your project requirements.

Behavior Driven Development is hugely helpful, given the right tools and the right process. We explore the value in BDD for your testing strategies, your team, and your organization.