Assert Yourself: An Introduction to Minitest
— -
layout: post
head-title: “Assert Yourself: A Detailed Minitest Tutorial”
title: “Assert Yourself: An Introduction to Minitest”
canonical: /assert-yourself-an-introduction-to-minitest
date: 2016–08–28 17:40:47 -0700
categories:
author: Pete Hanson
description: This post introduces automated testing with Ruby’s Minitest software, with an emphasis on using assertions. It discusses basic test scaffolding; assertions and refutations; skipping and flunking tests; setup and teardown.
— -
One of the biggest keys to producing quality software is properly testing your program under a wide variety of conditions. Doing this manually is tedious and error-prone, and frequently doesn’t get done at all. This leads to buggy software, and sometimes software that doesn’t work at all.
Fortunately, modern programming environments support automated tests. Such tests can be run repeatedly with minimal effort: it’s even possible to run them automatically prior to every commit or release. While automated testing doesn’t guarantee that your code will be bug free — the tests are only as good you make them — it can certainly reduce the number of bugs your users will encounter. Automated testing can also help you find and fix bugs earlier in the development cycle, and prevent a lot of needless debugging trying to track down a particularly nasty bug.
In this post, we will talk about Minitest, the standard software testing framework provided with Ruby. It isn’t the only software testing framework available, but being supplied automatically with Ruby is a major advantage.
In particular, we will discuss how to use Minitest assertions to test your software. We aren’t interested so much in the theory of automated testing, but in how to use Minitest.
<! — more →
## Definitions
A **testing framework** is software that provides a way to test each of the components of an application. These can be methods or entire programs; the framework should be able to provide appropriate inputs, check return values, examine outputs, and even determine if errors occur when they should.
Testing frameworks provide 3 basic features:
- a way to describe the tests you want to run,
- a way to execute those tests,
- a way to report the results of those tests.
There is a hierarchy to tests. Since there isn’t any formal agreement on exactly which terms are used to describe this hierarchy, we will use the following definitions:
- A **test step**, or simply a **test**, is the most basic level of testing. A test step simply verifies that a certain expectation has been satisfied. For example, in a to-do application, you may want to verify that a newly created to-do is not yet marked as completed. Test steps employ either an assertion or an expectation depending on your testing framework.
- A **test case** is a set of actions that need to be tested combined with any appropriate test steps. For example, a test case for the above test step may include creation of the to-do object, a call to the `#completed?` method on that object, and, finally, an assertion that the return value of the `#completed?` method is false. Typically, only one test step is used per test case; some developers insist on this, others are more flexible. For brevity, some of our test cases may show multple test steps.
- A **test suite** is a collection of one or more test cases that, taken together, demonstrate whether a particular application facet is operating correctly. We use this term quite loosely: a test suite can test an entire class, a subset of a class, or a combination of classes, all the way up to the complete application. The test suite may be entirely in one file, or it may be spread out over several files.
There are other terms you will encounter in the testing world, but for our purposes, these 3 terms will be sufficient.
## What is Minitest?
**Minitest** is a testing framework that comes with every standard Ruby distribution. It’s not quite as powerful and flexible as its cousin RSpec, but for most cases, Minitest will do everything you need. It provides multiple interfaces — assertions and expectations — and a choice of several different output formats.
In addition to being a testing framework, Minitest provides the ability to create and use mock objects and stubs, and the ability to run benchmarks. These topics won’t be discussed in this post, but may appear in a future post.
## Assertions or Expectations?
As mentioned above, Minitest provides assertion-based and expectation-based interfaces.
In the assertions-based interface, the test writer supplies one or more classes that represent test suites. Within each test suite are one or more methods that define test cases. Finally, each test case method has some code that exercises some aspect of the item under test and then runs one or more test steps to verify the results. Here’s a simple example of a test suite for testing a square_root method:
```ruby
require ‘minitest/autorun’
class TestSquareRoot < Minitest::Test
def test_with_a_perfect_square
assert_equal 3, square_root(9)
end
def test_with_zero
assert_equal 0, square_root(0)
end
def test_with_non_perfect_square
assert_in_delta 1.4142, square_root(2)
end
def test_with_negative_number
assert_raises(Math::DomainError) { square_root(-3) }
end
end
```
In the expectations-based interface, the test writer uses a [domain-specific language](https://en.wikipedia.org/wiki/Domain-specific_language), or DSL, to describe the tests:
```ruby
require ‘minitest/autorun’
describe ‘square_root test case’ do
it ‘works with perfect squares’ do
square_root(9).must_equal 3
end
it ‘returns 0 as the square root of 0’ do
square_root(0).must_equal 0
end
it ‘works with non-perfect squares’ do
square_root(2).must_be_close_to 1.4142
end
it ‘raises an exception for negative numbers’ do
proc { square_root(-3) }.must_raise Math::DomainError
end
end
```
As you can see, assertions are little more than basic Ruby code: all you need to know is what modules to require; how to name your classes (test suites) and methods (test cases); and the methods (assertions) you need to call to perform each test. On the other hand, expectations have a DSL that needs to be learned, with commands like `describe` and `it`, and those odd `must_*` methods being applied to the return values of `square_root`. The DSL is more English-like, but you still need to learn the DSL even if you know English.
For the remainder of this post, we will concentrate on using Minitest assertions.
## Writing a Simple Test Suite
Setting up a Minitest test suite isn’t difficult, but it also isn’t obvious when first starting out with Minitest. Before you start writing your first test suite, you need to know how to prepare it.
Typically, test suites are stored in in a special `tests` directory beneath your main application’s development directory. For example, if you are working on a to-do application that is stored in `/Users/me/todo`, then you will place your test suite files in `/Users/me/todo/tests`. This isn’t a requirement, but is good practice for source organization, particularly when working with large projects.
There are no universal conventions regarding how to name your test suites. In the absence of any rules imposed by projects guidelines, we recommend establishing your own naming conventions and sticking to them; for instance, a common convention is to name a test suite for a class named `ToDo` as `t_to_do.rb` or `to_do_test.rb`.
Once you know where your tests suites will live and how they will be named, you need to set up some scaffolding code for the tests. Your scaffolding code will look something like this:
```ruby
require ‘minitest/autorun’
require_relative ‘../lib/xyzzy’
class XyzzyTest < Minitest::Test
def test_the_answer_is_42
xyzzy = Xyzzy.new
assert(xyzzy.the_answer == 42, ‘the_answer did not return 42’)
end
def test_whats_up_returns_doc
xyzzy = Xyzzy.new
assert(xyzzy.whats_up == “Doc”, ‘whats_up did not return “Doc”’)
end
end
```
We start by requiring `minitest/autorun`; this package includes everything you need to run most basic tests, and it ensures that all of the tests in your test suite will be run automatically when you run the test suite.
Next we require the file(s) that contains the code we want to test. You will usually need to use `require_relative`, and will need to figure out the relative name of the module with respect to the test suite file. In this example, the source code is found in `../lib` with respect to the location of the test file.
We then define a class that inherits from `Minitest::Test` (or `MiniTest::Test`; you may use `Minitest` and `MiniTest` interchangeably). This class defines the test suite — a collection of one or more test cases. The name of this class is not important; however, it is common to append or prepend `Test` to the name, and the rest of the name is often the name of the class or module being tested. What’s important to Minitest is that the class inherits from `Minitest::Test`; Minitest runs tests in every class that inherits from `Minitest::Test`.
Finally, we have provided scaffolding for two example test cases: one that tests whether `Xyzzy#the_answer` returns `42`, and one that tests whether `Xyzzy#whats_up` returns `”Doc”`. Both test cases have a bit of code to set up the test (we create an `Xyzzy` object in both cases), and a test step that verifies each of the methods returns the expected answer. (We will return to the `assert` calls in a bit — for now, just understand that each of these tests succeeds if the specified condition is true. If the condition is false, the test fails, and the message string specified as the second argument is printed as part of the failure message.)
Note that each of the test cases is represented by a method whose name begins with `test_`; this is required. Minitest looks for and runs all methods in the test suite whose name begins with `test_`. The rest of the method name is usually a short description of what we are testing; as you’ll see later, well-named test cases will help make it easier to understand failures caught by Minitest.
Every assertion-based Mintest test suite you set up will look something like this. Some test suites may have multiple test suite classes; Minitest will run all of the test suite classes defined in the file. Some test suite classes will have only one or two test cases; others may have many test cases.
NOTE: If the code you are testing includes code that starts your app, see the section on testing startup code near the end of this post.
## Writing Tests
While we won’t be doing any actual development in this post, it’s important to understand how testing fits into the software development cycle. Ideally, your test cases should be run before writing any code. This is frequently called Test-Driven Development (TDD), and follows a simple pattern:
1. Create a test that fails.
2. Write just enough code to implement the change or new feature.
3. Refactor and improve things, then repeat tests.
This is often called Red-Green-Refactor. Red describes the failure step; green describes the getting things working; and, of course, refactor covers refactoring and improving things.
Once you’ve been through these steps, you can move on the next feature and repeat all of the above steps.
This post is not about TDD. However, it is useful to use the TDD approach in developing tests at the lowest levels.
### Example
Lets look at an example. Suppose we want to develop a method to calculate square roots rounded to the nearest integer. If we attempt to take the square root of a negative number, the method should return nil.
<aside style=”border: 1px solid gray; background-color: #f0ffff; padding: 0px 1em;”>
For brevity, we will write our `square_root` method in the same file as our tests — this is not usually good practice.
</aside>
#### First Test Case: square_root(9) should return 3
For our first test, we will take a simple case — most people know that the square root of 9 is 3, so let’s test that.
```ruby
require ‘minitest/autorun’
def square_root(value)
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
end
```
This is our first test — it uses `#assert_equal` to assert that the result of `square_root(9)` is `3`. (We will discuss `#assert_equal` in more detail later.)
We have also supplied a dummy `square_root` method just so the test case has something it can test. Note that we used the name `square_root` because our test case uses that name — the test case is already driving our development by describing the interface we want to use.
If we run this file, we get the following results:
```
Run options: — seed 51859
# Running:
F
Finished in 0.000886s, 1128.2034 runs/s, 1128.2034 assertions/s.
1) Failure:
SquareRootTest#test_that_square_root_of_9_is_3 [x.rb:9]:
Expected: 3
Actual: nil
1 runs, 1 assertions, 1 failures, 0 errors, 0 skips
```
We can see that there is a failure, and it occurred on line 9 where `#assert_equal` is called. Now that we have something that fails, let’s fix it:
```ruby
require ‘minitest/autorun’
def square_root(value)
Math.sqrt(value)
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
end
```
This time, we get:
```
Run options: — seed 42248
# Running:
.
Finished in 0.000804s, 1243.5305 runs/s, 1243.5305 assertions/s.
1 runs, 1 assertions, 0 failures, 0 errors, 0 skips
```
Notice the `.` just below `# Running:`; we should see one `.` for every successful test case. If we see `F`’s, we know that one or more test cases failed. If we see `E`’s, we know that one or more test cases were completely broken. You may also see some `S`’s here: an `S` indicates that a test case was skipped.
#### Second Test Case: square_root(17) should return 4
Our goal is to round the square root to the nearest integer, so the square_root of `17` should be rounded to `4`. Lets write that test case and test it:
```ruby
require ‘minitest/autorun’
def square_root(value)
Math.sqrt(value)
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
def test_that_square_root_of_17_is_4
result = square_root(17)
assert_equal 4, result
end
end
```
```
1) Failure:
SquareRootTest#test_that_square_root_of_17_is_4 [x.rb:15]:
Expected: 4
Actual: 4.123105625617661
```
We get our expected failure — we seem to be on the right track. The problem here is that we aren’t rounding the result, but are returning a Float value that closely approximates the square root of `17`. Let’s fix that:
```ruby
require ‘minitest/autorun’
def square_root(value)
Math.sqrt(value).to_i
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
def test_that_square_root_of_17_is_4
result = square_root(17)
assert_equal 4, result
end
end
```
This time, both tests pass.
#### Third Test Case: square_root(24) should return 5
The square root of `24` is approximately `4.9` which, rounded to the nearest integer, is `5`. Lets write another test case:
```ruby
require ‘minitest/autorun’
def square_root(value)
Math.sqrt(value).to_i
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
def test_that_square_root_of_17_is_4
result = square_root(17)
assert_equal 4, result
end
def test_that_square_root_of_24_is_5
result = square_root(24)
assert_equal 5, result
end
end
```
This fails with:
```
1) Failure:
SquareRootTest#test_that_square_root_of_24_is_5 [x.rb:20]:
Expected: 5
Actual: 4
```
We get our expected failure. The problem here is that we aren’t rounding the result, but are simply truncating it with `#to_i`. To round it, we need to use the `#round` method:
```ruby
require ‘minitest/autorun’
def square_root(value)
Math.sqrt(value).round
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
def test_that_square_root_of_17_is_4
result = square_root(17)
assert_equal 4, result
end
def test_that_square_root_of_24_is_5
result = square_root(24)
assert_equal 5, result
end
end
```
Success once again.
#### Fouth Test Case: square_root(-1) should return nil
In the real domain, square roots aren’t defined for negative numbers, and our requirements require that `#square_root` should return `nil` if it is passed a negative number. Once again, let’s write a test case, this time using `#assert_nil`:
```ruby
require ‘minitest/autorun’
def square_root(value)
Math.sqrt(value).to_i
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
def test_that_square_root_of_17_is_4
result = square_root(17)
assert_equal 4, result
end
def test_that_square_root_of_24_is_5
result = square_root(24)
assert_equal 5, result
end
def test_that_square_root_of_negative_number_is_nil
result = square_root(-1)
assert_nil result
end
end
```
This fails with an error:
```
SquareRootTest#test_that_square_root_of_negative_number_is_nil:
Math::DomainError: Numerical argument is out of domain — “sqrt”
x.rb:4:in `sqrt’
x.rb:4:in `square_root’
x.rb:24:in `test_that_square_root_of_negative_number_is_nil’
```
Note in particular that this problem is occurring in the call to `square_root`, not in the assertion. This means we have to do something in `square_root` to deal with negative inputs:
```ruby
require ‘minitest/autorun’
def square_root(value)
return -5 if value < 0
Math.sqrt(value).round
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
def test_that_square_root_of_17_is_4
result = square_root(17)
assert_equal 4, result
end
def test_that_square_root_of_24_is_5
result = square_root(24)
assert_equal 5, result
end
def test_that_square_root_of_negative_number_is_nil
result = square_root(-1)
assert_nil result
end
end
```
Note that we are now returning `-5` if the value is negative; this isn’t the correct value for `square_root`, but we need to have a failing assertion before we can do a proper test. Returning a value we know to be invalid is one way to handle this.
With this change, we now get the failure we want:
```
1) Failure:
SquareRootTest#test_that_square_root_of_negative_number_is_nil [x.rb:26]:
Expected -5 to be nil.
```
without introducing any new failures or errors. We can now modify `#square_root` to return the correct value:
```ruby
require ‘minitest/autorun’
def square_root(value)
return nil if value < 0
Math.sqrt(value).round
end
class SquareRootTest < Minitest::Test
def test_that_square_root_of_9_is_3
result = square_root(9)
assert_equal 3, result
end
def test_that_square_root_of_17_is_4
result = square_root(17)
assert_equal 4, result
end
def test_that_square_root_of_24_is_5
result = square_root(24)
assert_equal 5, result
end
def test_that_square_root_of_negative_number_is_nil
result = square_root(-1)
assert_nil result
end
end
```
Success once again.
At this point, we can now refactor things to improve the code if we need to do so. After each refactor, we should rerun the tests to ensure that our most recent changes have not broken anything.
## Test Sequence
One thing to keep in mind with files that contain multiple test cases (or multiple test suites) is that Minitest varies the order in which it runs the tests. In the example in the previous section for example, there are 4 tests, but the order in which they are run is completely at random. On one run, the 3rd, 1st, 4th, and 2nd tests may be run in that sequence, but on another run, the 2nd, 4th, 3rd, and 1st tests may be run.
This random sequence is intentional, and should be left as-is. Your tests should not be order-dependent; they should be written so that any test can be performed in any order.
If you have a sporadic issue that only arises when tests are called in a specific order, you can use the ` — seed` option to run some tests in a known order. For instance, suppose you run some tests and get the following output:
```
Run options: — seed 51859
…
1) Failure:
XxxxxTest#test_… [x.rb:9]:
Expected: 3
Actual: nil
3 runs, 3 assertions, 1 failures, 0 errors, 0 skips
```
but a subsequent run produces:
```
Run options: — seed 23783
…
3 runs, 3 assertions, 0 failures, 0 errors, 0 skips
```
you can rerun the first set of tests by using ` — seed 51859` on the command:
```
$ ruby test/tests.rb — seed 51859
```
If you absolutely, positively need to always execute things in order, you must name all your test methods in alphabetical order, and include the command:
```ruby
i_suck_and_my_tests_are_order_dependent!
```
As you might surmise, the developers of Minitest feel very strongly about this. Don’t do it!
## Simple Assertions
As we saw above, tests are written using a wide variety of different **assertions**. Each assertion is a call to a method whose name begins with `assert`. Assertions test whether a given condition is true; refutations test whether a given condition is false. For now, we’ll limit the discussion to assertions — refutations, the negation of assertions, will be described later.
### assert
The simplest possible assertion is the `#assert` method. It merely tests whether the first argument is “truthy”:
```ruby
assert(reverse(‘abc’) == ‘cba’)
```
If the condition is truthy, the test passes. However, if the condition is `false` or `nil`, the test fails, and Minitest issues a failure message that says:
```
Expected false to be truthy.
```
which isn’t particularly helpful. For this reason, `assert` is usually called with an optional failure message:
```ruby
assert(reverse(‘abc’) == ‘cba’, “reverse(‘abc’) did not return ‘cba’”)
```
which is slighty more informative. (As we’ll see in the next session, we can do even better than this.)
All refutations and almost all assertions allow an optional message as the final argument passed to the method. However, in most cases, the default message for methods other than `#assert` and `#refute` is good enough to not require a specific message.
<aside style=”border: 1px solid gray; background-color: #fff0f0; padding: 0.5em 1em;”>
The following assertions do not accept a message argument:
<ul>
<li><code>assert_mock</code></li>
<li><code>assert_raises</code></li>
<li><code>assert_silent</code></li>
</ul>
</aside>
Typically, `#assert` is used with methods that return true or false:
```ruby
assert(list.empty?, ‘The list is not empty as expected.’)
```
More often, `#assert` simply isn’t used; instead, we use `#assert_equal` with an explicit `true` or `false` value.
### assert_equal
The most frequently used assertion is `#assert_equal` — it tests whether an actual value produced during a test is equal to an expected value. For instance:
```ruby
assert_equal(‘cba’, reverse(‘abc’))
```
In this call, the first argument represents an expected value, while the second argument is the actual value — in this case, the actual value is the return value of `reverse(‘abc’)`.
`#assert_equal` uses the `==` operator to perform its comparisons, so you can compare complex objects if an `#==` method has been defined. If an explicit `#==` method is not defined, the default `BasicObject#==` is used, which only returns `true` if the two objects are the same object.
If the expected and actual values are equal, the test passes. However, if they are not equal, the test fails, and Minitest issues a failure message that says:
```
Expected: “cba”
Actual: “some other value”
```
Since the `==` operator is used, you must be careful when asserting that a value is explicitly `true` or `false`:
```ruby
assert_equal(true, xyzzy.method1)
assert_equal(false, xyzzy.method2)
```
The first of these assertions passes only if the return value of `xyzzy.method1` is exactly equal to `true`; it doesn’t check for truthiness like `#assert`. Similarly, the second assertion passes only if `xyzzy.method2` returns `false` exactly; it fails if `xyzzy.method2` returns `nil`.
### assert_in_delta
If you’ve spent any time working with floating point numbers, you’ve no doubt encountered the strange fact that floating point numbers are not usually stored as exact representations; small precision inaccuracies occur in the least significant digits of some floating point numbers. Thus, something like:
```ruby
value_a = 2
value_b = 1.9999999999999999
puts value_a == value_b
```
prints `true` even though the two values look like they should differ. This inaccuracy means that it is inadvisable to make comparisions for exact values in your program. The same holds true for your test cases: you should not assert that a floating point number has a specific value.
Instead, you should assert that a floating point number is **near** some value: how near is up to you. This is accomplished with the `#assert_in_delta` method:
```ruby
assert_in_delta 3.1415, Math::PI, 0.0001
assert_in_delta 3.1415, Math::PI, 0.00001
```
In these two tests, the 3rd argument is the “delta” value: if the expected and actual answers differ by more than the delta value, the assertion fails. If the 3rd argument is not supplied to `#assert_in_delta`, a delta value of `0.001` is assumed.
In this example, the first test succeeds because `3.1415` differs from the actual value returned by `Math::PI` (3.141592653589793) by less than `0.0001` (the actual difference is approximately `0.0000027`), while the second test fails because `3.1415` differs from `Math::PI` by more than `0.00001`.
### assert_same
The `#assert_same` method checks whether two object arguments represent the exact same object. This is most useful when you need to verify that a method returns the same object it was passed. For instance:
```ruby
assert_same(ary, ary.sort!)
```
As with `#assert_equal`, the first argument is the expected value; the second argument is the actual value.
Note that equality of objects is not the same thing as being the same object; two objects can be different objects, but can have equivalent state. Ruby uses the `#==` method to determine equality, but `BasicObject#equal?` to determine that two objects are the same. `#assert_same`, thus, uses the `#equal?` method; since `#equal?` should never be overridden, you can usually be assured that `#assert_same` uses `BasicObject#equal?`.
Note, too, that `BasicObject#==` does the same thing as `BasicObject#equal?`, but most objects override `#==`, while no class should override `#equal?`. In most cases, you can assume that `#assert_equal` tests for equivalent states, while `#assert_same` tests for the same object.
### assert_nil
`#assert_nil` checks whether an object is `nil`. This is most useful in conjunction with methods that return `nil` as a “no results” result.
```ruby
assert_nil(find_todos_list(‘Groceries’))
```
### assert_empty
`#assert_empty` checks whether an object returns `true` when `#empty?` is called on the object. If the object does not respond to `#empty?` or it returns a value other than `true`, the test fails. `#assert_empty` is most useful with collections and Strings.
```ruby
list = []
assert_empty(list)
```
Note in particular that the assertion is named `#assert_empty` without a `?`, but it checks the `#empty?` method.
### assert_includes
`#assert_includes` checks whether a collection includes a specific object. If the collection does not respond to `#include?` or if the method returns a value other other than `true`, the test fails.
```ruby
list = %w(abc def xyz)
assert_includes(list, ‘xyz’)
```
Note that the assertion is named `#assert_includes` with an `s` but no `?`, but it checks the `#include?` method which has a `?` but no `s`. Note also that this method departs from the “expected, actual” or “value” format used by some other assertions.
### assert_match
`#assert_match` is used when working with String objects: often, you don’t need to test an exact value, but just need to determine whether a String matches a given pattern. `#assert_match` uses regular expressions (regex) for those patterns.
Most often, `assert_match` is used with text whose content only needs to contain a few specific words, such as an error message:
```ruby
assert_match(/not found/, error_message)
```
## Setup and Teardown
Many test suites have some code that needs to be run either before or after each test. For example, a test suite that validates a database query may need to establish the database connection prior to each test, and then shut down the connection after each test. This can be easily accomplished with helper methods inside your test suite classes; however, you would need to remember to call both methods in every test case. Minitest provides a better way: each class that defines a test suite can have a `#setup` and/or a `#teardown` method to automatically perform any preprocessing or postprocessing required.
The `#setup` method is called prior to each test case in the class, and is used to perform any setup required for each test. Likewise, `#teardown` is called after each test case to perform any cleanup required. It is common to set instance variables in `#setup` that can be used in the actual test case methods. For example:
```ruby
require ‘minitest/autorun’
require ‘pg’
class MyApp
def initialize
@db = PG.connect ‘mydb’
end
def cleanup
@db.finish
end
def count; …; end
def create(value); …; end
end
class DatabaseTest < Minitest::Test
def setup
@myapp = MyApp.new
end
def test_that_query_on_empty_database_returns_nothing
assert_equal 0, @myapp.count
end
def test_that_query_on_non_empty_database_returns_right_count
@myapp.create(‘Abc’)
@myapp.create(‘Def’)
@myapp.create(‘Ghi’)
assert_equal 3, @myapp.count
end
def teardown
@myapp.cleanup
end
end
```
This test suite runs two test cases. Prior to each test case, `#setup` creates a `@myapp` instance variable that references a `MyApp` object. After each test case, `#teardown` calls `@myapp.cleanup` to perform any shutdown cleanup required by the `MyApp` class. In this example, set up and tear down consist of code that establishes and then drops a database connection. Elsewhere in the test suite, we can reference `@myapp` freely to access the object.
Note that both `#setup` and `#teardown` are independent and optional; you can have both, neither, or either one in any test suite.
## Testing Error Handling (assert_raises)
Testing involves not only checking that methods produce the correct results when given correct inputs, but should also test that those methods handle error conditions correctly. This is easy when the success or failure of a method can be determined by just testing its return value. However, some methods raise exceptions. To test exceptions, you need the `#assert_raises` method. We saw an example of this earlier:
```ruby
def test_with_negative_number
assert_raises(Math::DomainError) { square_root(-3) }
end
```
Here, `#assert_raises` asserts that the associated block (`{ square_root(-3) }`) should raise a `Math::DomainError` exception, or an exception that is a subclass of `Math::DomainError`. If no exception is raised or a different exception is raised, `#assert_raises` fails and issues an appropriate failure message.
## Testing Output
Many tests require that you examine the terminal output of your application. In this section, we’ll examine Minitest’s techniques for testing output.
### assert_silent
With many applications, methods, etc, the ideal situation is that no output at all is produced. To test for this situation, simply invoke the method under test in a block that gets passed to `#assert_silent`:
```ruby
def test_has_no_output
assert_silent { update_database }
end
```
If `#update_database` prints anything to `stdout` or `stderr`, the assertion will fail. If nothing is printed to `stdout` nor `stderr`, the assertion passes.
### assert_output
Sometimes you need to test what gets printed, and where it gets printed. For example, you may want to ensure that good output is sent to `stdout`, while error messages are printed to `stderr`. For this, we use `#assert_output`:
```ruby
def test_stdout_and_stderr
assert_output(‘’, /No records found/) do
print_all_records
end
end
```
The first argument for `#assert_output` specifies the expected output that will be sent to `stdout`, while the second argument specifies the expected output for `stderr`. Each of these arguments can take the following values:
- `nil` means the assertion doesn’t care what gets written to the stream.
- A string means the assertion expects that string to match the output stream exactly. (In particular, an empty string is used to assert that no output is sent to a stream.)
- A regular expression means the assertion should expect a string that matches the regex.
In our example above, the assertion expects that `#print_all_records` won’t print anything to `stdout`, but will output an error message to `stderr` that contains the regular expression pattern `No records found`.
### capture_io
An alternative to using `#assert_output` is to use `#capture_io`:
```ruby
def test_stdout_and_stderr
out, err = capture_io do
print_all_records
end
assert_equal(‘’, out)
assert_match(/No records found/, err)
end
```
This is equivalent to the example shown in the previous section. The chief advantage of using `capture_io` is it lets you handle the `stderr` and `stdout` separately. This is especially handy when you want to run multiple tests on one or both of `out` and `err` that can’t readily be handled with `#assert_output`.
## Testing Classes and Objects
Ruby is pretty lax about types: methods can both accept and return values of different types at different times. Sometimes, it is handy to verify that a value has the expected type, or that it can respond to a particular method. For these cases, Minitest provides several useful methods.
### assert_instance_of
`#assert_instance_of` asserts that an object is an object of a particular class; it is analogous to the standard Ruby method `Object#instance_of?`.
```ruby
assert_instance_of SomeClass, object
```
succeeds if `object` is a `SomeClass` object, fails otherwise.
### assert_kind_of
`#assert_kind_of` asserts that an object is an object of a particular class or one of its subclasses; it is analogous to the standard Ruby method `Object#kind_of?` or `Object#is_a?`.
```ruby
assert_kind_of SomeClass, object
```
succeeds if `object` is a `SomeClass` object or one of its subclasses, fails otherwise.
### assert_respond_to
In Ruby, you often don’t need to know that an object has a particular type; instead, you’re more interested in what methods an object responds to. Ruby even provides a method, `Object#respond_to?`, that can be applied to any object to determine if it responds to a given method. This carries over to testing as well: you’re often more interested in knowing if an an object responds to a given method, and `#assert_respond_to` provides this capability:
```ruby
assert_respond_to object, :empty?
```
This test asserts that `object` responds to the method `#empty?`, e.g., `object.empty?` is a valid method call. The method name may be specified as a symbol, as shown above, or as a String that will be converted to a symbol internally.
## Refutations
Most often, your test cases will be more interested in determining whether or not a specific condition is true. For example, you may want to know whether a method returns `true`, `5`, or `’xyz’`. For such situations, assertions are ideal; you write your assertion in such a way that it asserts whether the actual value has the expected value.
Much less often, you will encounter situations in which you are interested in the negation of a condition; for example, if you want to ensure that a method that operates on an `Array` returns a new `Array` instead of modifying the original `Array`, you need to assert that that result `Array` is **not** the same object as the original `Array`. You can write this as:
```ruby
ary = […]
assert ary.object_id != method(ary).object_id, ‘method(ary) returns original Array’
```
However, this isn’t particularly clear because we are forced to use the bare `assert` instead of one of the more specialized assertions like `assert_same`, which would be easier to read.
For these cases, Minitest provides **refutations**. Refutations are assertions that assert that something isn’t true. For example, we can write:
```ruby
ary = […]
refute(ary.object_id == method(ary).object_id,
‘method(ary) returns copy of original Array’)
```
This simplifies further to:
```ruby
ary = […]
refute_equal ary.object_id, method(ary).object_id
```
Better yet, we can use the negation of `assert_same`, namely `refute_same`, to be even more clear:
```ruby
ary = […]
refute_same ary, method(ary)
```
Most of the Minitest assertions have equivalent refutations that test the negation of the condition. In all cases, the refutation uses the assertion name with `assert` replaced by `refute`, and arguments for refutations are identical to the arguments for assertions. So, for example, the refutation of:
```ruby
assert_nil item
```
is:
```ruby
refute_nil item
```
<aside style=”border: 1px solid gray; background-color: #fff0f0; padding: 0.5em 1em;”>
The following assertions do not have a corresponding refutation:
<ul>
<li><code>assert_output</code></li>
<li><code>assert_raises</code></li>
<li><code>assert_send</code></li>
<li><code>assert_silent</code></li>
<li><code>assert_throws</code></li>
</ul>
</aside>
## Uncovered Methods
This post isn’t meant to be a complete reference to Minitest; for that, you should refer to the [Minitest documentation](http://www.rubydoc.info/github/seattlerb/minitest/Minitest/). However, here’s a short list of some other methods from the Minitest::Assertions module that you may find useful:
- `#assert_in_epsilon` is similar to `#assert_in_delta`, except the delta is based on the relative size of the actual or expected value.
- `#assert_operator` is used to test binary operation such as `#<=`, `#+`, etc.
- `#assert_predicate` is used to test predicates — usually used by expectations, not assertions.
- `#assert_send` calls an arbitrary method on an object and asserts that the return value is true.
- `#assert_throws` tests whether a block returns via a `throw` to a specific symbol.
- `#capture_subprocess_io` is similar to `#capture_io` except that is also captures output of subprocesses. If `#capture_io` doesn’t do what you want, try `#capture_subprocess_io` instead.
- `#flunk` forces a test to fail
- `#skip` forces a test to be skipped
- `#skipped?` returns true if a test was skipped. Sometimes useful in `#teardown`.
## Testing Startup Code
If the module you are testing has some launch code that starts the program running, you may have to make a small modification to that code. For example, the launch code in an `Xyzzy` module may look something like this:
```ruby
Xyzzy.new.run
```
If you run this code during testing, the results may be confusing as your application will start in addition to being tested. To avoid this, you need to modify the launch code so it doesn’t run when the file is required. You can do this like so:
```ruby
Xyzzy.new.run if __FILE__ == $PROGRAM_NAME
```
or
```ruby
Xyzzy.new.run if __FILE__ == $0
```
If you run the program directly, both `__FILE__` and `$PROGRAM_NAME` (or `$0`) reference the program file. If, instead, you require the file into your test module, `$PROGRAM_NAME` and `$0` will be the name of the test program, but `__FILE__` will continue to refer to the main program file; since the two names differ, the launch code will not run.
Some programs don’t have any obvious launch code like `Xyzzy.new.run`; for example, Sinatra applications don’t have any obvious launch code. In such cases, you may need to find a different way to prevent running the program when testing. With Sinatra (and Rack::Test), you must run the following before performing any route requests:
```ruby
ENV[‘RACK_ENV’] = ‘test’
```
This code is added to your test module.
## Conclusion
This concludes our introduction to Minitest, and, in particular, its assertions interface. As we’ve seen, using Minitest is simple and straightforward, with a wide variety of assertions and refutations that can help you write the tests required to determine whether your application works.
This isn’t the complete story about Minitest. In particular, we only briefly mentioned topics such as expectations, mocking and stubs, benchmarking, custom assertions and expectations, and the different reporting and execution options. While we hope to cover some of these topics in future posts, for now you will need to refer to the [Minitest documentation](http://www.rubydoc.info/github/seattlerb/minitest/Minitest/).
## Wrap-up
We hope you’ve enjoyed this introduction to Minitest, and to its assertions interface in particular. This and many other topics are discussed extensively in the curriculum at [Launch School](https://launchschool.com/). Logged in users (free registration) can also access many exercises on a wide variety of topics. Feel free to stop by and see what we’re all about.
