In our blog, we’ve previously had a Tag1 Team Talk about Goose, by Tag1 CEO Jeremy Andrews. Goose is a Locust-inspired load testing tool In Rust. Goose has been effective in helping Tag1 support its clients by ensuring their websites hold up under stress.

Since that initial post, Jeremy has made a number of improvements to Goose, outlined in the changelog. Here, we’ll summarize the changes and break them down for you, through version 0.10.

Gaggles

Version 0.6.0 added Gaggles. A Gaggle is a distributed load test, made up of one Manager process and one or more Worker processes. With Goose, a single process can use all available CPU cores. This makes Goose very powerful and fast and more likely to bottleneck on available network speed, an important aspect of a full load test. You'll likely need multiple computers with Workers to scale beyond a 1G network interface, and sometimes it's useful to generate load from different places/clouds/etc. In all cases, the Workers can send statistics to the Manager, which aggregates it all together and summarizes at the end.

Gaggles are an optional, compile-time feature. For a more detailed discussion of Gaggles see Show me how this flock flies: A working Gaggle in the Goose Load Testing Framework.

Async support

Async enables the simulation of considerably more "users" by giving up the CPU when a task is blocked. For example, when a Goose "user" requests a web page, it's blocked until the server responds. Without Async, Goose blocks an entire CPU core while waiting for that response. With Async, Goose makes the request, then gives up the core (ie, sleeps) allowing another "user" to make a request with that same core, and again gives up the core (sleeps). When a response comes back, the appropriate "user" gets woken to complete the next task. This creates a 2x boost in performance.

Goose as a load test library

Tag1 Vice President of Software Engineering Fabian Franz, along with the help of the community, made it possible to pass a closure to GooseTask::new. This change makes it possible to use Goose as a load test library within another application. Previously, load test task functions were standalone; now, another program can dynamically create task functions and pass them to the Goose load test process as needed.

To learn about Rust closures, see Rust’s Advanced Functions and Closures documentation. The Goose source code includes an example of how the closure is used in the software.

Another way Goose was reworked as a library is that it also now passes errors up the stack instead of calling exit(1). Goose generates valid errors which can be properly handled in applications, when previously it exited on errors. Rust uses a particular pattern for results: Result<Ok(), Err()>. Jeremy explains the pattern in the README, in the section If you’re new to Rust.

If you're new to Rust, main()'s return type of Result<(), GooseError> may look strange. It essentially says that main will return nothing (()) on success, and will return a GooseError on failure. This is helpful as several of GooseAttack's methods can fail, returning an error. In our example, initialize() and execute() each may fail. The ? that follows the method's name tells our program to return an error on failure, otherwise continue on.

Request throttling

The request throttling feature enables you to generate a standardized amount of load from different servers, even when those servers have different resources available, ensuring consistency of data. It is an API change that uses the --throttle-requests option when running Goose to optionally limit the maximum requests per second.

For a full explanation of how throttling works, see this section of the Goose source code.

Per-task metrics

Goose includes per-task metrics in addition to per-request metrics. Previously, there was no way to know for certain whether all Goose Tasksets and Goose Tasks ran. This is a problem we've experienced with Locust, in which you can accidentally run an invalid load test by, for example, setting weights too high and running too few users.

When displaying per-task metrics, all Tasksets and Tasks are listed, even if they did not actually run. Disable this feature with --no-task-metrics.

Improved metrics logging

Goose’s logging of metrics was significantly improved. Names, formats, and statistics are clearer and easier to understand.

  • Renamed stats and statistics to metrics for consistency and clarity. These are the finalized names:
    • --no-stats became --no-metrics
    • --no-reset-stats became --no-reset-metrics
    • --no-task-stats became --no-task-metrics
    • --stats-log-file became --metrics-file
    • --stats-log-format became --metrics-format
  • Added --metrics-format= to switch between json (default), csv, and raw formats
  • Added --metrics-file= to optionally log all requests to file specified, enabling you to keep track of your metrics over time. This file can be downloaded and analyzed as needed
  • Replaced --print-stats with --no-stats (which became --no-metrics.) Goose defaults to collecting and displaying metrics, when originally we defaulted to not collecting and displaying metrics. Disable this with the --no-metrics option
  • Default to resetting metrics, disable with --no-reset-stats, display spawning metrics before resetting

For more information on test metrics, and to see an example, see Logging Load Test Metrics.

Debug logging

Goose now includes the --debug-file option to write errors to the specified log file, and --debug-format= option to switch between json (default) and raw formats. This feature solves a common problem when writing load tests, and helps in quickly understanding why things aren't working as you expect, and/or detecting when the load test is generating so much load your infrastructure is failing, and so on.

For more information, see Load Test Debug Logging in the Goose documentation.

Custom defaults

GooseAttacks have been updated to give you more flexibility in configuring your testing runs. This means you can set all run-time options with custom defaults directly from your test and don't need to pass in command line options. Command line options however still can override the defaults. This change replaces GooseAttack::set_host() with more generic GooseAttack::set_default(), exposing new default settings. For a full list and a configuration example, see Defaults. The following list is a subset that may be most useful to customize:

  • Host: GooseDefault::Host (&str)
  • Users to start:GooseDefault::Users (usize)
  • Users to start per second: GooseDefault::HatchRate (usize)
  • Number of seconds for test to run: GooseDefault::RunTime (usize)
  • Only print final summary metrics: GooseDefault::OnlySummary (bool)
  • Metrics log file name: GooseDefault::MetricsFile (&str)
  • Debug log file name: GooseDefault::DebugFile (&str)
  • Maximum requests per second: GooseDefault::ThrottleRequests (usize)

Other improvements

Goose has a number of other updates as well. Internal test coverage was increased.

Several options are new or updated, enabling additional command line configuration of your load test application and output. These changes and addition include:

  • Some options were shortened to fit standard console width as part of the switch from structopt to gumdrop, enabling:
    • Restructuring of the help page to logically group related options.
    • Rewritten/simplified configuration descriptions to fit standard console width, see the difference between 0.9.1 and the current version.
  • Goose performs a checksum to confirm all Workers are running the same load test as the Manager, use --no-hash-check to ignore (not recommended).
  • --sticky-follow makes redirects of the GooseClient base_url sticky, affecting subsequent requests. When you make a request as part of a test, if that request redirects to a new URL (for example, from example.com to foo.example.com), all subsequent requests will go to the redirected URL (foo.example.com). This feature is off by default.
  • Added set_client_builder, allowing load tests to build Reqwest clients with custom options. This exposes the full power of the Reqwest library to anyone writing a Goose load test.

Version 0.7 integrated httpmock into testing load tests: With this change we started doing proper functional, integration, load, and comment testing to validate every PR and and commit into the mainline branch. This improves the stability and reliability of Goose as development moves forward. Tag1 thanks Alexander Liesenfeld for his help in making this possible.

Finally, this release also includes updates to documentation, reworded errors, enforcement of error.detail, increased precision in smaller metric values, and consistently built configurations from arguments.

All of these changes, in addition to the many other improvements made in the past year, have significantly improved the speed, usability, and performance of Goose and its ability to help you load test your website.

See our other Goose content to learn more about this highly scalable load testing software.

Photo by vivek kumar on Unsplash