I/O benchmarking using FIO (Flexible I/O Tester)

Go to file

dvosler a95166919c initial		2024-06-11 09:31:58 -04:00
README.md	initial	2024-06-11 09:31:58 -04:00
fio_slurm_batch.sh	initial	2024-06-11 09:31:58 -04:00
io_throughput_test-fio.sh	initial	2024-06-11 09:31:58 -04:00

README.md

FIO Benchmarking Script

This script performs disk I/O benchmarking using FIO (Flexible I/O Tester) on a specified network-mounted directory. It conducts both random and sequential read/write tests to evaluate the performance of the disk under various conditions.

Prerequisites

FIO installed on your system.
Bash shell.
Network-mounted directory accessible at the specified mount point.

Usage

Ensure the network mount point specified by the network_mount variable or -m option exists and is accessible.
Modify the script variables as needed to match your testing requirements:
- file_size: Size of the test file. Default is "8G".
- iodepth: Number of I/O operations to keep in flight. Default is "16".
- numjobs: Number of parallel streams for the test. Default is "4".
- runtime: Limits each test's runtime (in seconds). Default is "300" (5 minutes).
- ioengine: I/O engine to use. Default is "libaio".

Execute the script with the desired options:

./fio_throughput_test.sh -s <file_size> -d <iodepth> -j <numjobs> -r <runtime> -e <ioengine> -m <network_mount>

To see all available options, use the -h flag:
```
./fio_throughput_test.sh -h
```

Script Structure

Initialization: Sets up variables for the network mount, log directory, and FIO parameters.
Benchmark Tests:
- Random Write Test: Measures random write performance.
- Random Read Test: Measures random read performance.
- Sequential Write Test: Measures sequential write performance.
- Sequential Read Test: Measures sequential read performance.
Logging: Outputs the results to a log file named fio-<hostname>-YYYY-MM-DD.log in $HOME/log/fio-tests.

Modifying the Script

You may adjust the script parameters such as file_size, iodepth, numjobs, runtime, ioengine, and network_mount based on your specific testing needs or the capabilities of the disk being tested.

Note

This script is designed for use with network-mounted storage. Ensure that your network mount is correctly configured and that the specified mount point is accessible before running the script.

Running with Slurm

To run the FIO benchmarking script on multiple nodes using Slurm, you can use the included batch script.