To solve this problem, developers developed compression algorithms and tools capable of compressing data in real-time, reducing the size and processing power required. One of these tools is Zstandard, commonly known as Zstd.
To check if Zstd supports your desired language, check the resource provided below:
If you wish to look at benchmarking information about Zstd, use the link below:
This tutorial will show you how to compile and install the Zstd tool in Linux, then use it to perform data compression and decompression.
How to Install Zstd
To use Zstd, we need to install it by compiling from the sources. Depending on the system you are running and the configuration, you may need to install dependencies and tools to perform the compilation successfully.
Start by updating your system and installing GNU make using the command:
sudo apt-get upgrade
sudo apt-get -y install build-essential wget tar
Once we have all the tools installed, we can download the source files and compile them. Start by navigating where you have read, write, and execute permissions. ~/Desktop
Next, use wget to download the files into the directory.
Now unarchive the download file and navigate into the directory using the commands as:
The final steps are to install Zstd by using make and make install inside the Zstd directory.
sudo make install
Once the compilation and installation complete successfully, you can start using Zstd on your system to compress and decompress files.
How to Use Zstd
Zstd does not defer from popular compression and decompression methods at all. Although the underlying technology and implementation are different from other tools, compressing a file with Zstd is similar to tar and gzip syntax.
How to compress a file
To compress a file, call the zstd command followed by the -z flag, which tells zstd to do the compression, and finally, the name of the file to compress.
For example, the command below compresses the system-backup file into a .zst file.
The command output as shown below:
$ ls system-backup
You should see a file with .zst extension as:
Once the command executes, the file gets compressed and creates a filename .zst file that you can decompress.
Compress and remove the source file
As you can see from the above command, the source file does not get removed by default upon compression. You can specify to remove the source files by using the –rm flag:
system-backup :100.00% (1821109 => 1821164 bytes, system-backup.zst)
Specifying the –rm flag automatically removes the source file.
Get file info
To display related information about the Zstd compressed file, you can use the -l flag followed by the file name. The displayed information includes the file size, compression ratio, and the file checksum
*** zstd command line interface 64-bits v1.4.9, by Yann Collet ***
system-backup.zst # Zstandard Frames: 1
Window Size: 1.74 MB (1821109 B)
Compressed Size: 1.74 MB (1821164 B)
Decompressed Size: 1.74 MB (1821109 B)
Specify the compression level
To explicitly specify the compression level, use the – where the level is a value ranging from 1 – 19. The default compression level is 3. You can also unlock higher compression levels, i.e., level 20 – 22.
NOTE: The higher the compression level, the higher memory usage.
For example, to compress a file with a compression level of 10, use the command:
Specify Compression Speed.
Zstd also allows you to set the compression speed ranging from 1 – to. The compression speed is inversely proportional to the compression ratio. The default compression speed is 1, and the higher the value, the faster the compression speed.
For example, to use the maximum compression speed, use the command:
Specify Compression Format
You can also specify the compression format to use if you do not like the default zst compression. Formats include zstd, gzip, xz, lzma, and lz4.
Use the –format flag and specify the format as:
Compress file list
Suppose you have a list of files you would like to compress all at once. Zstd allows you to pass a file containing a list of files and compresses them recursively.
For example, a file list.txt containing the files
Once you save the file, you can pass the list to Zstd with all other options to perform on the files.
This command will compress all the files specified in the text file and remove them upon completion.
Decompress a file
To decompress a file, you can use the -d flag with the zstd command or simply use the unzstd command to decompress.
sudo zstd -d system-backup.zst
Verify file integrity
To test the integrity of a zst compressed file, use the -t flag as shown in the command below:
As we can see from the examples, Zstd is a powerful compression algorithm with numerous use cases. To learn more about how it works and its implementation, check the man page and experiment.
Thanks for reading.