Jul 20, 2023

Efficient, portable erasure coding tool

The zfec package implements an “erasure code”, or “forward error correction code”.

The most widely known example of an erasure code is the RAID-5 algorithm which makes it so that in the event of the loss of any one hard drive, the stored data can be completely recovered. The algorithm in the zfec package has a similar effect, but instead of recovering from the loss of only a single element, it can be parameterized to choose in advance the number of elements whose loss it can tolerate.

This package is largely based on the old “fec” library by Luigi Rizzo et al., which is a mature and optimized implementation of erasure coding. The zfec package makes several changes from the original “fec” package, including addition of the Python API, refactoring of the C API to support zero-copy operation, a few clean-ups and optimizations of the core code itself, and the addition of a command-line tool named “zfec”.

Checkout these related ports:
  • Zbase32 - Base32 Encoder/Decoder
  • Ytnef - Unpack data in MS Outlook TNEF format
  • Yj - Convert between YAML, TOML, JSON, and HCL
  • Yj-bruceadams - Command line tool that converts YAML to JSON
  • Xml2c - Convert an XML file into C struct/string declarations
  • Xdeview - X11 program for uu/xx/Base64/BinHex/yEnc de-/encoding
  • Wkhtmltopdf - Convert HTML (or live webpages) to PDF or image
  • Uulib - Library for uu/xx/Base64/BinHex/yEnc de-/encoding
  • Uudeview - Program for uu/xx/Base64/BinHex/yEnc de-/encoding
  • Unix2dos - Convert ASCII newlines between CR/LF and LF
  • Tuc - Text to Unix Conversion
  • Trans - Character encoding converter generator
  • Tnef - Unpack data in MS Outlook TNEF format
  • Ta2as - TASM to AT&T asm syntax converter (GNU AS)
  • Showkey - Display cooked key sequences (keycap-to-keystrokes mappings)