Binary Encodings: From Bits to Bytes

Binary data is rarely handled in its raw string of 1s and 0s. Instead, developers and systems rely on encodings that balance human readability with machine efficiency. While decimal and octal have specific historical or niche uses, Hex and Base64 have emerged as the industry standards for two distinct reasons: Bit Alignment and Protocol Compatibility.

Why not Decimal and Octal?

Variable-width digits makes it inefficient for high-speed binary parsing or memory-aligned operations, as the system must scan for delimiters to know where one number ends and the next begins.

You can see the red highlighted digits in Base8 (Octal) format. Those are digits which are formed by combining bits across byte boundaries.

Even if I generate decimal equivalent for each byte as shown in the image and concatenate them, different binary sequences can result in the exact same decimal string since there is no delimiter (no fixed-width). As a workaround, I need to make decimal to fixed-width like 001 to 255, which would increase the size of resultant output.

In hex, len(hexEncodedString) / 2 tells you the number of bytes represented exactly. In decimal, there is no formula, a 9-character decimal string could represent anywhere from 3 to 9 bytes. This breaks any protocol or storage format that relies on length for boundary detection.

Dominance of Hexadecimal

Hexadecimal (Base 16) is the native way to view binary data because of its perfect mathematical symmetry with the 8-bit byte.

• Perfect Nibble Alignment: Single hex digit represents exactly 4 bits. Therefore, exactly two hex digits (0x00 to 0xFF) represent one byte (0 to 255). Because of fixed-width boundary, multiple hex strings can be appended and still be parsed without any data corruption.

• Memory Efficiency: Hex is more compact than decimal for large values. Hex format is easy to parse since hex allows for

Necessity of Base64

Base64 is not designed for human readability, but for transport integrity. It is the winner in any scenario involving the movement of binary data over text-based protocols.

• Safe Character Set: ASCII control characters (e.g., Null, End-of-Transmission) can cause misinterpretation and data corruption. Base64 solves this by mapping binary data into 64 safe and printable ASCII characters \(A-Z, a-z, 0-9, +, /\)

• The Trade-off: While Hex is used for viewing data, Base64 is used for sending it. The primary cost is a 33% increase in data size, a price paid for the guarantee that the data arrives intact.

As a summary, every binary serialization format commits to a trade-off: compactness, human readability, ease of parsing, or bit alignment. We need to choose the right one based on our use case.