HMAC signs bytes, not text: binary webhook payloads in Standard Webhooks

The problem

A Standard Webhooks signature is v1, followed by the base64 HMAC-SHA256 of {msg_id}.{timestamp}.{payload}. The payload is whatever bytes were sent. The JavaScript and Python libraries turned it into text first:

• JavaScript: sign(), which verify() uses, called payload.toString() on a Buffer. That decodes UTF-8 and replaces every invalid sequence with U+FFFD, so the HMAC covered different bytes than were sent.
• Python: verify() called data.decode() on bytes, which raises UnicodeDecodeError for a non-UTF-8 body, even when the signature is wrong, instead of a verification error. sign() only expected str; given bytes, it interpolated their repr (b'...') and returned a wrong signature without complaint.

So a correctly signed gzip, protobuf or image body could not be verified. The Go library already signed the bytes as given.

The proposed fix

Both libraries now build prefix || payload bytes instead of interpolating the payload into a string:

const prefix = new TextEncoder().encode(`${msgId}.${timestampNumber}.`);
const toSign = new Uint8Array(prefix.length + bytes.length);
toSign.set(prefix);
toSign.set(bytes, prefix.length);

to_sign = f"{msg_id}.{timestamp_str}.".encode() + _payload_bytes(data)

The tests include a known-answer vector over all 256 byte values whose expected signature, according to the pull request, comes from the Go library. A test vector produced by an independent implementation is the right kind of test: it cannot simply mirror the code under test.

What I found in review

The change is correct. String input and the same text as UTF-8 bytes sign identically, so existing users see no difference. My notes were about inputs that are now accepted without an error.

Any typed array is now "bytes"

The JavaScript helper accepts anything for which ArrayBuffer.isView is true. It uses isView instead of instanceof Uint8Array so that arrays from another realm (an iframe, a vm context) still work, which is a good reason. But isView is also true for DataView, Uint16Array and Float64Array, and those would be signed as their raw platform-endian bytes. A signature that only works on one machine is worse than an error. Restricting to byte views keeps the cross-realm support:

ArrayBuffer.isView(payload) && (payload as any).BYTES_PER_ELEMENT === 1

Python's bytes() is not a type check

bytes(5) is five zero bytes and bytes([1, 2]) is b"\x01\x02". Passing an int or an already-parsed list would now produce a signature over unrelated bytes, while the JavaScript library throws. I suggested an explicit check:

if isinstance(data, str):
    return data.encode()
if isinstance(data, (bytes, bytearray, memoryview)):
    return bytes(data)
raise TypeError("Expected payload to be of type str or bytes")

A BOM now parses

On the JSON path, new TextDecoder().decode(bytes) strips a leading UTF-8 byte order mark; the old Buffer.toString() kept it, and JSON.parse then threw. That is probably intended and matches Python's json.loads(bytes), so the two libraries now agree. It deserves a test or a sentence in the description so the behavior is pinned.

Coordination

Pull request #343 also changes verify(), and its version still calls payload.toString() after the signature loop. Whichever lands second has to keep the byte-based handling, or a mechanical rebase brings the lossy decode back. I also suggested putting the 0-255 vector somewhere shared, such as the spec folder, so the Rust, C#, Ruby and PHP follow-ups reuse one vector instead of copying it.

What happened next

Takeaways

• An HMAC authenticates bytes. Any decode and re-encode between the wire and the HMAC changes the input.
• When a function starts accepting a wider type, check what it does with the inputs nobody intended to pass.
• Cross-language known-answer vectors belong next to the specification, not in one library's test file.