# Exercise 07 — The helper that only corrupts large documents **Failure mode:** message framing — treating a byte stream as a message stream **Language:** C · **Runtime:** about 2 seconds · **Difficulty:** the one small tests always pass --- ## The prompt > A helper process and its client talk over a Unix domain socket. Each message is a > self-describing frame: a header with a magic number, a sequence number, a payload length > and a checksum, followed by that many payload bytes. > > The author tested it with short messages, where one `write(2)` reliably produced exactly > one `read(2)`, and concluded that a stream socket delivers messages. > > The field reports: "it works perfectly in testing and corrupts replies in production, but > only for large documents, and only sometimes. Small documents are always fine." > > Prove where the bytes go, fix it, and tell me what your fix costs. Answer before you run anything: **what does the word "stream" in `SOCK_STREAM` promise, and what does it not?** --- ## What you are given | Path | What it is | | --- | --- | | `broken/framing.c` | The starting point. One `read` per message. | | `fixed/framing.c` | One correct repair. | | `solution.patch` | The diff between the two, applies with `patch -p1`. | | `check.sh` | Asserts the survival rate and the byte totals. | The sender writes each frame with a single `write_fully`, so the sender is provably not the problem: header and payload leave together, in order, complete. --- ## Reproduce the measurement ```bash cd exercises/07-stream-framing clang -O2 -g -Wall -Wextra broken/framing.c -o /tmp/fr_broken clang -O2 -g -Wall -Wextra fixed/framing.c -o /tmp/fr_fixed /tmp/fr_broken /tmp/fr_fixed ``` Observed on an Apple M4 Pro, macOS 26.3 (25D125): ``` broken fixed messagesSent 2,000 2,000 bytesRead 58,472,000 58,472,000 identical — nothing was lost readCalls 8,079 11,001 the FIX makes more calls goodMessages 328 2,000 lostSyncFrames 7,000 0 goodFraction 0.1640 1.0000 verdict misframed framed ``` **Reproduce these; do not quote them** — except the 2,000, which is structural. --- ## The evidence to collect 1. **`bytesRead` is identical in both runs.** This is the observation that tells you where *not* to look. The transport delivered every byte, in order, exactly once. The socket is not dropping anything, the sender is not truncating anything, and no buffer size needs tuning. The defect is entirely in how the receiver **grouped** bytes it already had. 2. **`goodFraction = 0.1640`.** The frames that survive are the small ones. The size table in the fixture is `{16, 64, 200, 4096, 65536, 131072, 32768, 8}` — the first three and the last fit comfortably in one delivery; the large ones do not. 3. **`lostSyncFrames = 7,000`.** Once a `read` returns a partial frame, the *next* `read` starts in the middle of a payload. The receiver then interprets payload bytes as a header, the magic number fails, and it never recovers. A framing bug is not a per-message failure — it is a **loss of synchronisation**, and everything after it is garbage. 4. **The `read` that returns less than you asked for is not an error.** It is the documented contract. `read(2)` returns "the number of bytes actually read", which may be fewer than requested, and a short return on a stream socket means "this is what is here now", not "this is the end". --- ## Success criteria - [ ] The broken build still mis-frames — you have not edited it. - [ ] Your fixed build reports `goodMessages == messagesSent` and `badMessages == 0`. - [ ] Your fixed build's `bytesRead` equals the broken build's, exactly. - [ ] Your receiver reads the header and the payload as **two exact-length reads**, each looping until satisfied. - [ ] You can say why your fix makes **more** `read` calls, not fewer, and why that is the right trade here. - [ ] You can name the one transport in this family that would not have had this bug, and what it costs instead. Run `./check.sh` to have all of that checked for you. --- ## Hints
Hint 1 — the length was always there `struct msg_header` has a `payload_len` field, and the broken receiver reads it. Look at what it does with it: it *compares* it against how many bytes happened to arrive. It never uses it to decide how many bytes to *ask for*.
Hint 2 — two reads, not one You cannot know how big a frame is until you have read its header, and you cannot read a header from a stream in one call either. So the shape is: read exactly `sizeof(header)` bytes, then read exactly `payload_len` bytes. "Exactly" is doing all the work in that sentence.
Hint 3 — what does "exactly" mean when read returns short? It means a loop. Track how many bytes you still need, call `read` for that many, add what you got, repeat until zero. Handle a return of 0 (clean end of stream) differently from a return of 0 *mid-frame* (a truncated frame, which is an error).
--- ## Solution
Reveal the solution ### The repair ```c static int read_fully(int fd, void *p, size_t n, long *calls) { unsigned char *b = p; size_t off = 0; while (off < n) { ssize_t r = read(fd, b + off, n - off); if (calls) (*calls)++; if (r == 0) return off == 0 ? 0 : -1; /* clean EOF, or truncated frame */ if (r < 0) return -1; off += (size_t)r; } return 1; } /* receiver */ struct msg_header h; if (read_fully(fd, &h, sizeof h, &calls) != 1) break; if (read_fully(fd, buf, h.payload_len, &calls) != 1) break; ``` ### Why the bug is size-dependent A stream socket hands the receiver whatever is in its buffer when the `read` runs. For a small frame that is usually the whole thing, because the sender's single `write` landed atomically in the socket buffer and the receiver had not been scheduled yet. For a frame larger than the socket buffer the sender's `write` necessarily completes in several kernel-side pieces, and the receiver can be scheduled between any two of them. That is why the bug is invisible in a unit test with 100-byte messages and reliable with 128 KiB ones — and why "it works on my machine" is a statement about message sizes, not about correctness. ### What the fix costs **More system calls**: 11,001 against 8,079, about 36% more. Each frame now takes at least two `read` calls instead of one, and large frames take several. This is the right trade — correctness is not optional and the alternative is not actually cheaper, it is just wrong — but naming the cost unprompted is what distinguishes a considered answer. If the call count mattered, the standard refinement is a **user-space buffer**: read as much as is available into a ring buffer with one large `read`, then hand out complete frames from it. That is exercise 05's lesson applied to exercise 07's problem, and it gets you both. ### The transport that would not have had this bug `SOCK_DGRAM` on a Unix domain socket **preserves message boundaries**: one send, one receive, no framing code. What it costs is a maximum datagram size (so a 128 KiB message needs fragmentation you now write yourself), and the need to size the receive buffer for the largest message you will ever accept, since a too-small buffer truncates rather than returning the rest later. The same choice exists one layer up: XPC and Mach messages are **message**-oriented and do this framing for you, which is one of the strongest practical arguments for using them instead of a raw socket between two processes you control. ### The repairs that do not work - **A bigger receive buffer.** The broken receiver already passes `sizeof(header) + MAX_PAYLOAD`. The kernel is not limited by your buffer; it is limited by what has arrived. - **`SO_RCVLOWAT`.** Sets a minimum byte count before `read` returns, which makes the bug rarer for a *known fixed* frame size and does nothing for variable ones. Rarer is worse. - **A delimiter instead of a length.** Workable for text, and it means scanning every byte and escaping the delimiter in payloads. A length prefix is cheaper and does not constrain the payload's contents.
--- ## Going further - Change `socketpair(AF_UNIX, SOCK_STREAM, ...)` to `SOCK_DGRAM` in the broken build, drop `MAX_PAYLOAD` to 8 KiB, and watch the broken receiver start passing. Then explain why that is not a fix but a change of contract. - Add a deliberately hostile `payload_len` to one frame and check what your receiver does. A length field read off a wire is attacker-controlled input; bounding it before you allocate or read is not optional. The fixture bounds it against `MAX_PAYLOAD`; make sure yours does too.