Excellent point. TFTP is typically used only in very tight quarters, these days the only time you're likely to run into it is when booting over a network connection. In that case, the code for downloading the kernel etc. has got to fit into the bootloader (or network card), and they don't have the room to implement the whole tcp/ip stack, so TFTP serves as a kind of poor man's TCP.
There is indeed an opening handshake (simpler than TCP's), but there isn't a closing one. Each TFTP data packet contains a 512 byte payload, except the last one. So when a packet shows up with less than 512 bytes, it's taken to mean that the transfer is complete.
Ultimately, TFTP was an excercise in using the least possible amount of code to transfer a chunk of data over IP. If you've got a TCP stack lying around, it is infinitely preferable to use that.