Some of my annoyances have already been discussed, but I'll list them here again anyway, for emphasis, and in no particular order:
Inconsistent font support.
Lack and inconsistent online documentation. All applications must as a minumum have a man page, that describes everything that the application can do, and all of its options. If you also want to included HTML documentation you can. All of the other online documentation applications are either an unworkable superset of man or a horible subset of HTML or impossible to find.
Inconsistent cut and paste. Swipe with button 1 and past with button 2 is the standard. Oh by the way, click one to position cursor, double click to grab the current word, triple click to grab the whole line (including the newline) so when you paste it, you don't also have to hit return!
Modal windows... It seems like a lot of GUI applications these days are written by MS writers. There is rarely a need to have a modal window, except for maybe an error window that MUST be acknowledged before continuing. But even most error windows don't have to be modal That way you can change to another window, to browse around, and maybe fix the problem right then, or cut and paste the correct solution from elsewhere... but with all the modal windows... you can't.
Why does it take forever (>5 seconds) to repaint your X screen after the screensaver kicked in? (even on a 2 Ghz machine with 512M of RAM. My old 486/66 with 16Meg unblanked the screen much faster under OpenServer than Linux does today on a _much_ bigger machine. This makes Linux feel sluggish.
On the support front, the biggest annoyance are companies that have Linux software done, and working but refuse to release it. ie. Intel Centrino wireless support, Lotus Notes, etc.