Ok, I have to respond to this.
"I have no problems updating and maintaining that code."
Emphasis on the "I". Who is going to maintain that code when you're gone? Documentation and code control is a great start, don't get me wrong, but I would hope you agree that it's easier to hire developers in C/C++/Java than Intel Asm these days.
"I like to download open source projects on the net and speed them up using assembler. The last one I downloaded and sped up, I got a 10x increase in speed over their C implementation."
Great, you got a 10x speed increase from the C implementation, with the tradeoff of limiting the application to Intel 686 only. Perhaps I'm making an assumption about your assembly skills, perhaps that should have read Sparc or PowerPC or Z80 assembly. Whatever. Properly written C is probably one of the most portable languages out there, and that is an undeniable advantage in the open source community..
To cap this argument, here's my thoughts on the subject: There are 2 classes of software, CPU-bound and IO-bound. CPU-bound apps are all processing, and could definitely help from knowledge of low-level techniques, be that assembly or just general optimization strategies. IO-bound apps, however, spend most of their time waiting for input or causing output. These apps could probably make the tradeoff of slightly less efficient code for quicker, better amd safer development. Determine which components of your system are which, and optimize accordingly.