Snonymous inner classes in Java play two roles, one is standing in for a reference to a method, to which delegates do represent an improvement, at least in terms of avoiding syntatic clutter.
The second semantic feature they provide is that of a closure (albeit with a few limitations), or in other words, a block of code which captures state from its lexical context. In this role they are very useful, although the wordiness of the construct in Java leads me to use them less than I would in languages with more spartan closure-like constructs.
So delegates should not be seen as a complete replacement for anonymous inner classes, but a specialization that helps reduce syntactical clutter for that specific use.
And you are correct that in our solution, attempting to create a delegate on a method that does not provide the correct type signature will not be discovered till runtime. I'm not sure how you would avoid that problem short of extending the compiler itself.