The Polyphonic C# Join Pattern: (aka nohup and Unix Pipes)

by M. David Peterson

It's amazing to me how seemingly complex programming techniques are really not that complex at all if you break them down into their basic Unix CLI equivalence.

Take, for example, Polyphonic C# where they introduce the concept of the asynchronous join pattern,

Continuations, futures, and whatnot - Thoughts on some asynchronous patterns - B# .NET Blog

Join patterns

If you've had a rendez-vous with Ada in a previous life, you'll be familiar with the concept of join patterns (or shorthand joins). Our Microsoft Research department has done quite some work on this field already, as you can read here: http://research.microsoft.com/~crusso/joins/. It's part of Polyphonic C#, which is by itself part of Cw and based on join calculus. We already got the LINQ inspiration from it, joins might be lurking around the corner too. There actually just two basic concepts to understand here:

* Asynchronous methods: imagine a keyword async as a modifier on methods (which would imply the method is a procedure, hence the void return type becoming redundant for those type of methods). No longer do you need to create separate versions of a method with or without the Async suffix to work around return type overloading limitations imposed by the runtime. You can think of the method as a wrapper around a task scheduling call, wrapping the entire body. Whether or not this spawns a new thread is another matter.
* Join patterns: also known as chords, is a declarative way of a WaitAll style of guard mechanism to enter a method.

Our monk would look like:



class Monk
{
public async RequestCard(string to) { ... }
public string GetCard() & public async RequestCard(string to) { ... }
}



Basically requests for cards queue up on the monk's GetCard operation, ready to get processed when the monk is awake. Actually this sample is convoluted and our monk suffers from some concurrency diseases (think of the cards it would hand out if there are multiple outstanding requests - any ordering guarantees without "ticketing"?) but I won't get in details for now - it's just (yet) another way of thinking about asynchronous processing with today's materials in the room. If you're inspired by join patterns, check out the Polyphonic C# paper.


Sounds interesting, and extremely cool. And I can't wait to start playing with this feature if and when it becomes a part of the core C# language specification. But I'm not sure it's something we haven't been able to do for *years* already In the mean time, for those that want to begin sinking their teeth into async/concurrent programming techniques, it seems to me this same general idea can be simulated using nothing more than nohup and Unix pipes. For example, take the following function which just does some random piping operation enclosed in a loop, taking an argument passed to it as an identifier as to which operation is currently taking place, prepending a formatted date in front of it, and then piping the result to sed to perform a search and replace operation on the current day of the week,

7 Comments

dk
2008-06-03 11:24:09
What are you implying about MS research?
M. David Peterson
2008-06-03 11:48:00
@dk,


>> What are you implying about MS research?


Nothing. MS Research is one of finest research establishments on the planet. I wasn't attempting to imply that Polyphonic C# joins weren't anything less than *fantastic*. Just that conceptually speaking this same general area of computer science can be thought of in terms that are easy to understand and put into use via tools we all use on a fairly regular basis. As far as I know there isn't an existing compiler that supports this (though come to think of it, maybe I better verify that for sure. If yes, I know what I'll be getting my hands dirty with this afternoon :D) so to highlight the potential future direction of one of my most favorite languages, this seemed like an appropriate real-world analogy to use.


Now to check and see if I missed the compiler download on the MSR site! :D

M. David Peterson
2008-06-03 11:55:52
@dk,


Hmmm... Weird. I hadn't realized that Polyphonic C# was absorbed by the COmega project. In fact, I was under the impression the COmega was no longer an active MSResearch project. But from the looks of it -- http://research.microsoft.com/comega/ -- I guess that isn't the case.


Nice!


/me is digging deeper.

dk
2008-06-03 13:54:13
Sorry for the cheap shot -- please feel free to throw a chair across the room.
M. David Peterson
2008-06-03 14:19:11
@dk,


Done and done. ;-)


No, I realized after your comment that the portion of the original post that now has a line through it definitely sent the wrong message. No chairs will be tossed (at least on your behalf ;-))


Thanks for bringing out the need to better communicate my overall point!

Bart
2008-06-03 14:33:06
Hi folks,


Thanks for reading my post and absorbing it. Indeed, lots of those seemingly rocket science research concepts are not that difficult indeed. Think of lambda calculus, which is no more scary than delegates (and agreed, expression trees by extension) that are well-known for ages; or think of monads, maybe the name of it sounds scary but LINQ just is one of these and apparently people can deal with those just fine.


Back to the comparison between join patterns and equivalent concepts in shells. I agree we can learn a lot of things from the shell world, and indeed we do. PowerShell 2.0 has jobs, and there's this great cross fertilization between IT tool concepts and framework concepts. But we should even go that far in the (very near) future, think of the Extended Type System in PowerShell 1.0 versus concepts in the DLR on extending types (and by "extension", C# 3.0's and VB 9.0's extension methods).


The reason I blogged about this topic is the Web 2.0 era where concepts like these become increasingly important in order to deal with distributed systems programming, not surprisingly something which IT admins have dealt with (at least on the scripting level) for ages hence the parallels between these concepts.


For research projects like Comega and Polyphonic C# it's always hard to judge their "dead or alive" status. Things get absorbed by other projects, spin-offs are created (like the Singularity project has its take on language specific details as well, in this case Spec#) and ultimately things get productized in product or incubation project groups (think of LINQ, the Parallel Extensions, work done in Volta, etc).


Thanks,
-Bart

M. David Peterson
2008-06-03 15:16:10
Hey Bart,


Thanks for taking notice of this post! I'm a *BIG* fan of your blog, finding tons and tons of insightful and meaningful info on a regular basis. For example, the post referenced in this entry. :)


>> Thanks for reading my post and absorbing it.


Thanks for writing it!


>> Indeed, lots of those seemingly rocket science research concepts are not that difficult indeed. Think of lambda calculus, which is no more scary than delegates (and agreed, expression trees by extension) that are well-known for ages;


Agreed. Given that we hacker types tend to be founded upon layers and layers of original creative thinking, yet this same creative thinking is founded upon the slightly more structured field of mathematics, the overlap of our creations is inevitable.


>> or think of monads, maybe the name of it sounds scary but LINQ just is one of these and apparently people can deal with those just fine.


It's funny to me how that works: It seems that familiarity of syntax, coding concepts, and usage patterns can easily represent the difference between high productivity and rapid hair loss for new concepts that -- at their very core -- are similar in function to an idea or process we are already familiar with.


>> I agree we can learn a lot of things from the shell world, and indeed we do.


I think the reason it's easier to grok something when thought of in terms of the shell is this is something that we interact with via individual steps rather that scripted tasks and functions which -- by their very nature -- tend to hide the more complicated functions and tasks behind generalized program names. Very declarative -- please take these values and give me something in return -- which ensures we can focus on the overlying macro process rather than the underlying micro processes. Just such an abstraction -- to me, anyway -- provides the perfect foundation for learning about higher level concepts rather than focusing on the lower level implementation details.


>> For research projects like Comega and Polyphonic C# it's always hard to judge their "dead or alive" status.


Oh, absolutely!


>> (think of LINQ, the Parallel Extensions, work done in Volta, etc).


All great and wonderful projects! I'm especially intrigued by Volta. *SO* much potential!


Thanks again for taking the time to follow-up with a response!