Graphical Composition in Avalon
Subject:   macos x already works like this
Date:   2004-03-11 04:58:13
From:   igriffiths
Response to: macos x already works like this

"Nobody waited 15 years for Quartz. So this argument is fatedly flawed. "

In which case, in what sense are people waiting 6 years for Avalon? It wasn't even announced until late in 2003, and is supposed to be shipping in 2006 or 2007. So that's not really a 6 year wait. (OK, I was being deliberately extreme in the example above to emphasize the point. But when was the last time the Mac drawing APIs had a substantial workover prior to Mac OS X?)

When you said that people will have waited for 6 years, I was assuming that you meant the gap between two technologies. So I no longer have any idea what you really mean by with this 6 year wait thing.

"You, on the other hand, stated that Quartz wasn’t up to snuff for your work. Your decision to forgo working with an excellent compositing engine for six years is still baffling in my honest opinion."

I think what I actually said was that it was disappointing. That's not quite the same thing. It's up to snuff in the same way as Windows is - I can usually make either of them do what I want given sufficient effort. In certain cases you have to abandon the 2D API and drop down to fully-accelerated pipelines like OpenGL or DirectX (on Quartz and Windows respectively) to get sufficient performance, but if you are determined enough, and have the time, you can usually bend them to your needs, even though you sometimes have to completely abandon the normal way of doing things.

But it shouldn't be that much effort. What I like about Avalon is that it will make it easy to use some UI features which currently require heroic programming to make it work fast enough. (Just to be clear, I'm hoping that future versions of Quartz will also make it easy to do things that are hard today. I just have no idea right now what Apple's plans are there.)

"You can’t argue with the fact that most of the functionality presented in your article is already facilitated by Quartz."

Yes, I'm not arguing with that. (I have been arguing that quite a lot of what's in Quartz is in Windows today, but I don't disagree with the above.) But then about half of what the article covers is the basics of composition, because most windows developers aren't familiar with that. Even though Windows has supported top-level composition since Windows 2000, most developers don't actually use it.

So is the fact that Windows 2000 style composition and more is available today in Quartz relevant to an article that explains some differences between current versions of Windows and the next version of Windows?

"You yourself say that OS X is pleasing to the eyes, so it should be fare to assume that the quality of the output generated by Quartz is not an issue."

If OS X really was a good showcase for the Quartz compositor, I would agree, but I don't think it is. It makes very unambitious use of the compositor.

The Mac looks nice principally because the graphic design is good, not because of the way the compositor works. An awful lot of the eye candy is done with bitmaps, and the composition techniques it uses don't show off Quartz's technical abilities. I had originally assumed (before I got my Mac) that the Aqua look was all done by exploiting the compositor, but it doesn't take long with the development tools to discover that this is not the case. Try and arrange a UI so that some of the widgets overlap, and it tends to end up looking quite a lot like the example in my article where I show how pseudo-transparency in Windows falls down. UIs have to avoid certain styles of layout to avoid this. (All of the recommendations in the user interface guidelines steer you against designs that cause a problem.)

The only things that exploit the compositor are the drop shadows and the semi-see-through bits. Since both of these effects have also been doable in Windows since Windows 2000, I don't think it does all that good a job of showing off the Quartz compositor. Most of what gives Aqua its distinctive look is all the bitmaps used for the various UI widgets and the window backgrounds.

Given all that, I have to disagree with the sentiment you expressed next:

"This is why people feel you could have mentioned Quartz as an example of how composition can be done well. "

The Aqua look is absolutely not a good demonstration of the Quartz compositor. It's a testament to the design skills of the artists at Apple, but it is in no way a technical tour de force. It barely uses the compositor. And what it *does* do with the compositor is basic stuff - they don't do anything you couldn't do with the Windows 2000 compositor.

I think this is a shame because, as I've already said, the Quartz compositor is more capable than the Windows 2000/XP one, so it's a pity that Mac OS X doesn't showcase this. (But I suspect the reason they don't is that this would raise the minimum spec required to run OS X - see the comments towards the end of this post on for a little detail on why it's comparatively expensive to exploit the the Quartz compositor.)

"You stance is way too negative toward Quartz. "

OK, you've got me there - I'll admit that. But I'm just defending what I wrote, and to do that I'm countering all of the arguments presented in this thread that I have disagreed with to a greater or lesser extent. This inevitably comes across as negative, because I'm always taking the contrary stance. So I apologise for that. I'm not as down on Quartz as it probably seems.

Just so it's clear, I think Quartz is a Good Thing. It's better in every respect than where GDI+ on Windows is today. I hope it continues to evolve - if, after Avalon ships, Quartz improves enough to regain its lead (or even steals a march by beating Avalon to it) then that can only be good for the quality of graphics systems in the industry as a whole. I want to see Apple and Microsoft continually strive to outdo each other, so that the state of the art moves ever forwards.

"Many of the features you herald in your article are many of the same features “hyped” when OS X was first released to the public three years ago."

Agreed. But I still don't understand why that would be relevant on a site targeted at Windows developers. As you already acknowledged, the first half of the article is about bringing people up to speed on composition, rather than talking about anything new. As I've already said, I don't believe that telling people that it shares some aspects in common with a system they are unfamiliar with is a useful observation.

And I think anyone already familiar with Quartz will recognize the stuff they are familiar with without me needing to point it out. So I honestly don't see how you think that references to Quartz would have made the article better for any readers. It would have been meaningless for those unfamiliar with Quartz, and redundant for those familiar with Quartz. Who benefits?

As for your example of layering the rectangle, the ellipse, and the text, thanks - that's a great example! So everything is fine up to the point your example goes to, but here's a followup step to be performed some time after the UI has already become visible. (So we're adjusting a UI that's already on screen.) This step shows the problem:

Perform the following operation: go back and change the transform on the Ellipse.

In Avalon, the compositor will handle the UI update - once you have changed the transform (i.e. just set a property on the relevant transform object), the ellipse's appearance will be changed appropriately. (So if you applied a rotation to the transform, the ellipse will rotate, but everything else will stay still.) You don't need to regenerate the imagery - it's enough to simply adjust the transform and the compositor does the rest.

In Quartz, the instruction "go back and change the transform on the Ellipse" don't even make sense. That's not something you can do because Quartz doesn't retain the transform - the transform was an object you created whilst doing the drawing, and even if you did hold onto it in your program, adjusting it later on won't have any impact on the display. As far as the Quartz 2D API is concerned, it's a transient entity.

With Quartz, what you've got after performing those drawing operations is a single surface containing the combined otuput of those three drawing primitives. If you want to rotate the ellipse, you have no choice but to clear the surface, and redraw the whole thing, using a suitably modified transform for the ellipse. In fact that applies to *any* change - if you wanted to change the text you would also have to repaint the entire surface for each change.

Now what you *could* do in Quartz is create three seperate drawing surfaces, one for the rectangle, one for the ellipse, and one for the text, and get the compositor to combine them. Then if you want to adjust the ellipse's transform, it's no longer necessary to redraw everything. But you still have to reset and redraw the surface containing the ellipse.

That wouldn't be so bad, but this goes directly against what the developer guides recommend for Mac OS X. They say that surfaces are relatively heavyweight, particularly if you enable full composition for them. (It's actually recommended that if at all possible, you disable the compositor support for your drawing surfaces unless you really need it.) So this is a technique which can easily end up consuming a lot of resources in the display subsystem.

If you tried to use element-level composition for every single element in your UI, you'd grind the display system into the ground for anything but the most powerful systems. (Or the simplest UIs.)

Avalon is designed to allow you do just this however. And it's able to do so without killing perf because it uses vector-level retention rather than bitmap-level retention. This enables it to retain a much, much larger number of independently composable entities because it's a lot cheaper to hold the vectors than the bitmaps. (Retaining a 200x100 pixel rectangle in bitmap form requires about 80KB. It takes a lot less space to retain the description of the rectangle.)

Ultimately a certain amount of redrawing still goes on in Avalon. But there are two key advantages. (1) it's all done for you, simplifying the programming model. And (2), it's much faster: crucially, Avalon can decide when bitmap retention will be worth the extra memory, or when its more efficient for it to just redraw all your imagery for you. But even when it redraws, it doesn't need to go and hit your application - the display server can do all the recomposition itself. And it can even retain vectors as instruction sequences in the graphics card's memory. When it has chosen not to retain a bitmap, all it has to do to redraw the retained vectors is tell the GPU to go and execute those instructions, which will be a lot faster than having the application run its repaint code.

"Would the simple vector drawing example above be turned into a stream of low level drawing primitives and stored in memory to be replayed whenever a dirty region needs redrawn?"

It depends. As far as I've been able to tell from my experiments, sometimes it does this, and sometimes it does actually retain bitmaps as well. (It appears to retain bitmaps for complex bits of the display, for example. So if you have a page full of text, it appears to decide that it would take so long to redraw that every time, its worth allocating a bitmap to hold the image. So I'm assuming there are some heuristics it uses to decide whether to retain vectors and bitmaps, or just vectors.)

1 to 1 of 1
  1. macos x already works like this
    2004-03-11 13:40:55  rosyna [View]

    • Ian Griffiths photo macos x already works like this
      2004-03-12 00:28:19  Ian Griffiths | O'Reilly Author [View]

      • macos x already works like this
        2004-03-14 10:23:25  mweiher [View]

        • Ian Griffiths photo macos x already works like this
          2004-03-15 05:51:48  Ian Griffiths | O'Reilly Author [View]

          • macos x already works like this
            2004-04-06 06:17:25  musnat [View]

          • Ian Griffiths photo macos x already works like this
            2004-03-17 04:33:50  Ian Griffiths | O'Reilly Author [View]

          • macos x already works like this
            2004-03-15 13:58:01  mweiher [View]

            • macos x already works like this
              2004-04-06 06:16:16  musnat [View]

            • Ian Griffiths photo macos x already works like this
              2004-03-17 04:29:36  Ian Griffiths | O'Reilly Author [View]

            • Ian Griffiths photo macos x already works like this
              2004-03-16 04:43:14  Ian Griffiths | O'Reilly Author [View]

              • macos x already works like this
                2004-03-19 17:33:59  mweiher [View]

                • Ian Griffiths photo macos x already works like this
                  2004-03-21 11:57:47  Ian Griffiths | O'Reilly Author [View]

                  • macos x already works like this
                    2004-03-22 12:32:11  mweiher [View]

              • macos x already works like this
                2004-03-18 16:28:15  mweiher [View]

                • Ian Griffiths photo macos x already works like this
                  2004-03-21 14:39:30  Ian Griffiths | O'Reilly Author [View]

                  • macos x already works like this
                    2004-05-23 22:06:41  glenlow [View]

                  • macos x already works like this
                    2004-03-22 15:32:48  mweiher [View]

                • macos x already works like this
                  2004-03-19 20:59:37  rosyna [View]

                  • Ian Griffiths photo macos x already works like this
                    2004-03-21 12:50:03  Ian Griffiths | O'Reilly Author [View]

                    • macos x already works like this
                      2004-03-22 12:20:26  rosyna [View]

1 to 1 of 1