It seems very reasonable this will be a future feature. I've long suspected iPad OS' stage manager feature shipping so half baked was really more of getting the platform ready to support multiple apps and easier manipulation (from a developer perspective) of the double buffered "window" textures - given Vision Pro is based on iPad OS.
With Stage Manager on macOS now, it feels like they have all the primitives in place to "transpose" macOS stage manager windows textures to Vision OS/ the iPad OS foundation.
Though this will be tricky to get right for all apps. Will be interesting to see if it's a macOS App store only feature/ API, opt-in, or some other option
They can already do this with the desktop composition software they use today. All the windows are virtualized onto backing layers that you can draw anywhere and add effects to. It’s how window shadows work, and how certain window effects are done.
Apple will always prioritize critical scenarios over nice to have. None of these things are technically difficult, it’s just time. I’m willing to believe they released too early, but at some point you have to start learning from real users.
Sorry but almost 10 years ago I could do this on Xorg where the worst problem is that compositors cannot redirect input (so you had to kludge new events from scratch). I cannot imagine it would take more than _half an hour_ for someone with macOS display compositor experience to implement it.
It probably actually is a quick initial implementation, but like everything else, it needs to be planned, ticketed, developed, tested, and slated for release.
Compositor work isn't tremendously hard when the graphics primitives are already done.
UI state handling and input is a majority of the work from there. I've implemented this work before. A large portion of my background is in compositors and UI primitives.
What's really cool about having window backing layer handles, is you can do all sorts of crazy fun stuff in the office and show off to your coworkers or tech demo that basically will never make it to production because it's totally impractical.
The best example of that which actually did end up in a final product in my opinion was Windows Flip 3D. Totally fun implementation, I'm sure.
And the reason Xorg is now dying in favor of a system that doesn’t have this capability is because the architecture that enabled it, while cool at the time, severely limited the graphics performance and capability of the applications.
Exactly that. A local only path that provides high performance.
Remember the original comment was the claim that Xorg could easily do remote windows years ago. This is only true using the low performance path that modern operating systems have universally rejected.
No, the original comment is that by writing an Xorg compositor I could project separate windows into separate surfaces at arbitrary positions in the 3D world. There's no need for network transparency and you can very well do with an HDMI cable (actually, I was using an HDMI cable -- this was 10 years ago).
Network transparency is exactly what people want. If you're going to say just tether it to your computer with a wire then you just aren't understanding the product.
> Ensemble is currently "pre-alpha": it's really more of a demo at this point. There's a lot of things I need to work on and until then it is unlikely I will be taking any code contributions.... The code is definitely not designed for general-purpose use yet, so don't expect much of it :)
Some of the code is factored out into individual libraries. For example the networking and serialization code is separate and it is likely that screen recording will get pulled out too at some point.
Literally nobody has done it. It's beyond ridiculous that you can't already show or duplicate an application window on any display you want and allow it to be controlled from anywhere it is visible.
Searching for ways to do this lead one into extremely niche software ecosystems. Please is there any collaboration app out there that makes it seamless to toss windows around like everyone actually wants?
Isn't this how ordinary spanning monitors works? It might be slightly awkward with AR goggles since the relative orientation of the displays will be constantly changing as your head moves, and what happens to a window you have half on and half off of the Macbook's screen when you look away? Or do you want to have the application jump between devices, like appearing on your fridge when you go for a drink? With the old X11 protocol and a daemon in the middle this was possible but the use cases were extremely limited and the security issues made it a pain in the ass to actually use. With distros moving away from X11 this is only going to get harder, and you have to ask yourself how much you really want it.
This would mean the goggles would be basically just a dumb display for the Mac. It would also be weird to try to move an AR app onto your Mac.
Nearly 20 years ago there was an OS X screensaver that would capture all the window buffers and then float them around the screen as they rotated various ways. Another app would save a snapshot of all windows as Photoshop layers with appropriate transparency (before blur was added). X11 was always network transparent, though I don’t think windows could easily be moved from one client to another.
Even classic Mac OS, which wasn’t designed to be “rootless” could be run that way could have its windows mixed with OS X windows in the classic days.
As others have pointed out, there is already a PoC app that is doing it, so it seems if Apple wants to, it’s completely in their power. However, does this match their vision? (pun intended) Time will tell.
So Apple could have done this, but did not. Why? (Speculation and leaks welcome)
- [Profit on basic innovation] Did they want to wait and see how their customers would adopt VisionOS's native free-floating windows, so as to avoid cognitive overload by commingling with MacOS windows?
- [Benevolence to fellow competitors] Did they not want to takeover the existing market of virtualized VR desktops?
I'm reasonably certain it's a combination bandwidth and tech issue.
The Vision Pro is effectively using AirPlay to mirror the whole screen. If you used AirPlay to mirror each window as a whole screen, you'd run out of bandwidth pretty quickly.
The windowing system in MacOS, Quartz Compositor, also isn't built to stream window information. Right now it has a big built in assumption that any windows its displaying are on a screen it also controls. It was probably too big a lift across teams to also re-write the graphics stack for MacOS for the launch of the Vision Pro. Hopefully they get it working in the future, but neither of these problems are easy to solve.
They shipped iPad Stage Manager half-baked, to get iPad developers ready for double-buffered windows, so they could eventually ship the visionOS macOS integration half-baked? Doesn't sound right at all to my ears, even though I'm stoked for my order!
EDIT: -5* doesn't make sense, this is the most polite way you can point out that getting macOS apps windowed on visionOS has ~0 to do with double-buffered windows on iPad OS. n.b. I didn't use half-baked, OP did.
I don't think it's half-baked. I think it's lightly toasted. :-)
I use iPad Pro as a kind of sidecar daily driver, in the magnetic dock magic keyboard w/ trackpad.
As I type this, the screen shows a traditional MacOS style dock across bottom, four Stage Manager window clusters I can tap with a thumb on the left, and Safari plus Messages taking 2/3 and 1/2 of screen respectively.
There's more app and pixel real estate than most Windows laptops, and bringing screen sets to the foreground or swapping them back to the side is so natural I almost feel like giving up that space on my Mac as well.
The big thing I saw happen from apps over the past two version of iOS is app devs realizing their windows will not always be full screen or 1/2 screen size, but arbitrary size.
By now, most iPad apps of any serious nature are effectively window size independent, making them play well with others in stage manager. It's easy to see how that would make them play well with the headset one day.
No, I'm saying they shipped iPad stage manager half baked for their own uses/ to refine for AVP. I'm positing that a major reason for macOS stage manager's existence is as a transport layer/ "texture formatter"
With Stage Manager on macOS now, it feels like they have all the primitives in place to "transpose" macOS stage manager windows textures to Vision OS/ the iPad OS foundation.
Though this will be tricky to get right for all apps. Will be interesting to see if it's a macOS App store only feature/ API, opt-in, or some other option