How Spatial Monitors Work: The Four-Layer Pipeline Behind Glasses-Free 3D

The short version

A spatial monitor produces depth by doing four jobs every frame: generating two different eye views, splitting them optically so each eye sees its own view, tracking where the viewer actually is, and remapping the panel’s pixels in real time to keep the image aligned with the viewer. The quality of the 3D image is determined by how tightly those four jobs are integrated.

This article walks through each step, explains where the bottleneck really lives, and clarifies why two products with similar optics can feel completely different.

Why depth on a flat panel is not free

A normal 2D monitor shows the same pixels to both eyes. That gives the brain no disparity to interpret, so the image stays flat. To produce 3D, the panel has to send different pixels to the left and right eyes, and it has to do it consistently while the viewer moves. That requirement is what turns a screen into a controlled optical system.

The work behind a spatial monitor is therefore layered:

1. Image generation. Two views, one per eye.
2. Optical direction control. A physical layer that sends different pixels in different directions.
3. Viewer-position control. Knowing where the eyes are right now.
4. Real-time pixel remapping. Re-laying out the panel’s sub-pixels to match the geometry.

Step 1 — produce two distinct views

The first job is to prepare two images of the same scene: one rendered or photographed from a virtual camera slightly to the left, one from a virtual camera slightly to the right. The horizontal offset between those cameras is typically tuned to roughly the average human inter-pupillary distance, around 63 mm.

Where those views come from depends on the application:

• a stereoscopic video file or live 3D render in medical imaging,
• two camera streams in a 3D movie,
• a game engine rendering the scene twice per frame,
• a CAD viewer exporting two offset projections.

Both views are then packed into a single rectangular frame in side-by-side (SBS) format. On a 4K panel, that means a 3840 x 2160 frame with two 1920 x 2160 halves.

Step 2 — optically split the views

An SBS frame is still just a flat picture. To produce depth, the panel needs an optical structure that sends the left half of the SBS frame into one direction in space and the right half into another direction. The optical element is therefore a directional guide for light.

Every commercial glasses-free 3D display today uses a lenticular lens bonded to the panel — a sheet of precisely shaped cylindrical micro-lenses that refract sub-pixels into different angular channels. The lens itself is passive; it does not know where the viewer is or what the content shows. It is a static optical grating that turns a multi-view image into angular windows.

Some displays switch between 2D and 3D modes by toggling the lens effect on or off (active switchable grating). Others are 3D-only (static grating).

Step 3 — track the viewer’s eyes

Without tracking, a glasses-free 3D display has a narrow sweet spot: a small region of space where the viewer’s eyes happen to sit in the correct angular windows. Lean sideways and the image blurs, doubles, or inverts.

To widen the useful area, the display needs to know where the viewer actually is, frame by frame. That is done with eye-tracking hardware — typically one or more infrared cameras plus structured-light or pupil-detection software. The result is a live eye-position estimate.

Tracking rate matters. A tracker running at 30 Hz samples eye position every ~33 ms, which is slow enough that the image visibly lags head movement. A tracker running at 180 Hz samples every ~5.6 ms, which keeps up with normal head motion.

Step 4 — remap pixels in real time

The lens is fixed, but the panel pixels underneath it are not. As the viewer moves, the system has to rewrite the panel’s sub-pixel pattern so that the left-eye pixels fall into the angular window pointing at the left eye, and the right-eye pixels fall into the window pointing at the right eye.

This is the most timing-sensitive step. It depends on:

• the current eye position from the tracker,
• the precise geometry of the bonded lens,
• the panel’s refresh rate,
• the alignment between rendering and panel scanout.

How fast and how accurately this step runs is what separates a stable 3D image from a wobbly one.

Where the pipeline actually lives

All four steps have to happen somewhere. Two architectural patterns dominate the market today.

Host-driven pipeline

Sony ELF-SR2, Samsung Odyssey 3D G90XF, Acer SpatialLabs View Pro 27, and Lenovo ThinkVision 27 3D all run the pipeline on the host computer. The host GPU and CPU handle the eye-tracking service, the stereoscopic rendering, the sub-pixel rearrangement, and the synchronization with the panel. The display itself is mostly a passive panel plus lens plus camera.

That approach is the reason those products are Windows-only and require a certified NVIDIA discrete GPU — see why most glasses-free 3D displays cannot work on a Mac and why the host GPU requirement is so strict.

Display-side hardware pipeline

3DV Spatial Displays move the pipeline into the display itself. An integrated FPGA handles eye tracking, stereoscopic pixel routing, coordinate mapping, and sub-pixel rearrangement in the panel’s clock domain. The host’s only job is to deliver an SBS 3D video stream.

Because the host is not in the pipeline, the display works on Windows, macOS, and Linux without any vendor software, and the host GPU requirement drops to “any GPU that can output 3840 x 2160.”

Why latency controls how the display feels

Every frame, the panel shows the viewer a stereo image. If the image lags behind the viewer’s actual head position, the brain detects a mismatch between expected and actual disparity, which it interprets as instability. The deeper the lag, the more uncomfortable the experience becomes.

Latency accumulates from several sources:

• camera exposure and image transfer,
• eye-pose estimation,
• stereoscopic view calculation,
• sub-pixel remapping,
• panel scanout.

Each of those stages is a chance for the image to fall behind. The advantage of putting the pipeline on the FPGA is not that the algorithm is faster — it is that several of those stages disappear from the host round-trip entirely. The total path gets shorter, which is what the user actually feels.

Why eye tracking matters even when the user is still

Even an apparently motionless viewer is making small head movements: breathing, micro-adjustments, leaning to read text on the side of the screen. Without tracking, those movements push the user outside the sweet spot. With tracking running fast enough, the system follows the user, and the 3D image stays stable without conscious effort.

This is one of the reasons an FPGA-driven pipeline with 180 Hz tracking feels qualitatively different from a 30–60 Hz host-driven pipeline — the latter feels stable in a controlled demo and unstable in real-world use.

What it looks like in real workflows

A spatial monitor that does these four jobs well is genuinely useful for work, not just for demos:

• radiologists and surgeons can review 3D volumes while keeping their usual clinical tools visible,
• industrial inspectors can rotate and section CT data without losing spatial context,
• designers can review assemblies and surface continuity in 3D,
• researchers can present volumetric data to a colleague at a shared workstation.

A spatial monitor that does them poorly is the opposite: eye strain in twenty minutes, a sweet spot the size of a fist, and a host machine that has to be specified and maintained like a small server.

Common questions about how it works

Do spatial monitors use the same hardware as VR headsets?
No. The optics, the panel, the tracking, and the content pipeline are all different. A VR headset sits a panel a few centimeters from the eye; a spatial monitor sits the panel at normal viewing distance. The physics and the user experience are not interchangeable.

Is the lenticular lens what makes one product feel different from another?
Rarely. All modern glasses-free 3D displays use a lenticular lens. The user-experience differences come from the pipeline around the lens, not the lens itself — see the architectural comparison.

What is the practical role of the FPGA?
The FPGA keeps the stereoscopic pixel routing in the panel’s clock domain, so the alignment does not drift with host load. That is why an FPGA-driven display remains stable while a host-driven one drifts when the workstation is busy.

Can I use a spatial monitor as a normal 2D display?
Many can. Displays with a switchable grating can disable the lens and run as a standard 4K panel. Displays with a static grating are 3D-only.

Where to read next

• What is spatial display — the category overview.
• How SBS produces a 3D image — what the host actually has to render.
• Vergence–accommodation conflict — why fast tracking also matters for comfort.
• Lenticular lens vs. integrated FPGA — same optical element, different pipeline.