Scientists Develop Camera System that Captures ‘5D’ Images

Jul 14, 2021

Jaron Schneider

An international team of scientists has developed an experimental camera system that can simultaneously capture five dimensions of information from a single snapshot. The concept works by combining two types of sensors together to gather a wealth of information all at the same time.

The researchers say that in order to better observe and understand unrepeatable or irreversible dynamic scenes, it is critical for ultra-fast optical imaging to be able to capture the structure of an object, its temporal evolution, and its spectral composition. Those who developed the new system came up against a wall though, as prior to their development no camera was able to simultaneously capture spatial, temporal, and spectral five-dimensional information of dynamic scenes.

“To break the limitation of the existing techniques in imaging dimensions, we develop a spectral-volumetric compressed ultrafast photography (SV-CUP) technique,” the researchers say in the abstract of their research paper.

SV-CUP’s configuration and principle. (a) System configuration of SV-CUP: M1 and M2, mirrors; ED, engineered diffuser; DS, dynamic scene; CL, camera lens; BS1 and BS2, beam splitters (reflection/transmission: 50/50); F1 and F2, filters; G, diffraction grating; L1 and L2, lenses; DMD, digital micromirror device; CMOS, complementary metal-oxide semiconductor camera; and SC1 and SC2, streak cameras. (b) Working principle of SV-CUP: C, spatial encoding operator; T, temporal shearing operator; K, spatial-temporal integration operator; S, spectral shearing operator; and M, spatial–temporal–spectral integration operator.

Boiling down the team’s research into easily understandable language is not particularly easy. But, basically, the researchers paired a super fast time-of-flight “streak” camera (referred to as a CUP or Compressed UltraFast Photography) with a CUP camera that captures wavelength (the color being emitted by an object) rather than the z dimension. By combining these things, the team was able to capture three dimensions in space (x, y, z) plus the time dimension (t) plus wavelength/color of light dimension (lambda) all in a single snapshot.

The 3D (x,y,z) information in ToF-CUP is coupled to the 4D (x,y,t,λ) information in HCUP and forms 5D (x,y,z,t,λ) information by image processing.

The SV-CUP camera appears to be an evolution of the one PetaPixel’s DL Cade covered in May of 2020 which is able to shoot 70 trillion frames per second. In this case, the researchers combine two CUP cameras, working in unison, to capture space, time, and wavelength information and merge the two outputs into a final 5D output.

“Our SV-CUP brings unprecedented detection capabilities to dynamic scenes, which has important application prospects in fundamental research and applied science,” the team writes.

As noted by Spie, systems like this one with the ability to record dynamic scenes on a picosecond (one trillionth of a second) — or even a femtosecond (a quadrillionth of a second) — timescales gives physicists, chemists, and biologists a way to get ultrafast multidimensional optical images that can be used to detect ultrafast phenomena. While this particular system can’t record quite that fast — it appears to be working at around two picoseconds — the number of dimensions it can capture at the same time at that speed is likely to be hugely helpful to certain scientists.

SV-CUP’s 5D imaging: (a) experimental arrangement for imaging the photoluminescent dynamics of a 3D mannequin coated with CdSe quantum dots; (b) reconstructed data cube of the 3D mannequin; (c) selected reconstructed images of the 3D mannequin at some representative times and wavelengths; (d) photoluminescent dynamics calculated from (c) (blue line) and measured by a streak camera (red line); (e) fluorescence spectrum calculated from (c) (blue line) and measured by a spectrometer (red line); (f) time-resolved spectroscopy extracted from (c); and (g) calculated fluorescence lifetimes at some selected spectral components.

The full paper titled “Single-shot spectral-volumetric compressed ultrafast photography” can be read at Spied Digital Library.

Image credits: Header image depicts a snapshot of five-dimensional imaging with temporal-spatial-spectral resolutions. | Credit: S. Zhang, East China Normal University.