Apple AR/VR chip exposed, TSMC production SoC has no AI engine, and the image sensor is extremely large
More details of Apple AR/VR head display custom chip are emerging.
According to The Information, Apple AR/VR completed the research and development of chips such as On-chip System (SoC) last year, which does not have a built-in AI acceleration module. Apple chips have stronger performance than third-party SoCs, including better wireless data transmission, compression, decompression and higher energy efficiency.
The report shows that SoC and two other Apple chips have reached what the semiconductor industry calls milestones. The physical design of the chip has been completed and is ready for trial production. According to people familiar with the matter, TSMC is producing these three chips, at least one year away from mass production.
This is a critical stage in the development of this headset. According to previous news, its headset will be released as early as next year, but Apple may further delay the release date.
1 SoC: No AI acceleration module, can communicate wirelessly with mobile phones
People familiar with the matter revealed the details of Apple AR/VR headset custom chip SoC.
It adopts TSMC’s 5nm process. Because Apple AR/VR headset is expected to be released as early as next year, this production technology may not be the most advanced at the time of release.
The SoC does not need the same small and powerful as the iPhone built-in SoC, which uses the most advanced technology of TSMC and most encapsulates on the chip, the person familiar with the matter.
Apple AR/VR headset custom chip is not as powerful as SoC on iPhone, iPad and Macbook, lacking artificial intelligence and machine learning capabilities, that is, there is no built-in Apple’s neural engine.
On the other hand, the chip can communicate wirelessly with the host device, which may be a mobile phone, computer or tablet computer.
In this way, the host can handle the more computing power required to display virtual reality (VR), mixed reality (MR) and augmented reality (AR) images.
The SoC has built-in CPU and GPU, which can run in a less powerful stand-alone mode, the person familiar with the matter said.
It is not uncommon to uninstall the computing power required by AR/VR headsets to other devices like this.
Existing VR headsets like Facebook Oculus Quest can run software in standalone mode, and users can also connect their headset to more powerful computers through wireless or wired connections to play higher-order games.
Magic Leap has also made an AR headset, which users can connect to a small and light package through cables to achieve image processing capabilities.
2 Apple AR/VR Image sensor and display driver IC have been developed
Apple’s chips are better than those produced by third-party general-purpose chips in some functions.
For example, Apple chips do better in transmitting wireless data between the headset and the host, compressing and decompressing video, and using power as efficiently as possible to maximize battery life.
If Apple wants to copy the resolution and details that users see in real life, while displaying digital images and information in front of users, these functions may be crucial to processing ultra-high-resolution videos of the external cameras of the headset.
Another person familiar with the matter said that Apple has completed the design of the image sensor and display driver of the chip.
Apple’s CMOS image sensor version is said to be unusually large, close to a head display lens, to capture high-resolution image data from the user’s surrounding environment for AR technology. The person familiar with the matter said that TSMC has been struggling to produce chips without defects, and the production period was very low during the trial production.
Display Driven Integrated Circuit (IC) is the interface between the chip and the headset. If no third party makes chips that can handle the ultra-high-resolution display of the headset, Apple may have to design display drivers from scratch.
An Apple spokesman did not comment on the above details.
3 Conclusion: Apple AR/VR has changed the direction of head display design again.
In 2018, Jony Ive, then Apple chief design officer, opposed plans to bind Apple’s headset devices to wireless hubs, Bloomberg reported last year. He eventually won the support of Apple CEO Cook. This also means that the performance of this device will not be so powerful.
However, Jony Ive later left Apple. In January this year, Bloomberg reported that Apple still intends to add some of the most advanced and powerful chips to the headset.
New details of custom chips bound to consoles such as iPhone indicate that Apple has once again changed the design direction of its headset. In July 2020, The Information reported that Apple had passed the prototype phase of the lenses to be used for the device, and that they were in trial production.
Earlier this year, Apple AR/VR device was revealed to have ultra-high-resolution displays and advanced eye tracking technology, and will be driven by an internally designed chip. Apple AR/VR plans to launch a more fashionable AR glasses after this device, which may be released as early as 2023.