Meet Qualcomm's Snapdragon 855: AI boosts, a smarter camera, mobile gaming—and bye-bye, JPEG

A year ago, Qualcomm unveiled the Snapdragon 845, the brains behind flagship smartphones like the Google Pixel 3, the U.S. version of the Samsung Galaxy S9, OnePlus phones, and others. Now, Qualcomm’s next-generation Snapdragon 855 promises those platforms even more enhancements: dedicated logic blocks for digital assistants, revamped camera logic for computer vision, specific gaming boosts. It also gives the traditional JPEG file format the boot.

According to Qualcomm executives, the goal for the Snapdragon 855 is to “unlock” AI and XR (mixed reality), with the new 5G capabilities leading the way. The company claims that it’s offering the first commercial mobile platform to support this trifecta.

Qualcomm’s next-generation 855 is due to ship during the first half of 2019, meaning that phone makers will be able to design and announce their own Snapdragon 855-based phones for launch later in 2019.

Qualcomm’s Snapdragon chips are truly systems-on-a-chip (SoC), with an improved Adreno GPU and Kryo CPU, a Hexagon DSP that’s being repurposed for AI, and an increasingly more intelligent Spectra camera signal processor—often a key feature for phone buyers. Though each of the 855’s subsystems has been improved in its own right, Qualcomm also made one significant, overall improvement: While the Snapdragon 845 was manufactured on a 10nm process, Qualcomm has made the leap to 7nm with the Snapdragon 855.

Travis Lanier, Qualcomm’s senior director of product management, put it simply: The Snapdragon 855 will deliver 45 percent more performance than the 845 in the Kryo GPU, and 20 percent more performance in the Adreno GPU.

snapdragon 855 mobile platform chip compaison us coin 2 — Qualcomm hasn’t disclosed specifics such as the power consumption of the Snapdragon 855, but we know the chip is small.

Connectivity: Improvements beyond just 5G

Keith Kressin, senior vice president of product management at Qualcomm, suggested that every generation of wireless technology took a decade to develop. What’s 5G bringing? “Massive amounts of connectivity,” Kressin said, together with a new ecosystem of applications that no one quite understands quite yet.

“One question that comes up quite frequently: What is 5G going to do for me?” said Durga Malladi, the senior vice president for 4G and 5G for Qualcomm. For a network operator, it means that users will stream more high-bandwidth movies, with less latency. 5G also enables connected PCs, he said.

Snapdragon Tech Summit Snapdragon 855 — While 5G is revolutionary, you’ll probably see more benefit in the LTE enhancements.

More than 20 global operators and 20 global hardware makers are on board with 5G, Malladi said.

Though Qualcomm executives touted the advantages of 5G on Tuesday, cellular connectivity is just one portion of the Snapdragon 855, executives said. And for now, while the chip will work with 5G, the spec is not native to the platform: Though the Qualcomm X50 modem will perform 5G millimeter wave and operate in the sub-6GHz spectrum bands, the X50 isn’t integrated into the Snapdragon 855. (AT&T and Verizon appeared on stage Tuesday to support Qualcomm’s 5G initiative.)

Instead, it’s far more likely that phones will connect using what Qualcomm calls the world’s first 2Gbps LTE modem, the X24 modem, which Qualcomm announced in February and which is integrated into the Snapdragon 855.

But phones connect via Wi-Fi as well as cellular, and the Snapdragon 855 makes improvements here, too. Qualcomm is also characterizing the 855 as 802.11ax “ready,” also known as WiFi6. While today’s 802.11ac devices can push a maximum of 3Gbps, shared between several devices, 802.11ax will be able to support up to 14Gbps. It will also support 802.11ay, a 60GHz Wi-Fi solution, capable of 7Gbps, and believed to be most applicable for short-range, high-bandwidth apps like untethered VR. There’s Bluetooth 5.0, too.

Making cameras smarter with the Spectra ISP

Each time you point a Snapdragon-equipped smartphone at a subject and take a picture, the first core that image passes through is the Spectra image signal processor, or ISP. Within the Snapdragon 855, the new core is called the Spectra 850.

Traditional smartphones process color, white balance, exposure, and other optical characteristics to create an image. But as cameras have become smarter, they’ve begun recognizing objects, people, and other details of a scene. Until now, that’s required tapping into the CPU, GPU, and DSP for extra processing power. The result, has been features like portrait mode, which can intelligently recognize the subject of a picture and then blur the remaining background. Now, Snapdragon-equipped smartphone cameras will be able to do that for video, too.

What Qualcomm did was to recognize what portions of the CPU, GPU, and DSP the Snapdragon accessed, merged them with the color pipeline, and pulled the whole thing into a separate logic core. In the 855, Qualcomm created what it calls the first ISP optimized for computer vision.

That’s resulted in a “huge speed boost” in computer-vision applications as well as 4X reduction in power savings. Put another way, it will enable the 855 to perform all the traditional camera functions more quickly and at lower power, while enabling a new range of features.

From a performance standpoint, the Spectra ISP can support 22 megapixels at 30 frames per second using concurrent dual cameras; or 48MP at 30fps using a single camera. The 855 will do 4K, HDR10+ video capture in portrait mode at 60fps, and at 30 percent less power than the 845, said Judd Heape, senior director of product management at Qualcomm.

Even better, the Spectra 850 can now perform depth sensing at 60Hz, meaning that Snapdragon 855-equipped cameras will now be able to take the “portrait mode” of still images and apply it to video. Likewise, since Snapdragon 855 cameras can now distinguish and identify multiple objects, phones will be able to “pull out” the subject of a video and replace it with another background, in real time. There are even cinematograph capabilities, where part of the scene can be in motion.

All this, unfortunately, means a big change in the way Qualcomm Snapdragon phones store photo—the company is moving to the HEIF image format. Though JPEG has been the preferred file format for the last 20 years, it can’t store the complexity of HDR, computer vision, and so forth. HEIF can, storing everything from burst-mode photos to alpha masks, and even video, all within one “image” format.

snapdragon 855 heif — This is what HEIF will offer as a file format.

HEIF also acknowledges that more and more smartphones include multiple cameras—and with HEIF, you can store data from all of them. The goal? Shoot once, share everything.

Cinema Core: Saving power while playing back video

While all of those features are designed to create content, Qualcomm’s introducing a “Cinema Core” optimized for video playback. Cinema Core contains both H.264 and VP9 decoders in hardware, optimizing the video format used by YouTube. Again, the goal is to significantly reduce power while playing back video.

Hiren Bhinde, a director of product management, drew a distinction between the typical desktop PC monitor (1080p with just 8-bit color depths) versus what a smartphone now offers: 2K resolution, HDR, and much deeper color depths.

Cinema Core also supports what Qualcomm calls the next generation of HDR video, specifically HDR10+ playback, offering more dynamic range per frame then before. It will also offer 120fps playback, as well as 8K video playback of HDR video. From an audio perspective, Qualcomm’s introducing Qualcomm aptX Adaptive, a low-latency audio technology, as well as what it calls True Wireless Stereo Plus, for completely wire-free stereo audio.

Snapdragon Elite Gaming Platform: Boosting mobile gaming

When most people think of mobile gaming, they think of timewasters like Candy Crush. In Asia, however, mobile games are increasingly thought of as mobile counterparts to desktop games like Fortnite. Qualcomm is introducing what it calls the Snapdragon Elite Gaming Platform to satisfy that market.

Mobile games will generate over $70 billion in revenues in 2018 alone, said Qualcomm’s Hiren Bhinde. Over 586 million mobile gamers are in China alone—twice the population of the United States, he said.

Qualcomm’s putting a number of different features in its Elite Gaming Platform basket, but it all centers around the Adreno 640 GPU found within the Snapdragon 855, offering a 30 percent boost in performance over the Snapdragon 845. The Snapdragon 835 added HDR video playback; the 845 added HDR video capture. With the 855, the company is adding true HDR rendering gaming capabilities, supporting 10-bit color depths and the Rec 2020 color gamut. All told, over a billion colors are supported.

Qualcomm’s also banking on what it calls physical-based rendering—though it’s the surface textures that are being modeled on physical objects, such as stone. Small imperfections and porosity are modeled, with the idea that rendered objects will have a more lifelike look. About 30 different surfaces will be modeled.

Hexagon DSP: Making AI smarter

Although the Kryo GPU, Adreno GPU, and Hexagon DSP are all capable of running the complex math libraries that power AI, Qualcomm executives said that the Hexagon 690 has been literally rebuilt for AI. In fact, AI performance will be three times more powerful than on the Snapdragon 845, Qualcomm executives said, processing 7 trillion AI operations per second.

While Qualcomm added a pair of vector engines and a tensor engine as additional accelerators, a more interesting addition is a dedicated voice assistant core, specifically designed to power assistants like the Google Assistant. That core is designed to help assistants identify your voice. Google also said that in Android 9, the Android Neural Networks API is running entirely on the Hexagon 690, said PJ Jacobowitz, a senior marketing manager at Qualcomm. That core is designed to be always on, in a low-power mode, listening for a “wake word” like “OK, Google.”

Rajan Patel, senior director of engineering for Google Augmented Reality, said that the 855 would reduce latency by about three times. A high-bandwidth, low-latency 5G connection will make it easier to download large assets for augmented reality, he said.

The Hexagon is also used for real-time noise-reduction algorithms to cut background noise in video calls from airports or clubs, and for third-party apps that can even show you what you might look like with a different hairstyle.

qualcomm snapdragon 855 hexagon sharpness — Qualcomm’s Snapdragon 855 chip, and its Hexagon DSP, can power apps like this that can interpolate extrra detail in an image, executives said.

The Hexagon DSP will also be used to power what Qualcomm is calling a “3D Sonic Sensor,” an ultrasonic biometric sensor that can be mounted underneath a display to log in users via their fingerprints. Most smartphones use capacitive sensors, which detect the electrical impulses given off by a fingertip. Qualcomm believes that the way in which an ultrasonic sensor “paints” your fingertip’s whorls and lines with ultrasonic sound is the future, in that it works while wet and through contaminants like oil.

A less powerful Kryo CPU?

If a phone were a PC, the microprocessor inside of it would be front and center. However, the prevalence of apps, services, camera hardware, and other features tend to diminish the importance of the CPU hardware. Who buys a phone for its processor?

In part, that’s why Qualcomm doesn’t usually talk much about its Kryo CPUs. But the new Kryo 485 core includes something unusual: a “prime core.”

Typically, a Snapdragon chip includes four “performance” cores and four “efficiency” cores, the latter optimized for lower power. The Snapdragon 845 uses four ARM A75 cores at 2.8GHz and four A55 cores running at 1.8GHz. Qualcomm says the Kryo 485 within the Snapdragon 855 is 45 percent more powerful.

prime core specs — Here are the speeds of each of the Snapdragon 855’s Kryo cores.

But there are some interesting differences between the 845 and the 855. With the Snapdragon 855’s Kryo 485, there’s still the low-power “efficiency” cores, also running at 1.8GHz. But now there are three “performance” cores running at a slower 2.42GHz. But Qualcomm has added a new, even faster “prime” core running at a faster 2.84GHz.

That raised a few questions: Is the CPU running at diminished performance? If so, is that deliberate, perhaps because those tasks are being handed off to some of the other specialized logic blocks? Or is the CPU architecture simply more efficient? The latter answer turned out to be correct, Lanier said in a Q&A after the keynote: the Snapdragon 855 architecture is more powerful because of a higher instructions-per-clock efficiency, including larger caches.

Updated at 1:13 PM Hawaii time with additional details.

Note: Because Qualcomm would provide certain information only via an in-person briefing in Hawaii, with no alternative venue in the continental United States or by virtual means, we accepted the company’s offer to pay for my flight and hotel in order to get the story.