Triple ISPs: Concurrent Triple-Camera Usage

Interestingly enough, during yesterday’s keynote event, Qualcomm described themselves as a camera company, which is a funny way to see things, but actually somewhat makes sense given the large leaps in smartphone camera capture capabilities over the recent years.

The new Snapdragon 888 pushes the envelope in terms camera abilities by adding a whole new independent third ISP to the SoC, allowing the SoC to now run three independent camera modules concurrently, opening up new use cases for vendors and camera applications.

The new triple-ISP architecture now increases the overall pixel processing throughput by 35% to 2.7Gigapixels/s, allowing for concurrent usage of up to three 28MP sensors with zero shutter lag captures. Alternatively, you can use a combination of 64+25MP sensors with ZSL, or a single 84MP sensor with ZSL. There’s still also support for ultra-high-resolution sensors up to 200MP, but image captures here don’t support ZSL.

Allowing concurrent captures of three sensors now allows for the holy trifecta of ultra-wide-angle, wide-angle and telephoto modules to capture a scene at the same time, allowing for more interesting use-cases such as image stitching and image fusion to happen to seamlessly.

One interesting capability that Qualcomm was advertising is triple-stream 4K HDR video recording. That’s a bit of an odd-ball use-case as I do wonder about the practical benefits, but I do at least hope that the new triple ISP system allows for more seamless switching and zooming in and out between the various camera modules during video recording.

Video recording capabilities this year don’t seem to have changed, compared to the Snapdragon 865. This means 4K120 or 8K30 are still the peak capture modes, supporting also slow-motion of 720p960. Formats are also unchanged, with HEVC encoding in HDR formats such as HDR10+ or Dolby Vision being supported.

AV1 decoding didn’t make the cut this year unfortunately, which means wide-spread adoption in mobile for the codec will be delayed for another year.

While the video encoding formats haven’t changed, the image processing capabilities for HDR capture has. Thanks to the new ISPs and the raw performance throughput, the new Snapdragon 888 will be able to capture 4K HDR footage with the more advanced computational HDR processing being applied on each and every frame of the video.

Qualcomm claims that the new ISP in the Snapdragon 888 is the first to support new next-generation staggered HDR sensors.


Source: OmniVision

These are sensors that can have multiple rolling shutters, meaning sensor line readouts, active at the same time on the sensor. Instead of taking multiple exposures one at a time sequentially by scanning out the sensor matrix from start to finish, the sensor will start another exposure immediately after the completed line read-out, reducing the time in-between exposures greatly. This should allow for significant less motion ghosting between the exposures and a sharper resulting recombined HDR image capture than current generation sensors which only have a single active line readout on the sensor.

While we haven’t actually heard of such sensors from Samsung or Sony yet, Qualcomm is adamant that we’ll be seeing smartphones in 2021 employing this new technology.

A further improvement for still-picture captures is the advancement of the new multi-frame noise reduction engines inside of the ISPs. It’s said that the quality of the noise reduction has been improved this generation, allowing for even better low-light captures with the native capture mode (no computational photography).

Hexagon 780: A Whole new IP for AI & DSP; Adreno 660 GPU Conclusion & First Impressions
POST A COMMENT

123 Comments

View All Comments

  • powermacg5@ - Thursday, December 3, 2020 - link

    Hello everyone. Excuse my ignorance. Difference between 11 Apple A14 AI teraflops and 26 teraflops AI Snapdragon 888. Even the 865, seems to have more teraflops than the A13, but I can't quite understand the difference in normal use. It does not seem to me that the various terminals that use Snapdragon, make great use of AI. Reply
  • The Hardcard - Thursday, December 3, 2020 - link

    AI bragging usually involves integer operations as they work for inference and ALUs of all types can do more of them than floating points. So TOPS, not teraflops.

    I don’t know if the full comparison is available because Qualcomm’s figure includes every unit on the SOC without breaking down how many are from CPU, how many from GPU, and how many are Hexagon.

    Apple’s figure is just from the Neural Engine. They use their GPU, but I haven’t seen them give a public figure. The also brag about putting matrix accelerators in the CPU (big cores only?) but don’t get into how many TOPS the whole SOC can do. Maybe it’s in the developer docs?
    Reply
  • techconc - Thursday, December 3, 2020 - link

    It's important to note that these "TOPS" ratings are marketing driven and not an Apples to Apples comparison. Apple advertises the speed of their Neural Engine only (11 TOPS). Qualcomm's advertised TOPs number represents the theoretical capacity of ALL of their computing units including CPU, GPU, DSP, etc. Apple's number would be much higher as well if they included all of those other things not to mention their dedicated matrix multiplication units, etc. Reply
  • powermacg5@ - Thursday, December 3, 2020 - link

    Hello everyone. Excuse my ignorance. Difference between 11 Apple A14 AI teraflops and 26 teraflops AI Snapdragon 888. Even the 865, seems to have more teraflops than the A13, but I can't quite understand the difference in normal use. It does not seem to me that the various terminals that use Snapdragon, make great use of AI. Reply
  • iphonebestgamephone - Thursday, December 3, 2020 - link

    Better data gathering. Reply
  • Anymoore - Thursday, December 3, 2020 - link

    Qualcomm mentioned "most advanced 5nm", that's not Samsung, especially not LPEarly version of it. Reply
  • kwinz - Thursday, December 3, 2020 - link

    Thats disappointing. Especially no AV1 hardware decode. Reply
  • techconc - Thursday, December 3, 2020 - link

    "Another rationale for the foundry switch could be manufacturing capacity. As Apple is eating up a lot of TSMC’s early 5nm capacity with the A14 and M1, Qualcomm probably saw Samsung’s 5LPE as the safer choice this year..."

    This is the most likely scenario and justification to move to Samsung.
    Reply
  • techconc - Thursday, December 3, 2020 - link

    "One interesting capability that Qualcomm was advertising is triple-stream 4K HDR video recording. That’s a bit of an odd-ball use-case as I do wonder about the practical benefits..."

    Nothing oddball about it. Apple's A13 has this capability as well. There are pro apps like Filmic Pro which allow you to do things like capture a documentary type of interview on multiple cams at the same time. It may be something of an edge case, but it can be useful. It makes sense that Android phones start to get some parity on this type of feature.
    Reply
  • Plumplum - Saturday, December 5, 2020 - link

    Many things on this 888 are old...
    About this triple stream 4k HDR, not only Mediatek Dimensity 1000 has it, but 820 and 800 have it too...
    We can notice the 3 ISP too...avaiblable since...Helio P60...even Unisoc has soc with 3 ISP!
    If market wasn't ruled by Qualcomm, we would already have zooming capabilities across 3 cameras for a long time like Apple does.
    Still no AV1 hardware support...Qualcomm want us to pay licences for mpeg5!

    At least 888 is able to do 5g carrier aggregation. 865 wasn't...it means, until mmwave will be available in many years, in many countries, even a Dimensity 700 would be able to reach faster 5g speed than 865.

    Plus 5nm from Samsung, not that much better than TSMC's 7nm.
    TSMC's 7FF : 96.5Mtransistors/mm2
    TSMC's 7FFP : 113.9
    Samsung 5LPE : 126.5
    TSMC's N5 : 173.1
    Efficiency won't be as good on 5LPE!
    Reply

Log in

Don't have an account? Sign up now