Large portions of the human brain are dedicated to the perception and cognition of visual information. Any attempt to reproduce the workings of the eye is bound to result in unique challenges. Yet two tools help: signal processing algorithms inspired by neuroscience, and fast computing devices. RayShaper is working on putting these tools to use for future generations of digital vision applications.
The job of a computational vision system is to input an image in the form of pixels and output a code consisting of streams of bits. That’s what an encoder does. The design of encoders then depends on what kind of image is being processed and, more importantly, what is at the receiving end of the bit streams. Is a human enjoying a Netflix video, or is a machine monitoring scenes for detection of safety events?
A smart vision systems should look at a video before it compresses it. Specifically, a smart encoder should look at each frame in the video to see what it’s dealing with. Is the frame largely made up of one color? Is there text overlaid on the picture? How complex is the picture?
Further, a smart encoder should monitor and track the motion in the video. Is it a basketball game? Or is it a serene lake-view?
Human intelligence answers these kind of questions at lightning speed. As it’s impossible to include a human in every vision system, we need to embed artificial intelligence instead — a digital eye.
An AI-powered encoder equipped with digital eyes can discover the features of a video and use deductions to setup processing parameters. It can make smart choices about processing tradeoffs in real-time for highest compression and best possible quality.
Consider that on one hand we have standards like JPEG and HEVC with broad applications, and on the other hand we have specific use cases, commercial applications. Standards specify large sets of parameters that encoders must select and convey to decoders. Standards don’t tell application developers what methods or paradigms to use for the selection of encoder parameters. Here’s where AI can play key roles. RayShaper