Why SDV Need Augmented Reality

interview with Tero Sarkkinen CEO, Basemark 

Curiosity about BMW I Vision DEE, one of the highlights of CES, led to an interview with Tero Sarkkinen, CEO of Basemark, who provided the 'AR on Video' application on BMW iX's center display. Despite the stupidity of asking Basemark, an AR party, about killer technology in the era of software-defined vehicle, he replied, "It's a merging ADAS with AR, AI and HMI." This was the concept of enhancing the reliability of ADAS not only in terms of customer experience, but also in terms of solutions that take into account all levels of the value chain, including processors.

Written by Sangmin Han_han@autoelectronics.co.kr

 한글로 보기 





Hello! As a HUD fan, I thought I should have met you a long time ago. How is the expertise in this field that started with Rightware connected to Basemark and AR? Please introduce it. 

Thank you, it’s great to meet you too! There is a great and a logical connection between Rightware and Basemark. I founded Rightware in 2009 and we became the leading provider of HMI design and deployment software for automotive industry, it’s all about being able to design and execute great looking user interfaces in various screens of the car. Then in 2015, I had an opportunity to buy out the Rightware processor performance benchmarking team. That’s what I did and hence Basemark was born.
Now, AR requires a whole lot more than just graphics. We started developing our engines for VR and AR graphics already back in 2016 and as time went on, we added a whole new engine, a compute engine, that can process all the computing required for computer vision, geospatial calculations and so on. So now we have the full AR stack that’s optimized for automotive processors.



What was your impression of CES 2023 and the keynotes of major car makers such as BMW? What does a self-driving, software-defined car mean for Basemark? Could AR HUD be the killer application of the SDV era? 

Well, when I saw the BMW announcement of the beautiful iVision Dee concept car where AR HUD plays a key role, I couldn’t be happier. It totally validated our business case. 
About autonomous drive (AD) We saw already a few years back that full autonomous driving will take substantially longer to achieve than the industry was touting. That’s why we focused our compute for fast development of ADAS functions rather than only for AD. Our tooling serves still as a good platform for AD development but it’s even better for ADAS. 
What we hear from our OEM customers is that currently, drivers don’t actively use ADAS functionality as they don’t trust the technology. AR visualization helps the driver to see what the car detects (lanes, cars, pedestrians, etc.) and thus creates more trust in ADAS technology. 
The killer application is to merge ADAS with AR and AI - that’s exactly what we’ve done. Recently, I posted news about our related work (see below) and received a tremendous amount of positive feedback. This gives us confidence that we’re at the right place at the right time.

What is the meaning of ‘AI’ emphasized by AR HUD solution companies? 

I think in general the AI comes from the fact that modern AR HUD has to have geospatial models in order for it to be able to render graphics so they seem to be placed at the right distance and hence correlate with real-world object. At Basemark, we’ve spent years developing AI models, algorithms and applications and optimizing those to run efficiently on automotive hardware. 



HUDs have long been seen as an important future for vehicle HMI strategies. I've been looking into the future with AR HUDs since 10 years ago. Although the first-generation AR HUD is now being applied to automobiles, AR HUD still remains as a secondary display for small vehicles. What is AR HUD now and why? 

We segment automotive AR into two groups: 1) AR HUD and 2) Video AR. The latter is easier for OEMs to implement since it uses the screens already existing in the car design: Instrument cluster and/or IVI display. In contrast, AR HUD is more difficult, as the state of today’s technology requires separate hardware: Projector, optics, reflective display or similar. Therefore, it’s more costly, it generates heat, it consumes energy so the driving range is smaller and it’s challenging to get the system automotive safety certified. 
However, there are new technologies coming that enable the whole windscreen to be the surface where one can render graphics. It will take some time before that tech will be ready for industrialization, meaning cost-wise, performance-wise and safety certifiable, but it’s coming.



When will we see the next generation of AR HUDs with a wider field of view, greater range, and more information including POI? What's left? 

At Basemark, we’re super excited about this future. It enables us to create so much more immersive, helpful and experiences for the driver and great entertainment for the passengers as well. In time, I forecast that the separate instrument cluster display will be totally replaced with a full windscreen HUD display.
As outlined above, it’s coming but will take probably some years before you can buy a car with it due to the long development cycles of the industry. 








Basemark applied the AR on Video solution in collaboration with BMW. What does this mean? 

We have been collaborating with BMW already for many years, working on AR topics and we’re very excited to see now vehicles shipping with AR that we helped to create. We hope to be able to publish some more great news going forward.



Why should OEMs, Tier 1s and semiconductor companies partner with Basemark? 

That’s actually a great question, because it contains companies from all levels of the value chain. We have a lot of customers from each of these levels and they’re choosing us because we gain invaluable information from each group and it enables us to design products that fullfil OEMs needs, offer greatly increased productivity for the Tier-1 developers and work efficiently on the various processors. There’s not many companies around that can simultaneously hit all these three targets.



Please introduce the company's core competencies, products, and strengths from AR engine, chip design, development, verification, and lifecycle maintenance. 

Our core competence is in a fast and efficient processing of compute and graphics within the embedded system. It might be a bit abstract way of saying it, but in practice it means that for the first time, you can use the one and the same ECU for both AR, AD, ADAS and HMI applications, all in a functionally safe way. Nowadays OEMs have to use two separate ECUs for these and it of course results in not only higher manufacturing costs, but also in higher energy consumption and higher latency, all which are clearly something that OEMs want to avoid.
Our product brand name is Rocksolid and it includes firstly, Rocksolid AR applications, e.g. AR Lane Departure Warning, secondly, of a Rocksolid AR Studio, which is the development studio and SDK for automotive software development teams, and thirdly, of Rocksolid middleware that acts much like a game engine but one that optimizes the processing of both compute and graphics. To our knowledge, no one else has something similar. This tool increases the development efficiency by at least a factor of ten compared to development done without these tools. 



  What's next for Basemark, both business and technical? 

Business-wise, we see continued growth. We have many OEM programs that have been started and that are starting and it’s certainly busy time everywhere in our organization. We’re probably going to grow to over 100 hard core technology experts this year, so we also will pause a bit and celebrate this great milestone. 
In technology, we are already working with some leading companies related to display technology and we continue to build our network as we see it as one of the key enabling technologies for us. I am so anxious to one day showing up to a customer with our demo car outfitted with full windscreen HUD display with a beautiful application designed by our tools. That moment will blow your socks out!



Do you have a special relationship with Korea? How is your business with Korea? Finally, tell us what you want to hear.

We do have customers in Korea and One of our talented engineers is from South Korea. He’s Seyoung Park and he’s actually the product owner of Rocksolid AR, so that’s a great connection with Korea. But I have also a special personal connection, I can eat Korean barbeque three times a day - it’s soo good! I’ve been visiting Korea more than 30 times and I always love to be back there. Hopefully soon enough again!

---