好奇目前先進駕駛輔助系統(Advanced Driver Assistance Systems；ADAS)領域供應商在深度學習的發展狀況為何。剛好看了一篇關於 Mobileye的報導，又把NVIDIA BB8的技術文章拿來翻了一下，有趣的比對。

Mobileye目前使用的solution是deep learning based 嗎？好奇。

NVIDIA BB8倒是展示了一件事，是不是可以利用end-to-end的作法，讓學習的網路由目前路面影像的輸入直接決定方向盤該轉幾度？這跟之前得先偵測出路面、分割線、道路邊緣的作法大大不同。這也暗示，對於其他深度學習的應用，也可以直接最佳化最後的標的，中間的特徵值或是決策，直接交給網路學習決定。

We do not work on ADAS (advanced driver assistance systems) solutions. However, we are curious about how they develop the solutions in the “deep learning” paradigm. As the outsiders, it’s hard to evaluate their competitiveness. However, it’s fun to evaluate the “relativeness”(but in an imprecise manner) from their PRs.

BTW, are Mobileye’s current deployed solutions deep-learning-based? Anyone knows?

Just saw the following news from MIT Technology review. It looks like the very similar goal from NVIDIA’s BB8, which takes raw camera pixels to determine steering commands during lane (or road) following.

Mobileye is to test their neural network system (augmented with reinforcement learning) in the second half of 2017, which might be 3-4 quarters late from NVIDIA’s demo, more a proof of concept (see the following). Definitely, there will be more advanced features such as lane merging in heavy traffics, risk management triggered by other drivers, etc. It’s hard to judge from the PRs.

NVIDIA’s BB8 is interesting and worthy to glance. It’s more like a proof or concept demo showing that we can skip the intermedia representations — recognizing the objects (e.g., lane markings, outline of the road, objects, trees,) on the roads — that many prior work relied on for steering a vehicle but focus on learning the proper turning radius to keep the car properly in lane. To train the end-to-end system (from images to steering radius, BB8 only uses a common convolutional neural networks and 72 hours of training videos (of three cameras, left, center, and right). No reinforcement learning is adopted yet as shown in their arXiv paper. There are definitely more fancy technical components they can adopt. I bet they are doing as well.

Note that BB8 only describes how to follow on lane and road. It does not evaluate on lane changes, turns from one road to another, or other risk handling tasks. The data augmentation process for teaching the networks to recover from a poor position or orientation by adding the (synthesized) shifts and rotations from the images is interesting. I bet the similar idea can benefit data augmentation for learning from sequences of data.

For BB8, I am referring to NVIDIA’s research work, Bojarski et al., “End to End Learning for Self-Driving Cars,” in arXiv, submitted on April 25, 2016.