We have been working on content analysis (machine intelligence over large-scale image/video streams) since 2002 as having my PhD in Columbia Univ. Yes, in the old days, we used no deep learning but hand-crafted features. It’s so tough for dealing with such “beast” signals.
Even in the old days, we were working on “large-scale” concepts (e.g., LSCOM [1]), which was huge at that time (2004) but only had 449 concepts (classifiers). We delivered one of the leading video research and detection system in NIST TRECVID benchmarks (e.g., [2]) over the “largest” dataset ( ~250 hours, at that time).
It’s so exciting to see the huge improvement in the past years, especially, triggered by the availability of quality training data, huge computation resources, and the advent of machine learning techniques.
These days, during my sabbatical visit to IBM Watson Research, I am so excited to witness and contribute the “beauty” — the things we had strived for for years have already turned live in life (e.g., the first-ever AI movie trailer [3], detecting the live golf TV streams for the highlights [4]).
We did learn a lot in this winding trajectory from the mistakes, the setbacks, and the excitements. However, I do feel lucky and grateful that we believe in and continue with what we have been doing. I can project that there will be much more exciting moments to come!!!
[1] M. Naphade, J. R. Smith, J. Tesic, S.-F. Chang, W. Hsu, L. Kennedy, A. Hauptmann, J. Curtis, “Large-Scale Concept Ontology for Multimedia, ” IEEE Multimedia Magazine, 13(3), 2006.
[2] Shih-Fu Chang, Wei Jiang, Winston Hsu, Lyndon Kennedy, Dong Xu, Akira Yanagawa, Eric Zavesky, “Columbia University TRECVID-2006 Video Search and High-Level Feature Extraction,” in NIST TRECVID workshop, Gaithersburg, MD, Nov. 2006.
[3] “IBM Watson creates the first AI-made film trailer – and it’s incredibly creepy,” http://www.wired.co.uk/article/ibm-watson-ai-film-trailer
[4] https://twitter.com/IBMSports/status/851256590816735232
Get loud! @TheSergioGarcia's clinching putt at #TheMasters registered a 7.9 on @IBMWatson's #Cognitive Highlight meter. #IBMsports pic.twitter.com/48leTacoP0
— IBM Sports (@IBMSports) April 10, 2017