I’ve mentioned in the past my excitement about cloud based machine learning, particularly the cognitive services offered by the likes of Google, Microsoft and IBM. Well, add Seattle-based Algorithmia to the fray. The company, which launched in 2013 and has been offering an online marketplace for algorithmic web services announced this week support for the Caffe, TensorFlow and Theano deep learning frameworks. With this new support in place, 3rd party developers can upload trained models to Algorithmia where they can be used by Algorithmia customers on a public or pay-per-call basis. The company seeded the market with 16 of their own open source implementations of popular research papers, including Google’s InceptionNet and Parsey McParseface based on Tensorflow, a real estate and illustration tagger based on Caffe, and an image stylizer based on the Theano-based ArtsyNetworks project for folks who want to build the next Prisma.

Incidentally, if you hear about all these deep learning frameworks and you want to know which one to learn, and you’re satisfied by making your choice based on popularity, you might find a recent tweet by Keras lead Francois Chollet interesting. He periodically ranks the major frameworks based on GitHub activity and tweets the results. If you want a summary, TensorFlow is leading in every area but GitHub issues. I’ll link to the tweet in the show notes.

Justin Johnson, a PhD student at Stanford posted a new github project to benchmark a bunch of different convolutional neural network architectures on different GPUs and the Intel Xeon, with and without CUDA. If you’re not living in the deep learning for image recognition world, it’s helpful to review the CNNs he looked at. The first is AlexNet, the 8-layer network described in the landmark 2012 paper by researchers Krizhevsky, Sutskever and Hinton, which ushered in the deep learning age after outperforming the field in the 2012 ImageNet competition. The next are various flavors of VGG, the model that won the ImageNet challenge in 2014 and described by Simonyan and Zisserman in their paper. And next he looks at ResNet, a model first proposed by Microsoft researchers in late 2015 and whose variants still represent the state of the art in image recognition. The ResNet paper has been on my reading list for a while and I may have mentioned it previously on the show. Really, if you want to really understand deep learning for image recognition, you could do a lot worse than just sitting down to understand and implement these three papers.

Anyway, Justin’s results were pretty interesting if not really unexpected:

  1. Across all CNN models, the GTX 1080 GPU is 10-15% faster than the Titan X.
  2. ResNet outperforms VGGboth in terms of speed and accuracy.
  3. Using CUDA via the cuDNN library is 2-3x faster than not.
  4. Using GPUs are 35-50x faster than dual Xeon processors.

Now, if you don’t happen to have a monster box at home or in the office but you still want to do deep learning on the cheap, you’ll want to take a look at Zohar Jackson’s AWS-SPOT-BOT project on GitHub. It automates finding, launching and configuring GPU spot instances on Amazon EC2.

A couple more quick notes:

  • If you’re a Python fan you certainly know the Jupyter Notebook. Well at the SciPy conference this week the Jupyter team presented its next evolution, called Jupyter Lab. I haven’t watched the video but the screenshot looks pretty slick, like a full in-browser IDE for data analysis.
  • And if you’re a Java user, sorry to hear that. I mean, there’s something for you too, in the recent release of version 4.0 of the DeepLearning4J framework. Headliner features are multi-GPU support for standalone and Spark, the addition of CuDNN support, and a bunch of new data structures, algorithms and native operations.