Deep Learning with Apache Spark and TensorFlow - The Databricks Blog
Neural networks have seen spectacular progress during the last few years and they are now the state of the art in image recognition and automated translation. TensorFlow is a new framework released by Google for numerical computations and neural networks. In this blog post, we are going to demonstrate how to use TensorFlow and Spark together to train and apply deep learning models.
You might be wondering: what's Apache Spark's use here when most high-performance deep learning implementations are single-node only? To answer this question, we walk through two use cases and explain how you can use Spark and a cluster of machines to improve deep learning pipelines with TensorFlow:
- Hyperparameter Tuning: use Spark to find the best set of hyperparameters for neural network training, leading to 10X reduction in training time and 34% lower error rate.
- Deploying models at scale: use Spark to apply a trained neural network model on a large amount of data.
Read full article from Deep Learning with Apache Spark and TensorFlow - The Databricks Blog
No comments:
Post a Comment