All About DeepMind
DeepMind has built a neural network that can access an external memory like a conventional Turing machine. The result is a computer that mimics the short-term memory of the human brain.
source: MIT Technology Review
At the RE.WORK Deep Learning Summit in London last month, three research scientists from Google DeepMind, Koray Kavukcuoglu, Alex Graves and Sander Dieleman took to the stage to discuss classifying deep neural networks, Neural Turing Machines, reinforcement learning and more.
Google DeepMind aims to combine the best techniques from machine learning and systems neuroscience to build powerful general‑purpose learning algorithms. Formerly DeepMind Technologies, Google acquired the company in 2014, and now uses DeepMind algorithms to make its best-known products and services smarter than they were previously. Google's acquisition (rumoured to have cost $400 million) of the company marked the a peak in interest in deep learning that has been building rapidly in recent years.
DeepMind’s area of expertise is reinforcement learning, which involves telling computers to learn about the world from extremely limited feedback. As deep learning expert Yoshua Bengio explains: “Imagine if I only told you what grades you got on a test, but didn’t tell you why, or what the answers were - it’s a difficult problem to know how you could do better.”
However DeepMind has created software that can do just that. They hit headlines when they created an algorithm capable of learning games like Space Invader, where the only instructions the algorithm was given was to maximize the score. Within 30 minutes it was the best Space Invader player in the world, and to date DeepMind's algorithms can able to outperform humans in 31 different video games. This algorithm has been described as the "first significant rung of the ladder" towards proving such a system can work, and a significant step towards use in real-world applications.
We caught up with Koray Kavukcuoglu and Alex Graves after their presentations at the Deep Learning Summit to hear more about their work at Google DeepMind.
Can you explain your recent work in the Deep QNetwork algorithm?
Koray: The research goal behind Deep Q Networks (DQN) is to achieve a general purpose learning agent that can be trained, from raw pixel data to actions and not only for a specific problem or domain, but for wide range of tasks and problems. In order to tackle such a challenge, DQN combines the effectiveness of deep learning models on raw data streams with algorithms from reinforcement learning to train an agent end-to-end. We have developed novel components into the DQN agent to be able to achieve stable training of deep neural networks on a continuous stream of pixel data under very noisy and sparse reward signal. And more recently we have developed a massively parallel version of the DQN algorithm using distributed training to achieve even higher performance in much shorter amount of time.
What are the main areas of application for this progress?
K: DQN is a general algorithm that can be applied to many real world tasks where rather than a classification a long term sequential decision making is required. It is a very scalable RL method and we are in the process of applying it on very exciting problems inside Google such as user interactions and recommendations. In general, DQN like algorithms open many interesting possibilities where models with memory and long term decision making are important.
Can you explain your recent work in the neural Turing machines?
Alex: The basic idea of the neural Turing machine (NTM) was to combine the fuzzy pattern matching capabilities of neural networks with the algorithmic power of programmable computers. A neural network controller is given read/write access to a memory matrix of floating point numbers, allow it to store and iteratively modify data. As Turing showed, this is sufficient to implement any computable program, as long as you have enough runtime and memory. The key innovation is that all the memory interactions are differentiable, making it possible to optimise the complete system using gradient descent. By learning how to manipulate their memory, Neural Turing Machines can infer algorithms from input and output examples alone. In other words they can learn how to program themselves.
What are the main areas of application for this progress?
A: There has been a recent surge in the application of recurrent neural networks — particularly Long Short-Term Memory — to large-scale sequence learning problems. In areas such as speech recognition, language modelling, handwriting recognition and machine translation recurrent networks are already state-of-the-art, and other domains look set to follow. Neural Turing machines may bring advantages to such areas, but they also open the door to problems that require large and persistent memory. One such example would be question answering.
What are the key factors that have enabled recent advancements in deep learning?
K: Perhaps the biggest factor has been the huge increase of computational power. This has made it possible to train much larger and deeper architectures, yielding dramatic improvements in performance. More is more when it comes to neural networks. Another catalyst has been the availability of large labelled datasets for tasks such as speech recognition and image classification. At the same time our understanding of how neural networks function has deepened, leading to advances in architectures (rectified linear units, long short-term memory, stochastic latent units), optimisation (rmsProp, Adam, AdaGrad), and regularisation (dropout, variational inference, network compression).
What sectors are most likely to be affected by deep learning?
A: All industries where there is a large amount of data and would benefit from recognising and predicting patterns could be improved by Deep Learning.
What advancements excite you most in the field?
K: One of the most exciting developments of the last few years has been the introduction of practical network-guided attention. Attention models are now routinely used for tasks as diverse as object recognition, natural language processing and memory selection. Other areas we particularly like are variational autoencoders (especially sequential variants such as DRAW), sequence-to-sequence learning with recurrent networks, neural art, recurrent networks with improved or augmented memory, and stochastic variational inference for network training.
What developments can we expect to see in deep learning research in the next 5 years?
K & A: A lot will happen in the next five years. We expect both unsupervised learning and reinforcement learning to become more prominent. We also expect an increase in multimodal learning, and a stronger focus on learning that persists beyond individual datasets.
The next Deep Learning Summit is taking place in San Francisco on 28-29 January, alongside the Virtual Assistant Summit. For more information and to register, please visit the event website here.
For further discussions on deep learning, machine intelligence and more, join our group on Linkedin.
This interview was originally posted on the RE.WORK Blog.
source: https://www.linkedin.com/pulse/deep-minds-interview-googles-alex-graves-koray-sophie-curtis
At DeepMind, we conduct state-of-the-art research on a wide range of algorithms, from deep learning and reinforcement learning to systems neuroscience, towards the goal of building Artificial General Intelligence. A key factor in facilitating rapid progress is the software environment used for research. For nearly four years, the open source Torch7 machine learning library has served as our primary research platform, combining excellent flexibility with very fast runtime execution, enabling rapid prototyping. Our team has been proud to contribute to the open source project in capacities ranging from occasional bug fixes to being core maintainers of several crucial components.
With Google’s recent open source release of TensorFlow, we initiated a project to test its suitability for our research environment. Over the last six months, we have re-implemented more than a dozen different projects in TensorFlow to develop a deeper understanding of its potential use cases and the tradeoffs for research. Today we are excited to announce that DeepMind will start using TensorFlow for all our future research. We believe that TensorFlow will enable us to execute our ambitious research goals at much larger scale and an even faster pace, providing us with a unique opportunity to further accelerate our research programme.
As one of the core contributors of Torch7, I have had the pleasure of working closely with an excellent community of developers and researchers, and it has been amazing to see all the great work that has been built on top of the platform and the impact this has had on the field. Torch7 is currently being used by Facebook, Twitter, and many start-ups and academic labs as well as DeepMind, and I’m proud of the significant contribution it has made to a large community in both research and industry. Our transition to TensorFlow represents a new chapter, and I feel very excited about the prospect of DeepMind contributing heavily to another great open source machine learning platform that everyone can use to advance the state-of-the-art.
source: https://research.googleblog.com/2016/04/deepmind-moves-to-tensorflow.html
DeepMind has built a neural network that can access an external memory like a conventional Turing machine. The result is a computer that mimics the short-term memory of the human brain.
source: MIT Technology Review
At the RE.WORK Deep Learning Summit in London last month, three research scientists from Google DeepMind, Koray Kavukcuoglu, Alex Graves and Sander Dieleman took to the stage to discuss classifying deep neural networks, Neural Turing Machines, reinforcement learning and more.
Google DeepMind aims to combine the best techniques from machine learning and systems neuroscience to build powerful general‑purpose learning algorithms. Formerly DeepMind Technologies, Google acquired the company in 2014, and now uses DeepMind algorithms to make its best-known products and services smarter than they were previously. Google's acquisition (rumoured to have cost $400 million) of the company marked the a peak in interest in deep learning that has been building rapidly in recent years.
DeepMind’s area of expertise is reinforcement learning, which involves telling computers to learn about the world from extremely limited feedback. As deep learning expert Yoshua Bengio explains: “Imagine if I only told you what grades you got on a test, but didn’t tell you why, or what the answers were - it’s a difficult problem to know how you could do better.”
However DeepMind has created software that can do just that. They hit headlines when they created an algorithm capable of learning games like Space Invader, where the only instructions the algorithm was given was to maximize the score. Within 30 minutes it was the best Space Invader player in the world, and to date DeepMind's algorithms can able to outperform humans in 31 different video games. This algorithm has been described as the "first significant rung of the ladder" towards proving such a system can work, and a significant step towards use in real-world applications.
We caught up with Koray Kavukcuoglu and Alex Graves after their presentations at the Deep Learning Summit to hear more about their work at Google DeepMind.
Can you explain your recent work in the Deep QNetwork algorithm?
Koray: The research goal behind Deep Q Networks (DQN) is to achieve a general purpose learning agent that can be trained, from raw pixel data to actions and not only for a specific problem or domain, but for wide range of tasks and problems. In order to tackle such a challenge, DQN combines the effectiveness of deep learning models on raw data streams with algorithms from reinforcement learning to train an agent end-to-end. We have developed novel components into the DQN agent to be able to achieve stable training of deep neural networks on a continuous stream of pixel data under very noisy and sparse reward signal. And more recently we have developed a massively parallel version of the DQN algorithm using distributed training to achieve even higher performance in much shorter amount of time.
What are the main areas of application for this progress?
K: DQN is a general algorithm that can be applied to many real world tasks where rather than a classification a long term sequential decision making is required. It is a very scalable RL method and we are in the process of applying it on very exciting problems inside Google such as user interactions and recommendations. In general, DQN like algorithms open many interesting possibilities where models with memory and long term decision making are important.
Can you explain your recent work in the neural Turing machines?
Alex: The basic idea of the neural Turing machine (NTM) was to combine the fuzzy pattern matching capabilities of neural networks with the algorithmic power of programmable computers. A neural network controller is given read/write access to a memory matrix of floating point numbers, allow it to store and iteratively modify data. As Turing showed, this is sufficient to implement any computable program, as long as you have enough runtime and memory. The key innovation is that all the memory interactions are differentiable, making it possible to optimise the complete system using gradient descent. By learning how to manipulate their memory, Neural Turing Machines can infer algorithms from input and output examples alone. In other words they can learn how to program themselves.
What are the main areas of application for this progress?
A: There has been a recent surge in the application of recurrent neural networks — particularly Long Short-Term Memory — to large-scale sequence learning problems. In areas such as speech recognition, language modelling, handwriting recognition and machine translation recurrent networks are already state-of-the-art, and other domains look set to follow. Neural Turing machines may bring advantages to such areas, but they also open the door to problems that require large and persistent memory. One such example would be question answering.
What are the key factors that have enabled recent advancements in deep learning?
K: Perhaps the biggest factor has been the huge increase of computational power. This has made it possible to train much larger and deeper architectures, yielding dramatic improvements in performance. More is more when it comes to neural networks. Another catalyst has been the availability of large labelled datasets for tasks such as speech recognition and image classification. At the same time our understanding of how neural networks function has deepened, leading to advances in architectures (rectified linear units, long short-term memory, stochastic latent units), optimisation (rmsProp, Adam, AdaGrad), and regularisation (dropout, variational inference, network compression).
What sectors are most likely to be affected by deep learning?
A: All industries where there is a large amount of data and would benefit from recognising and predicting patterns could be improved by Deep Learning.
What advancements excite you most in the field?
K: One of the most exciting developments of the last few years has been the introduction of practical network-guided attention. Attention models are now routinely used for tasks as diverse as object recognition, natural language processing and memory selection. Other areas we particularly like are variational autoencoders (especially sequential variants such as DRAW), sequence-to-sequence learning with recurrent networks, neural art, recurrent networks with improved or augmented memory, and stochastic variational inference for network training.
What developments can we expect to see in deep learning research in the next 5 years?
K & A: A lot will happen in the next five years. We expect both unsupervised learning and reinforcement learning to become more prominent. We also expect an increase in multimodal learning, and a stronger focus on learning that persists beyond individual datasets.
The next Deep Learning Summit is taking place in San Francisco on 28-29 January, alongside the Virtual Assistant Summit. For more information and to register, please visit the event website here.
For further discussions on deep learning, machine intelligence and more, join our group on Linkedin.
This interview was originally posted on the RE.WORK Blog.
source: https://www.linkedin.com/pulse/deep-minds-interview-googles-alex-graves-koray-sophie-curtis
At DeepMind, we conduct state-of-the-art research on a wide range of algorithms, from deep learning and reinforcement learning to systems neuroscience, towards the goal of building Artificial General Intelligence. A key factor in facilitating rapid progress is the software environment used for research. For nearly four years, the open source Torch7 machine learning library has served as our primary research platform, combining excellent flexibility with very fast runtime execution, enabling rapid prototyping. Our team has been proud to contribute to the open source project in capacities ranging from occasional bug fixes to being core maintainers of several crucial components.
With Google’s recent open source release of TensorFlow, we initiated a project to test its suitability for our research environment. Over the last six months, we have re-implemented more than a dozen different projects in TensorFlow to develop a deeper understanding of its potential use cases and the tradeoffs for research. Today we are excited to announce that DeepMind will start using TensorFlow for all our future research. We believe that TensorFlow will enable us to execute our ambitious research goals at much larger scale and an even faster pace, providing us with a unique opportunity to further accelerate our research programme.
As one of the core contributors of Torch7, I have had the pleasure of working closely with an excellent community of developers and researchers, and it has been amazing to see all the great work that has been built on top of the platform and the impact this has had on the field. Torch7 is currently being used by Facebook, Twitter, and many start-ups and academic labs as well as DeepMind, and I’m proud of the significant contribution it has made to a large community in both research and industry. Our transition to TensorFlow represents a new chapter, and I feel very excited about the prospect of DeepMind contributing heavily to another great open source machine learning platform that everyone can use to advance the state-of-the-art.
source: https://research.googleblog.com/2016/04/deepmind-moves-to-tensorflow.html
I think things like this are really interesting. I absolutely love to find unique places like this. It really looks super creepy though!!
ReplyDeleteBest Machine Learning institute in Chennai | python machine learning course in chennai | Machine Learning course in chennai