Resources >‏ In the Media >‏ Latest News Details

Latest News Details

Reading a neural network’s mind

Publication Date:
12/12/2017
Category:
In the Media
Neural networks MIT.jpg
Neural networks, which learn to perform computational tasks by analyzing huge sets of training data, have been responsible for the most impressive recent advances in artificial intelligence, including speech-recognition and automatic-translation systems.

During training, however, a neural net continually adjusts its internal settings in ways that even its creators can’t interpret. Much recent work in computer science has focused on clever techniques for determining just how neural nets do what they do.

In several recent papers, researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and the Qatar Computing Research Institute have used a recently developed interpretive technique, which had been applied in other areas, to analyze neural networks trained to do machine translation and speech recognition.

They find empirical support for some common intuitions about how the networks probably work. For example, the systems seem to concentrate on lower-level tasks, such as sound recognition or part-of-speech recognition, before moving on to higher-level tasks, such as transcription or semantic interpretation.

But the researchers also find a surprising omission in the type of data the translation network considers, and they show that correcting that omission improves the network’s performance. The improvement is modest, but it points toward the possibility that analysis of neural networks could help improve the accuracy of artificial intelligence systems.

“In machine translation, historically, there was sort of a pyramid with different layers,” says Jim Glass, a CSAIL senior research scientist who worked on the project with Yonatan Belinkov, an MIT graduate student in electrical engineering and computer science. “At the lowest level there was the word, the surface forms, and the top of the pyramid was some kind of interlingual representation, and you’d have different layers where you were doing syntax, semantics. This was a very abstract notion, but the idea was the higher up you went in the pyramid, the easier it would be to translate to a new language, and then you’d go down again. So part of what Yonatan is doing is trying to figure out what aspects of this notion are being encoded in the network.”

The work on machine translation was presented recently in two papers at the International Joint Conference on Natural Language Processing. On one, Belinkov is first author, and Glass is senior author, and on the other, Belinkov is a co-author. On both, they’re joined by researchers from the Qatar Computing Research Institute (QCRI), including Lluís Màrquez, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and Stephan Vogel. Belinkov and Glass are sole authors on the paper analyzing speech recognition systems, which Belinkov presented at the Neural Information Processing Symposium last week.

To read more click here.



Follow Us

  • YouTube
  • Twitter
  • Facebook
  • RSS Feed
  • Linkedin
  • github-web.png
Back to Top

In the Media

Forbes fake news pic.jpg

Can AI Put An End To Fake News? Don't Be So Sure

07/10/2018

Fake news was the Collin’s word of the year for 2017 with good reason. In a year where politics-as-usual was torn apart at the seams, high-profile scandals rocked our faith in humanity and the ...

Read More

roadtracer.png

MIT/QCRI system uses machine learning to build road maps

22/04/2018

Map apps may have changed our world, but they still haven’t mapped all of it yet. Specifically, mapping roads can be difficult and tedious: even after taking aerial images, companies still have to ...

Read More

Economist story pic.JPG

Improving disaster response efforts through data

08/02/2018

Extreme weather events put the most vulnerable communities at high risk. How can data analytics strengthen early warning systems and and support relief efforts for communities in need? The size and ...

Read More

Events

Past Events

2018

Eman interns pic 2017.jpg

QCRI Summer Internship Program

Download ICS File 06/05/2018  - 05/07/2018 , Hamad Bin Khalifa Research Complex

Each year, Qatar Computing Research Institute organizes a summer internship program for undergraduate students studying computer science, computer engineering and other disciplines. The internship is unpaid, and QCRI does not provide any visa support.

Read More

Regina

Public Talk by Prof. Regina Barzilay "Artificial Intelligence for Oncology: Learning to Cure Cancer from Images and Text"

Download ICS File 27/03/2018 ,

Artificial Intelligence for Oncology: Learning to Cure Cancer from Images and Text A talk by Professor Regina Barzilay, MIT CSAIL Winner of 2017 MacArthur ‘genius grant’ At Education City Student ...

Read More

Slide1.JPG

QCRI & MIT-CSAIL Annual Project Review 2018

Download ICS File 27/03/2018 ,

Executive Overview Sessions Open to public Date:    Tuesday, March 27, 2018 Time:    9:00AM – 3:00PM Venue:  HBKU Research Complex Multipurpose Room To view full agenda, please click here . To RSVP, ...

Read More

News

Darb Al Saai QCRI 2017.JPG

QCRI to offer kids’ computing activities at this year’s Darb Al Saai

03/12/2018

Tech fun and robotics computing activities will be available to children attending the annual family celebration from December 12 to 20.

Read More

Sofiane Abbar in his office.jpg

Global experts in artificial intelligence for transportation to visit Qatar for TASMU-QCAI workshop

18/11/2018

Urban computing experts from Europe, the US and Qatar are to discuss state-of-the-art advances in artificial intelligence for transportation with local stakeholders.

Read More

Francisco Martin - Source_TelefonicaOpenFuture_.jpg

QCAI to Conduct Joint Machine Learning School with BigML

16/10/2018

Two-day crash course to provide hands-on introduction to machine learning for industry practitioners, developers, graduate students and undergraduates.

Read More