If your new year’s resolution is to learn a foreign language, you have more options for doing so than ever before, from traditional lessons to home study options and mobile phone apps. But you might not have considered using a technique based on computer science and physics over grammar and vocabulary.
Emmanuil Kneller, general director of IstraSoft, pictured at the Skolkovo innovation centre. Photo: Sk.ru.
It is precisely this technique, however, that forms the basis of the language-learning programmes developed by IstraSoft, a resident company of the Skolkovo Foundation. And if it sounds like a fad, think again: IstraSoft’s programmes have been using voice recognition technology to teach foreign languages for a quarter of a century, and have proven to be a tried and tested method.
The project is the brainchild of Emmanuil Kneller, general director of IstraSoft. At the collapse of the Soviet Union, Kneller, a scientist with experience in making both software and hardware, turned his focus to what he calls speech technology.
“We started doing research into the field of speech: speech technology, recognition, and speech and the tongue, from a physical point of view rather than from a linguistic point of view,” he explains.
Programmes are available to learn English, Russian and German. Photo: IstraSoft.ru.
“We were looking for the physical parameters of the voice signal that create the sensation of the sound of speech. … For a computer to understand, it has to measure the physical properties of a signal, for example, the stress in a word – the time on that syllable is about twice as long as the others.”
Once Kneller and his partners had identified physical parameters, they set about creating computer programmes for language learning, and IstraSoft was born. The programmes allow students to speak into their computer’s microphone and then watch the computer illustrate how their pronunciation differs from that of a native speaker.
“Usually programmes teach you grammar, but they don’t teach you how to listen to speech and how to articulate properly, where to position the tongue to get the right sound,” says Kneller, who compares speech to music.
“Speech is an acoustic signal consisting of successive sounds. It’s a melody that has its own rhythm in each language that is determined by the articulation accepted in that particular language,” he says.
“In childhood, we hear our parents, and we learn the trajectories of those sounds. We don’t learn grammar in childhood, but we know straight away whether something is right or not, because our brain’s settings determine different groups and trajectories, and we’re used to them. When you find yourself in a place where another language is spoken, you have no system of differentiation, and it just sounds like noise. You can’t identify individual words or even sentences,” says Kneller.
A digital Professor Higgins
IstraSoft’s programmes accordingly work on the principle of first teaching the student to make the correct sounds, and then to correctly pronounce entire words and sentences. Named after Professor Higgins, the phonetics professor who bets that he can teach a Cockney flower girl to talk like a lady in George Bernard Shaw’s play “Pymalion,” the programmes promise to teach students to speak languages – English, Russian and German – without an accent.
“A teacher can hear when you are wrong but can’t always explain how to correct it,” says Kneller.
“We visualize the sounds. With words, the programme shows when the rhythm and sounds – and therefore the articulation – is wrong. So first you master the sounds, then the words.”
The programmes visualize the difference in pronunciation between the student's speech and the standard. Photo: IstraSoft.
Exactly the same principle and technology has been used by IstraSoft to produce a programme to help deaf people learn to speak.
Foreign-language students can also practice their language and pronunciation with tongue-twisters, poems and other texts available within the programmes, all read in impeccable Queen’s English by the late Bill Shephard, who for many years worked on the University of Cambridge’s English-teaching programmes.
IstraSoft's programmes can be used in conjunction with lessons, or as self-contained courses, says the company, a resident of the Skolkovo Foundation’s IT cluster since 2011.
One of the company’s Russian programmes has been used to teach Russian to first-year foreign students at the People’s Friendship University in Moscow for the last five years, having been tested at Moscow State University’s programme for Chinese students.
“They did an experiment: there was a group that only worked with the programme, and a group that never even saw it,” said Kneller.
“They came to the conclusion that those working with the programme got the hang of pronunciation and improved their listening far more quickly, and were also more motivated,” he said, adding that the format of the programme is familiar to and welcomed by young people who have grown up playing computer games.
IstraSoft has attracted financing from the Education Ministry and various foundations, and its programmes have been used in 40,000 Russian schools since 2002, as well as around the world via the Russky Mir foundation, which promotes Russian abroad.
Students can practice making the same sound in different words until they have perfect it. Photo: IstraSoft.
Staying innovative
When Kneller graduated as a Soviet scientist in 1966, computers were anything but commonplace. But by the collapse of the empire, he had already accumulated extensive experience in developing processors and personal computers at institutes. He had also worked for the U.S. tech giant Hewlett Packard for a long time, and was part of the team that created the first Russian fonts for laser printers.
In the quarter of a century since IstraSoft was set up, much has changed (though the retro design of the website and programmes reflects the company’s venerable age). The company’s latest language-learning programmes are available online, and versions for iPhone have been launched. What was once all stored on the computer has now been moved over to the cloud.
Nor have other technologies stood still in that fast-moving period, but Kneller insists his company still has no competitors offering exactly the same product.
“Recognition systems need a lot of data to work,” he says.
“Google has a lot of data, but systems on phones and so on need internet access to use Google in order to work. We have a slightly different principle,” he said.
This independence from the internet is the company’s innovative strength, says Olga Avryasova, a project manager specializing in robotics within Skolkovo’s IT cluster.
“Nearly all other voice recognition technology requires internet access, and that’s a huge disadvantage,” she told Sk.ru.
“With Istrasoft, all the data is included ‘on board’ the programmes, and that’s really great. It’s a solution that looks set to be really in demand.”
One market opportunity identified by Kneller is the introduction of the company’s programmes to European language-learning centres for immigrants across Europe, but he concedes this is generally more difficult than selling the system to private schools, as state organizations already have approved programmes and tend to be slower to introduce changes.
“Also, there’s a different mentality in Europe: they don’t associate physics and computer programmes with language learning,” he says.