news.bridge is dead, long live news.bridge!

,
hina_peggy_gmf

news.bridge team members Hina Imran and Peggy van der Kreeft overseeing on-the-fly subtitle creation at GMF 2019 in Bonn

Hello and welcome to the project wrap-up post – which actually isn’t a project wrap-up post. Funding by the Google DNI officially ran out on June 30th and the consortium is about to disband, but work on news.bridge will continue (hooray!). There’s a new team (more on that later), and a decision to polish up a platform that is already quite shiny.

Lots of happy users

Deutsche Welle editors have been testing news.bridge in various use cases for several months now. As of June 2019,  content in English, German, Hindi, Portuguese, Russian, and Spanish has been produced and evaluated. Test runs in Arabic, Farsi, Swahili, and Turkish are underway. External beta testers include a wide range of media companies from Euronews to 1&1/GMX.DE, but also international freelance journalists.

While news.bridge still has a couple of glitches and not all features have been implemented yet, the platform has been met with a lot of applause.  More conservative testers called it “very good and very useful”, feedback from enthusiasts sounded something like this: “Fantastic! Magic! When can I have this?”

news.bridge saves users a lot of time when re-versioning and adapting content for another language (or format).

HLT out and about

The fact that our consortium has been invited to dozens of language technology and innovation events also demonstrates the heightened interest in news.bridge. Recent event highlights include sessions, talks and presentations at the MESA Content Workflow Management Forum (London), ECIR (Cologne), MDN Workshop (Geneva), and, of course, our major live testing operation at GMF (Bonn).

2018 saw us speak at Subtech1 (Munich), the Workshop on Corpus Analysis of Time-Based Arts and Media (Berlin), a Google DNI Summit in Paris – and many other great events. In late November, we also partnered with the SUMMA project to host the Language Technology Hands-On Days in Bonn – another fruitful get-together that drew almost 100 participants.

New features, new services, new projects

news.bridge_screenshot_2019_06

Screenshot of the latest news.bridge user interface.

As news.bridge is growing internally and externally into what could become a proper SaaS platform, we’re working on redesigning the GUI, adding new functionalities (e.g. with regard to file import/export), and implementing new APIs: The EU’s MT service eTranslations is already available (albeit currently only to users at public institutions), and Microsoft has signaled their interest in adding some of the Azure Cognitive Services to our HLT mash-up solution.

In addition, news.bridge will be used as one of the output platforms for GoURMET, a major EU-funded project focusing on (better) machine translation for low-resource languages and domains. Hopefully, we’ll soon be able to offer you state-of-the-art MT for some very exotic language pairs.

Become a part of the news.bridge family

In case you were wondering if you can still join us as a beta tester: Yes, you can! Simply write to info@newsbridge.eu – and get a free, fully functional trial account. Furthermore, we’re always open to tech companies offering NLProc APIs (ASR, NMT, TTS). Our aim is to get as many high-quality services as possible under one umbrella.

Last, but not least, we’d like to thank everybody who has contributed to making news.bridge such a successful project. The past 18 months were a blast – and the future is wide open. HLT FTW!

DW uses news.bridge prototype for automated subtitling in day-to-day news operations

,

nb_lang_sub

Right from the start, project news.bridge has been about creating a language technology platform fit for daily production workflows at modern media organizations. Our goal is to provide a collection of transcription, translation, subtitling and voice-over services that reporters and editors are able to use hassle-free. In the last couple of months, we’ve taken another big step towards this goal: DW newsrooms have started to work with the news.bridge prototype.

Kudos for pioneering spirit go to both the Hindi and Portuguese language teams, who were the first to use our (still unfinished) platform for the automated production of subtitles for program items on an actual publishing schedule. Needless to say, there were a couple of hiccups, but all in all the subtitling process went really well, and the results speak for themselves.

DW Hindi used news.bridge to turn English audio into Hindi subtitles in a short web video on activists fighting against female genital mutilation in Guinea:

https://twitter.com/dw_hindi/status/1068876286259531778

nb_dw_hindi

DW Portuguese for Africa worked with news.bridge to create Portuguese subtitles from German audio for an episode of Euromaxx’s “Baking Bread”. The video focuses on pão de milho, Portugal’s famous cornbread:

https://www.dw.com/pt-002/baking-bread-o-que-nos-ensina-a-broa-de-milho-sobre-portugal/av-48283900

nb_dw_brasil

By now, a team of editors from different newsroom has started adapting long-form videos from German and English. A series of documentaries on celebrated art school Bauhaus will soon be published with Russian and Brasilian Portuguese subtitles, for instance. The Turkish, Indonesian and Swahili desks have also experimented with news.bridge.

Just a couple of days ago, the DW Brasil department announced they’re preparing the production of subtitles for no less than 12 (German-language) instalments of DW’s Reporter series. We think news.bridge will be just right for the heavy lifting in the translation process.

LIUM publishes new papers on named entity extraction and speech recognition

,

news.bridge partner LIUM has released two new academic papers on computation and language: End-to-end named entity extraction from speech and TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation. The papers were collaboratively written by Antoine Caubrière, Yannick Estève, Sahar Ghannay, Antoine Laurent, Natalia Tomashenko (University of Le Mans), François Hernandez and Vincent Nguyen (Ubiqus), and Emmanuel Morin (University of Nantes).

End-to-end named entity extraction from speech shows that it’s possible to recognize named entities in speech with a deep neural network that directly analyzes the audio signal. This new approach is an alternative to the classical pipeline which applies an automatic speech recognition (ASR) system first and subsequently analyzes the transcriptions. LIUM’s new method not only allows to simultaneously deal with speech recognition and entity recognition, it also makes it possible to obtain named entities only — and ignore the other words. The approach is interesting for at least two reasons:

  1. The system is easier to deploy (because you only need to set up a neural net).
  2. Performance will most likely be better (because the neural net is optimized for named entity extraction, whereas in the classical pipeline the different tools are not jointly optimized for the same task).

End-to-end named entity extraction from speech is available on arXiv under this link.

TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation is about describing (and providing) a new LIUM TED talk corpus as well as documenting experiments with it.

TED LIUM basically follows two aims: Train and improve acoustic models — and fix flawed TED talk subtitles while at it. Via its own ASR system, tailor-made for processing TED talks, LIUM creates new transcriptions of original audio. Subsequently, these transcriptions are compared to the old, often inferior subtitles. LIUM saves reliable segments, gets rid of unreliable material, applies some heuristics and finally provides the subtitles in file formats used by the international speech recognition community.

One of the (rather surprising) insights of the paper is that when transcribing oral presentations like TED talks, augmenting training data from 207 hours to 452 hours (+ 218.36%) doesn’t significantly affect state-of-the-art ASR system (i.e. Hidden Markov Models coupled with deep neural networks using a pipeline of different processes: speaker adaptation, acoustic decoding, language model rescoring). The word error rate (WER) dropped by merely 0.2%, from (an already low) 6.8% to 6.6%. The system seems to have reached a plateau.

However, training data augmentation absolutely benefits emergent ASR systems (fully neural end-to-end architecture, only one process, no speaker adaptation, no heavy language model rescoring). In this case, the same augmentation of training data led to a 4.8% drop in the WER. At 13.7%, it’s still rather high, but significantly lower than the 18.5% achieved by an Markov-based system at a comparable development stage in 2012.

Conclusion: Emergent fully neural ASR systems aren’t bad at all, very sensitive to training data augmentation, and can probably be exploited further. The big question in this context: How much data does it take to reach competitive results?

TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation is available on arXiv under this link and was also submitted to and accepted by SPECOM 2018.

Insights from our first user testing sessions

, ,

newsbridge_user_testing

Getting early input from the people you are designing for is absolutely essential – which is why we invited about a dozen colleagues to give the latest beta version of news.bridge a test run at the DW headquarters in Bonn last month. We had two really inspiring sessions with journalists, project managers and other media people working for DW and associated companies — and we have gained a number of useful insights. While some of them are too project-specific to share (news.bridge is not in public beta yet), there are also more general learnings that should make for an interesting journo tech blog post. Here we go:

Infrastructure and preparation

When inviting people to simultaneously stream and play around with news videos, make sure you have enough bandwidth. This may sound trivial, but it’s important, especially in Germany (which doesn’t even make the Top 20 when it comes to internet connection speed).

To document what your beta testers have to say as quickly and convenient as possible, we recommend to prepare digital questionnaires (e.g. Google Forms) and send out a link well before the end of the session. That way, you get solid feedback from everyone. It’s also a good idea to add a screenshot/comment feature (e.g. html2canvas) to the platform that is being tested. In addition, open discussions and interview-type interactions provide very useful feedback.

Testing automatic speech recognition (ASR) tools

Thanks to artificial neural networks, ASR services have become incredibly sophisticated in the last couple of years and deliver very decent results. Basically all of our test users said the technology will significantly speed up the tiresome transcription process when producing multilingual news videos.

However, ASR still has trouble when:

  • people speak with a heavy dialect and/or in incomplete sentences (like some European football coaches who shall not be named)
  • people speak simultaneously (which frequently happens at press conferences, for example)
  • complicated proper names occur (Aung San Suu Kyi, Hery Rajaonarimampianina)
  • homophones occur (merry, marry, Mary)
  • there is a lot of background noise (which is often interpreted as language and transcribed to gibberish)

As a result, journalists will almost certainly have to do thorough post-editing for a while and also correct (or add) punctuation, which is crucial for the subsequent translation.

Testing machine translation (MT) tools

What has been said about ASR also applies to MT: The tech has made huge leaps, but results are not perfect yet. Especially when you are a professional editor and thus have high standards. Something really important to remember:

The better and more structured your transcript (or uploaded original script),
the better the translation you end up with.

As for the limits of machine translation during our testrun, we found that “exotic” languages like Pashto (which is really important for international broadcasters like DW) are not implemented really well. Few services cover them, and the translation results are subpar. This is not a big surprise, of course, as the corpus used to train the algorithms is so much smaller than that of a major Western language like French or German. This also means that it is up to projects like news.bridge to improve MT services by feeding the algorithms high-quality content, e.g. articles from DW’s Pashto news site.

While MT tools are in general very useful when producing web videos — you need a lot of subtitling in the era of mobile social videos on muted phones — there are some workflows that are hard to improve or speed up. For example: How do you tap into digital information carriers that are an individually branded, hard-coded part of a video created in software like Adobe Premiere? Well, for now we can’t, but we’re working on solutions. In the meantime, running news.bridge in a fixed tab and copy-pasting your translated script bits is an acceptable workaround.

Testing speech synthesis

Sometimes, computer voices are indispensable. For example, when you’re really curious about this blogpost, but can’t read it because you’re on a bike or in a (traditional) car.

In news production however, artificial readers/presenters are merely a gimmick. At least for the time being. That’s because once your scripts are finished, reading/recording them isn’t that time consuming and will provide much nicer results. Besides, synthetic voices aren’t yet available in all languages (once again, Pashto is paragon).

Nevertheless, news.bridge beta testers told us that the voices work fairly well, and even sound pretty natural in some cases. They can be trained, by the way, which is an interesting exercise we will try out at some point.

HLT services and news production in a nutshell

If we had to sum up the assessment of our beta testers in just a few sentences, they would read something like this:

HLT services and tools are useful (or very useful) in news productions these days: They get you decent results and save you a lot of time.

news.bridge is a promising, easy-to-use mash-up platform, especially when it comes to transcribing and translating and creating subtitles (another relevant use case is gisting).

news.bridge is not about complete automation. It’s about supporting journalists and editors. It’s about making things easier.

Hello, world!

We’re live: Our website is up, our Twitter is up, and the platform itself (please register via email) is ready for upgrades, rewiring and in-depth testing.

DNI project news.bridge officially started in January 2018, and we’ll be working on it until the end of June 2019. Stay tuned for updates on new features, new partners, upcoming events, and dispatches from the world of human language technology (HLT).