AWS Machine Learning Blog

In the Research Spotlight: Zornitsa Kozareva

As AWS continues to support the Artificial Intelligence (AI) community with contributions to Apache MXNet and the release of Amazon Lex, Amazon Polly, and Amazon Rekognition managed services, we are also expanding our team of AI experts, who have one primary mission: To lower the barrier to AI for all AWS developers, making AI more accessible and easy to use. As Swami Sivasubramanian, VP of Machine Learning at AWS, succinctly stated, “We want to democratize AI.”

In our Research Spotlight series, I spend some time with these AI team members for in-depth conversations about their experiences and get a peek into what they’re working on at AWS.

Dr. Zornitsa Kozareva joined AWS in June, 2016, as the Manager of Applied Science for Deep Learning, focusing on natural language processing (NLP) and dialog applications. Zornitsa is a recipient of the John Atanasoff Award, which was given to her by the President of the Republic of Bulgaria in 2016 for her contributions and impact in science, education, and industry; the Yahoo! Labs Excellence Award in 2014; and the RANLP Young Researcher Award in 2011. You can read more about Dr. Kozareva on her website, or visit Google Scholar to find her 80 papers and 1464 citations.

Getting into the field of natural language processing

Zornitsa’s interest in the field of natural language processing dates back to 2003, when she was doing her undergraduate studies in computer science in her native Bulgaria. In her third year of undergrad, she applied to the Leonardo Da Vinci Program, which is funded by the European Commission. She was selected to conduct research on multilingual information retrieval at the New University of Lisbon, Portugal. “This was a really great experience. I learned how to build a search engine; how to innovate, write, and publish scientific papers; and, most importantly, how to share my findings with the rest of the research community. For an undergrad such as myself, this opened my eyes to a brand new horizon.”

From that moment, Zornitsa says that she was “mesmerized by machine learning and its ability to solve natural language problems. I became super passionate about the field and I decided that I wanted to pursue a PhD in NLP.”

In 2004, Zornitsa went to Spain for graduate studies, where she worked on “a wide spectrum of topics, including information extraction, semantics, and question answering. This is how my career in NLP started.”

While working toward her PhD, Zornitsa had the opportunity to do a full-year internship. “I picked the Information Sciences Institute, located in Los Angeles, because I wanted to work with world-renowned leaders in the NLP field, such as Dr. Eduard Hovy. For a year, I worked with Dr. Hovy and Dr. Ellen Riloff conducting research on knowledge extraction. It was a great learning experience, and I also received valuable career advice. Right after I graduated, I decided that I wanted to come back to the US and continue to enhance my scientific career.”

In 2009, she became a Research Scientist at the Information Sciences Institute (ISI). At ISI, she spearheaded multimillion-dollar research grants funded by the Defense Advanced Research Projects Agency (DARPA) and Intelligence Advanced Research Projects Activity (IARPA). The research focused on topics such as machine reading, which aims at teaching machines to read and understand text just like humans do; information extraction from unstructured documents on the Web; metaphor interpretation; and sentiment analysis.

Zornitsa compared working on grants to running a mini start-up. “You have to be good at pitching your idea, in order to get it funded. Then you need to figure out which types of people should be hired. Next, you have to meet the milestones that the funding entities expect. And, at the same time, continue to innovate and publish stellar research.” Zornitsa learned how to perfect and balance these skills from Dr. Eduard Hovy, Dr. Jerry Hobbs, and Dr. Kevin Knight. “We were all in the same institute and we worked together. I was very fortunate to have met them, worked with them, and learned from them. That is something for which I am very grateful.” In 2011, Dr. Kozareva joined the University of Southern California as a Research Assistant Professor.

Why did you move out of academia?

“Academia had been great for me. I learned a lot about writing grants; raising funding; operating, delivering, and publishing the research; and teaching. But I had a couple of pursuits. First, I wanted to learn how to build systems that can handle billions of data points. This is something that you can’t really do in academia–you don’t have that much data. You work on much smaller datasets. I was interested in how to build systems that work at a large scale. Second, in research, people often say, ‘Oh that’s an easy problem–that’s a solved problem.’ But when you move to industry, you see that it is much more challenging to make those systems work. Moving to industry allows me to solve ‘What does it take to make research come to life?’ And, most importantly, it allows me to see how the technologies I build could impact the lives of other people. I joined Yahoo, in 2014.”

At Yahoo, she worked on mobile search and product ads. “There were a lot of NLP challenges. For instance: How do you automatically detect that a particular query has a shopping intent? How do you extract the semantic information from the query so you can display the relevant information? I worked with various teams to understand the scope of this work and outline the tasks that needed to be executed.” It quickly became clear to senior leadership that Zornitsa had a knack for guiding people. She could certainly drive the technical aspects, but she could also organize people “to deliver results–which is very important to the business.” She started and grew a new group that focused on query understanding for mobile search and ads.

Why did you leave Yahoo?

“I believe that people should continue to evolve. When we built NLP applications (for Yahoo), we built them for a specific user segment. I was curious about how you build these systems for an even bigger customer segment. How do you build them for big corporations that might not have expertise in NLP and machine learning, but still want to solve these problems? AWS is a pioneer in the Cloud. AWS has this vision of doing the heavy lifting for customers so that they can focus on building what they are good at and the products they want. I decided that this was something I wanted to be part of, and that it was time for me to embark on a new adventure.” 

On Joining AWS

Alex Smola reached out to Zornitsa to lead NLP efforts at AWS. He was familiar with her work, and needed someone with research and practical experience in building NLP systems. She thought it was a good fit, and she liked the charter that Alex described. “I have the role of defining what the products should look like, thinking about the customers–understanding who would be using this product; thinking about how you build everything end-to-end–the whole stack from the scientific knowledge to what these machine learning systems should look like; evaluating them at scale; and caring about the quality of what you are producing.”

“We live in the era of artificial intelligence, where the goal is to build systems with humanlike capabilities. We see a lot of progress in self-driving cars and the Internet of Things, but at the core are the conversational assistants that enable us humans to communicate with machines. Until now, developers couldn’t build conversational systems, because they needed to understand a lot about NLP and speech recognition. You had to worry about scalability. How do you test what you built? How do you integrate it? One of the amazing services that we built here at AWS is Amazon Lex, which allows you to build conversational interfaces for your apps using voice and text. And this service is super easy to use–developers don’t have to worry about the infrastructure or the machine learning and NLP components. That is something that I’m passionate about, and I’m proud that we have built it. Now any developer, with or without machine learning or speech expertise, can build these kinds of applications.”

How do you like it at AWS?

“The mission we are on–building things that are used to help people who might not have the necessary expertise–is awesome. And that’s the future.”

When she’s not working on NLP challenges at AWS, Zornitsa enjoys playing beach volleyball and traveling to new places.

About the Author

Victoria Kouyoumjian is a Sr. Product Marketing Manager for the AWS AI portfolio of services which includes Amazon Lex, Amazon Polly, and Amazon Rekognition, as well the AWS marketing initiatives with Apache MXNet. She lives in Southern California on an avocado farm and can’t wait until AI can clone her.