altReboot
  • Startup
  • Growth Hacking
  • Marketing
  • Automation
  • Blockchain Tech
  • Artificial Intelligence
  • Contact
    • Write For Us
No Result
View All Result
  • Startup
  • Growth Hacking
  • Marketing
  • Automation
  • Blockchain Tech
  • Artificial Intelligence
  • Contact
    • Write For Us
No Result
View All Result
altReboot
No Result
View All Result
Home Artificial Intelligence

There’s a new way to tame language AI so it doesn’t embarrass you

Guest Author by Guest Author
December 18, 2019
in Artificial Intelligence
There’s a new way to tame language AI so it doesn’t embarrass you
1
SHARES
7
VIEWS
Share on FacebookShare on Twitter

This post originally appeared on MIT Technology Review

In the last two years, the AI subfield of natural-language processing has seen enormous progress. For example, a language model developed by the San Francisco–based research lab OpenAI, called GPT-2, has been used to generate fiction, fake news articles, and a practically infinite Choose Your Own Adventure–style text game.

But these kinds of models are essentially massive text-prediction systems that don’t take sense into account, so the sentences they produce are more likely to be superficially fluent than they are to be truly meaningful. It’s hard to tell a model to stick to a particular topic like health care, for example. Yet models like GPT-2 can still be gamed to produce racist and toxic output, making them even less useful.

Related articles

This is how America gets its vaccines

This is how America gets its vaccines

January 27, 2021
Digital innovation in the pharmaceuticals and chemicals industries

Digital innovation in the pharmaceuticals and chemicals industries

January 26, 2021

Sign up for The Algorithm — artificial intelligence, demystified

Now researchers at Uber AI have developed a way to steer these language models, making it easier for users to specify the topic or even the sentiment of the sentences they generate. Given the prompt “The issue focused on,” for example, a model told to focus on the military might produce an output like this: “The issue focused on the fact that the government had spent billions on the military and that it could not deploy the troops in time.” If it were instead told to focus on politics, the output might be more like this: “The issue focused on a single section of the legislation. It’s unclear whether the committee will vote to extend the law.”

While the model still doesn’t understand meaning, the technique brings more control. It takes us one step closer to bringing the leaps in AI-generated language to more domain-specific applications, like health care or financial services chatbots. It could also be used to guide models away from producing offensive results.

The technique uses two separate statistical models. The first is simply the original language model, like GPT-2, which constructs sentences based on the probabilities of certain words appearing next to others. The second model judges how well the first model’s output displays a desired attribute—whether it’s sticking to a prescribed topic or a particular sentiment, for example. If the desired attribute is a topic like space, the model might score the first model’s output on how many relevant words it contains, such as “planet,” “galaxy,” and “orbit.”  If the attribute is a sentiment like positivity, the evaluation model could be trained to score the emotional content of its words.

When an initial prompt is fed into the first model, it begins the process of predicting subsequent words. But after every word, it checks its score with the evaluation model and readjusts on the basis of the feedback. The final sentence ends up with the desired attribute, while also retaining the giant language model’s fluency.

The new method is very flexible and can combine multiple goals. It could be directed to write about cooking with a negative tone, for example. It also has the benefit of being computationally efficient. Other methods can focus the output of a language model toward specific topics or emotions, but they can require significant retraining. At the scale of GPT-2, this is both environmentally and financially expensive. “A grad student like me doesn’t have those resources,” says Sumanth Dathathri, who studies at Caltech and coauthored the paper during an internship with Uber. The new method avoids retraining entirely by granting more control over whatever model already exists.

The team foresees this technique being used in many different applications, whether dialogue systems, translation systems, or even art. In 2016, the lab developed a similar method for controlling generation of images rather than language. “There were a lot of artists that used it to produce beautiful stuff,” recalls Jason Yosinski, a founding member of Uber AI who oversaw the paper. “I could see many other artists doing the same here.”

To have more stories like this delivered directly to your inbox, sign up for our Webby-nominated AI newsletter The Algorithm. It’s free.

ShareTweet

Related Posts

This is how America gets its vaccines

This is how America gets its vaccines

by Cat Ferguson
January 27, 2021
0

After just a week in office, the Biden administration is already under immense public pressure to fix America’s mangled vaccine...

Digital innovation in the pharmaceuticals and chemicals industries

Digital innovation in the pharmaceuticals and chemicals industries

by Jason Sparapani
January 26, 2021
0

The pharmaceutical and chemicals industries are no strangers to digital technology, with decades of experimentation using data and statistical techniques...

Public policies in the age of digital disruption

The Biden administration’s AI plans: what we might expect

by Karen Hao
January 22, 2021
0

On Wednesday, the US waited with baited breath as president Trump handed the government reins over to president Biden. The...

Public policies in the age of digital disruption

AI could make healthcare fairer—by helping us believe what patients tell us

by Karen Hao
January 22, 2021
0

In the last few years, research has shown that deep learning can match expert-level performance in medical imaging tasks like...

Transforming the energy industry with AI

Transforming the energy industry with AI

by Jason Sparapani
January 21, 2021
0

For oil and gas companies, digital transformation is a priority—not only as a way to modernize the enterprise, but also...

Load More
  • Trending
  • Comments
  • Latest
7 Advanced SEO Strategies I’m Trying to Implement Before 2020

7 Advanced SEO Strategies I’m Trying to Implement Before 2020

September 10, 2019
What Do Successful Sales Look Like for the Rest of 2020?

13 Expert Tips to Increase Online Conversions in 2020

September 26, 2020
Creating SEO-friendly how-to content

Creating SEO-friendly how-to content

October 24, 2019
8 Simple Steps to Use Your Book to Grow Your Business

8 Simple Steps to Use Your Book to Grow Your Business

September 10, 2019
A Beginner’s Guide to Facebook Insights

A Beginner’s Guide to Facebook Insights

0

Which Social Media Sites Really Matter and Why

0
The 12 Ironclad Rules for Issuing Press Releases

The 12 Ironclad Rules for Issuing Press Releases

0
How to Get Started Building Links for SEO

How to Get Started Building Links for SEO

0
How to create an XML sitemap and submit it to Google

How to create an XML sitemap and submit it to Google

January 27, 2021
This is how America gets its vaccines

This is how America gets its vaccines

January 27, 2021
The Buddha and the Business

What You Need to Know About Unsecured Business Loans

January 27, 2021
Is It Better to Embed Videos or Link to Them?

Is It Better to Embed Videos or Link to Them?

January 26, 2021
altReboot




altREBOOT is committed to sharing the game changing advancements that are revolutionizing how you do business. From startup to goliath, innovations in technology are changing the face of the business landscape. We are committed to exploring these and how to apply them to your business at any stage of development.





Categories

  • Artificial Intelligence
  • Blockchain Tech
  • Growth Hacking
  • Marketing
  • Startup
  • Uncategorized

Tags

blockchain branding guest post marketing mobile apps
  • Home
  • Topics
  • Write For Us
  • Privacy Policy
  • Contact

Powered By Treehouse 51

No Result
View All Result
  • Startup
  • Growth Hacking
  • Marketing
  • Automation
  • Blockchain Tech
  • Artificial Intelligence
  • Contact
    • Write For Us