Researchers at OpenAI decided that a system that scores well at understanding language could too easily be manipulated for malicious intent.

The AI Text Generator That's Too Dangerous to Make Public

Halim

If I had to guess based on the notifications I get:

I often login to Facebook after weeks and I see bunch of notifications suggesting me to add women (every time somehow I only see Women in my notification Suggestions without any mutual connects whatsoever)

Other times, it’s about Events/Posts from people I have lost in touch with.

KPIs could be -Whether the Notification Type-N actually led to the user completing a predetermined action

In case of the Suggested friends, how many tines did the user clicked to check out the profile and based on that increase the frequency of these suggestions.

What KPIs / outcomes could THIS AI be keyed into optimizing?

Sounds like the bucket system FB used to regulate user requests generated by 3rd party developers on their platform back in the day.  I.e. request bucket allowance varies with engagement

From the paper:

“We introduced a new policy that uses Horizon to train a
Discrete-Action DQN model for sending push notifications
to address the problems above. The Markov Decision Process
(MDP) is based on a sequence of notification candidates
for a particular person. The actions here are sending
and dropping the notification, and the state describes a set
of features about the person and the notification candidate.
There are rewards for interactions and activity on Facebook,
with a penalty for sending the notification to control the
volume of notifications sent. The policy optimizes for the
long term value and is able to capture incremental effects
of sending the notification by comparing the Q-values of
the send and don’t send action.”

There’s this fictional story on SlateStarCodex about a startup working on generating controversial discussions on Reddit to influence political opinion:

https://slatestarcodex.com/2018/10/30/sort-by-controversial/

The writer goes in great detail about the underlying algorithms and the mechanics to make it sound like it’s absolutely possible to do. 

Also, someone posted a link to a Facebook research paper highlighting how one of their Notification Engine generates personalised notifications which can be borderline controversial: 

https://arxiv.org/abs/1811.00260

Halim

A group of likeminded people in TechCrunch Social are talking about this.