I don't think AI will take over the world in the ways we expect, such as becoming sentient and intently carrying out malicious acts. These are scenarios that we can very much imagine and organizations are readily setting up AI Safety teams to prevent such cases of misalignment. In other words, I'm not worried about these scenarios because they are already on our radar when we talk about "Risks of AI."
What worries me is how AI will change our abilities to think critically and increase addiction problems more than ever.
Here's a run-down of my top concerns:
1. Decreased Attention Spans
The time to find an answer to a question has been greatly reduced by LLMs. This, of course, enables people to finish jobs faster than ever and it has greatly increased the "productivity" of my work as well. But when it comes to solving difficult problems, the time we take thinking deeply about how to solve the problem is as important as getting the answer right. And this process is taken away from us when we overuse LLMs in our lives.
Math olympiad problems are a great example of this. Take the example question below.
\(\text{What is the last two digits of the following expression? }\)
\(\ 1! + 2! + \cdots + 18! + 19! + 20!\)
Although the first instinct is to compute all of them and then add them up, the key is to find a pattern to the series of consecutive factorials.
Note: For those curious, the last two digits from 10! and onwards all become zeros. This is because there's a trailing zero for every pair of (2,5) when the number is factorized and every number from 10! and above has two or more pairs of (2,5).
Solving such problems trains us to not panic when we see difficult problems and instead to think critically about efficient approaches we could take. And, ironically, we come up with answers faster when we take the first couple minutes to just think about the problem without jumping into the equations or writing code right away.
This is indeed my grumpy grandpa moment who's waving his cane complaining about the world moving too fast, but I'm worried LLMs lower our bar on the expected time finding an answer to difficult problems.
Note: I'm sure people said similar stuff when we transitioned from encyclopedias to the internet. And if you ask me how this is any different, I honestly cannot give you a good answer why it feels different. Maybe it's because we can't judge the credibility of an LLM like how we judge credible sources on the internet? I'm still trying to figure this out.
2. Addictive Generated Content
Recommendation systems in Instagram, Facebook, and YouTube perform extremely well recommending content to us we love, to a concerningly-addictive extent. But I'm worried about a future when generative models become so fast and powerful that they generate addictive content for us.
They could be images, videos, or even custom games or worlds we can spend time in. I could be talking to someone who I loved but passed away, my favorite game character, or so much more.
I know there are pros to this and it is an awesome technology for sure. The more privileged ones in our world will benefit from increased entertainment and therapeutic applications. But I'm worried that the less privileged ones of our world may suffer from mass addiction and seek such technologies as a place to spend most of their time in. My definition of "less privileged ones" are people who are more prone to addiction whether the contributing factors may be wealth, surrounding environment, or social relations.
In other words, powerful generative models may create a greater disparity between ones who can protect themselves against addiction and those who cannot.
Sohl-Dickstein explained this concisely in his blogpost[1] when "Future AI will be able to dynamically generate the text, audio, and video stimuli which is predicted to be most compelling to me personally, based upon the record of my past online interactions."
Though I sound like an AI pessimist in this post, I'm actually quite the opposite. I'm excited to see how far AI will advance in my time along with how far science will advance with the help of AI. But to be prepared for such a world, we need to think of the un-thinkable harms that it may bring since they have the potential to rewire our brains in ways we might not want.
References
[1] "Brain dump on the diversity of AI risk" by Jascha Sohl-Dickstein.