Artificial Intelligence: A dangerous, potentially fatal misnomer
It is a categorical mistake to associate "intelligence" with a non-human entity. For that matter, disagreement is unlikely if one simply says that the very concept of intelligence Is derived from human experience and human behavior. The phrase "Artificial intelligence" (henceforth just AI) is therefore an oxymoron since it suggests the possibility that a non-human entity could have a trait that is fully the same as — in all respects, so identical with — a trait that is characteristic of human beings.
That misunderstands human nature.
All human behaviors and abilities are governed by the complex processes and interactions of mind and body, which express themselves in physical terms through electrical signals and physiological reactions. Human thinking is inseparable from human emotions. (Telling examples, even proofs of that, are the placebo and nocebo effects.)
It seems to me realistic, therefore, to worry that the hyped potential benefits of AI will be less influential than the damaging suppression of unorthodox minority views that have, in the past, correctly identified flaws in the majority consensus.
Most important for human beings and human societies are such things as love, loyalty, empathy; manners; ethics, morals, values. Those are governed by emotions mediated by such factors as knowledge, intelligence, cultural and parental and social environment and history. Nothing "artificial" could come to understand — to feel — those.
Using AI for anything except purely data-gathering and statistical analysis would be a recipe for disaster. Decisions about policies and actions need to remain at the mercy of human judgment.
Human beings are not perfect. There is no reason to imagine that human-designed machines or protocols can be perfect either; and they are likely to have unwanted corollaries that are recognized only when it is too late to prevent further harms.