Some philosophers and machine learning experts have speculated that superintelligent Artificial Intelligences (AIs), if and when they arrive on the scene, will wrestle away power from humans, with potentially catastrophic consequences. Dan Hendrycks has recently buttressed such worries by arguing that AI systems will undergo evolution by natural selection, which will endow them with instinctive drives for self-preservation, dominance and resource accumulation that are typical of evolved creatures. In this paper, we argue that this argument is not compelling as it stands. Evolutionary processes, as we point out, can be more or less Darwinian along a number of dimensions. Making use of Peter Godfrey-Smith’s framework of Darwinian spaces, we argue that the more evolution is top-down, directed and driven by intelligent agency, the less paradigmatically Darwinian it becomes. We then apply the concept of “domestication” to AI evolution, which, although theoretically satisfying the minimal definition of natural selection, is channeled through the minds of fore-sighted and intelligent agents, based on selection criteria desirable to them (which could be traits like docility, obedience and non-aggression). In the presence of such intelligent planning, it is not clear that selection of AIs, even selection in a competitive and ruthless market environment, will end up favoring “selfish” traits. In the end, however, we do agree with Hendrycks’ conditionally: If superintelligent AIs end up “going feral” and competing in a truly Darwinian fashion, reproducing autonomously and without human supervision, this could pose a grave danger to human societies.

一些哲学家和机器学习专家推测，超级智能人工智能（AI）一旦出现，就会夺走人类的权力，并可能带来灾难性的后果。丹·亨德里克斯（Dan Hendrycks）最近支持了这种担忧，他认为人工智能系统将通过自然选择进行进化，这将赋予它们自我保护、统治和资源积累的本能驱动力，这是进化生物的典型特征。在本文中，我们认为这一论点目前并不具有说服力。正如我们所指出的，进化过程在许多维度上或多或少都是达尔文主义的。利用彼得·戈弗雷-史密斯的达尔文空间框架，我们认为进化越是自上而下、由智能机构指导和驱动，它就越不具有达尔文主义范式。然后，我们将“驯化”的概念应用于人工智能进化，虽然理论上满足自然选择的最低定义，但它是通过有远见的智能代理的思想，基于他们想要的选择标准（可能是特征）例如温顺、服从和非攻击性）。在这种智能规划的存在下，我们尚不清楚人工智能的选择，即使是在竞争激烈、残酷的市场环境中进行的选择，最终是否会偏向“自私”的特征。然而，最终，我们确实有条件地同意亨德里克斯的观点：如果超级智能人工智能最终“变得野性”并以真正的达尔文式方式竞争，在没有人类监督的情况下自主繁殖，这可能会对人类社会构成严重危险。