Itdaily - Zuckerberg writes AI manifest: 8 misleading claims and why they are wrong

Zuckerberg writes AI manifest: 8 misleading claims and why they are wrong

Mark Zuckerberg in a dark suit and striped tie, looking seriously ahead in a room.

Meta’s CEO shares in a very long blog post what the future of AI development should look like. It turns out that what is good for humanity happens to coincide with the company’s ambitions. Although, upon closer inspection, most claims are somewhat more nuanced.

In a lengthy blog post that takes on the airs of a manifest, Mark Zuckerberg shares what the future with AI should look like. The text contains positive ideas, including broad access to AI, but primarily serves to frame the company’s own agenda as a strategic plan for all of humanity.

Veiled within the text, we find eight of Meta’s agenda items disguised as plans for humanity. We list them here, but first note that the existence of the manifest itself testifies to an interesting self-image. Zuckerberg sees himself as the billionaire CEO of a company that has already been condemned several times for unethical practices, yet apparently feels positioned to explain in detail what role AI should play for the whole of humanity.

Before we delve deeper into the manifesto, it is important to note that Zuckerberg’s company has since been convicted in the US for intentionally addictive design choices in its products (and is under investigation in the EU), has been linked (by Amnesty) to a major outbreak of ethnic violence against the Rohingya in Myanmar, and is associated with various privacy breaches.

With that reality in mind, it is not surprising that the manifesto is primarily a plea for its own interests. We look at the most important claims.

1. Meta is good, the major competitors are worse

“Meta is the company primarily focused on building personal superintelligence for everyone,” it states. “Most other labs focus on building AI for companies, governments, or other institutions, so if those labs take the lead, the balance of power will shift in favor of larger institutions over individuals.”

In his text, Zuckerberg describes how Meta is actually the only player that wants to build AI for ordinary people. Other companies, de facto Anthropic and OpenAI, are said to be closed institutions and pose a risk to the balance between humans and AI in the future.

In this way, Zuckerberg portrays Meta as the major player standing up for the little guy, in contrast to the other parties. However, that claim has no foundation whatsoever. Other AI labs, including OpenAI and Anthropic, offer free versions of their solutions with fewer and fewer restrictions. Meta integrates its AI into its own products, such as WhatsApp and Facebook.

2. Delay equals risk

“Any policy that delays American model releases, even by a month, could add significant risk to American leadership while foreign models race ahead.”

The manifest states that there is hardly a greater risk than the delay of AI research. That would not cause more, but rather less safety. The reason: if American model builders, including Meta, are held back, devilish competitors from other countries will take over.

Here, Zuckerberg frames delay as an unacceptable risk. Any conversation about slowing regulation is thereby undermined. This, however, goes against a call from nearly 1,300 AI researchers, including some from Meta, who are practically begging the US government to take the lead in slowing measures. They believe that such a move is essential in the name of safety.

3. AI training must be open

“Some have tried to portray distillation as harmful, but I think it’s important to protect the principle that you can learn from anything you can observe.”

The CEO argues here for far-reaching openness in training. Model distillation is a technique where a powerful (closed) AI model is used to train new, smaller, and specialized AI models. These smaller models are then often more efficient at their task than the large teacher model.

Distillation is high on the agenda because Chinese companies are accused of using the technique without permission to create their own LLMs. They are said to secretly use closed models from Anthropic and OpenAI for this purpose. Allowing distillation means a blow to the potential profitability of the closed LLMs of those Meta competitors.

Furthermore, it would mean a major legal step toward general openness regarding AI training. Meta, like other AI developers, is under fire for using copyrighted data. More tolerance around training thus benefits Meta, but certainly not the owners of copyrighted products.

4. Data centers are positive and face unfair resistance

“We help keep electricity prices low by building our own energy-generating infrastructure wherever we invest. This ensures that not only do we not consume energy that could have gone to local communities, but in some cases, we even supply a surplus of cheap energy back to the communities.”

“Our data centers are also designed to be among the most water-efficient in the world. We are committed to being water-positive, which means that by 2030 we will restore more water than we use in the watersheds in which we operate.”

“Yet, one significant disadvantage that the US has compared to countries like China is that it is harder to build infrastructure here.”

Zuckerberg portrays data centers here as a positive story for the environment, without drawbacks. Resistance to data centers then stems at most from misunderstanding. Future plans regarding water use and energy generation are meant to reinforce that image, even if it does not align with today’s reality.

Aerial view of a Meta data center with large white buildings, cooling units and green lawns
Meta and Zuckerberg see their own data centers as forces for exclusively positive change, and even necessary investments to save the world from bad (Chinese) AI.

The manifest also links back here to the second point regarding delay. Too few data centers and too many rules regarding their construction could give the great enemy China a head start. In Zuckerberg’s vision, there is no reason to be hesitant with building permits for Meta data centers.

5. The government can watch (but not afterwards)

“I also suggest that frontier AI labs share intermediate training checkpoints of new models for government use and review, rather than waiting until training is complete.”

Again, Zuckerberg uses the argument of the risk of delay, refuted by hundreds of experts, to streamline his own plans. What sounds like a plan to allow control is actually a technique to prevent a thorough check of a new LLM just before launch.

The government may then watch during training, instead of performing an assessment afterwards. This allows Meta to continue launching LLMs at its own pace. There is no problem with intermediate checks, except that the Meta boss proposes them here as an alternative to thorough oversight afterwards, or at least a system that should speed up approval after training.

6. World’s biggest privacy bogeyman becomes guardian

“It will have strong privacy and security options… similar to how encryption works on WhatsApp.”

“We will build a fully private mode where even Meta cannot see or grant access to your information.”

Those are big words from a man who claimed a decade ago that privacy is no longer the norm. Evolving insights exist, but there is little reason to believe that the phenomenon has changed Zuckerberg’s vision. After all, Meta still screams and kicks at every privacy rule it has to comply with from the EU.

The link to WhatsApp’s security shows in itself that the plans for encryption are there, but it is important to note that WhatsApp uses encryption technology from Signal. That was already in the application when Meta bought it. It is not as if the company is a champion of end-to-end encryption.

Further in the manifest, it becomes clear how far Zuckerberg wants to stay away from the real privacy barricades:

“At the same time, we must also ensure that the government has access to the tools and resources it needs to enforce our laws and protect our safety.”

Knowing that this refers to the US government, which Zuckerberg already preemptively does not want to offend, it is clear that claims about privacy are only there to create goodwill.

7. New oversight model as an example

“I don’t think it’s in my, Meta’s, or the world’s best interest for me or anyone else to be the sole decider on how superintelligence is deployed. That’s why Meta is implementing a governance structure that gives our independent board of directors the authority to approve safety criteria for releasing models and to assess whether each model release meets those criteria. While Meta is a founder-controlled company, the CEOs of all frontier labs currently have extensive authority over model releases, so we encourage others to implement structures similar to this one.”

Meta’s oversight structure is a non-event. There is no guarantee of real independence for the board, and members certainly do not represent the population or humanity for which Meta claims to be building AI. Furthermore, Zuckerberg remains at the helm.

By calling on other AI labs to also implement such a system, Zuckerberg seems to be profiling Meta once again as a champion of good governance. At the same time, he proposes (limited) self-regulation as an adequate oversight mechanism. Historically, there is no reason to think that self-regulation in a publicly traded company, embroiled in a self-declared race, will provide sufficient oversight.

8. AI may improve itself, and preferably as quickly as possible

“Meta, other frontier labs, and clouds… must together build out a sufficiently large amount of compute so that we can allocate enough to recursive self-improvement to remain competitive.”

“Letting an AI system or anyone else drive its own goals is not inherently harmful in itself, even if its goals are not fully aligned with many people, as long as we maintain a balance of power that gives the advantage to humans.”

In his manifest, Zuckerberg writes that AI improving itself does not really pose an inherent problem. This also goes directly against the fears of more than a thousand AI experts.

He thinks everything will turn out fine and is apparently willing to bet the entire well-being of humanity on it, as long as multiple companies are working on self-improving AI at the same time. According to him, Meta and other (American) companies must then have access to sufficient computing power so that their self-improving AI is not overtaken by someone else’s (China).

In this way, Zuckerberg puts forward an additional argument to justify the massive expansion of compute.

A dangerous plea

Throughout the text, many considerations are presented as fact, even though they go against what many experts think. Furthermore, Zuckerberg takes on the role of predicting the challenges and possibilities of AI for all of humanity. In doing so, at no point does it really concern any region other than the US. Moreover, there is not a single point or argument that would demonstrate that Meta is not doing well. On the contrary: anyone reading the text would almost think that Meta is the only one that can save the world.