Recently, Mark Zuckerberg stated his belief that the launch of Llama 3.1 will mark a pivotal moment in the tech industry, leading developers to favor open-source solutions. He expressed optimism about the potential of AI to benefit everyone globally and invited others to join in this endeavor.

Drawing from my own experiences, I’ve found that my RSS reader, Stratum, can be tailored to automatically summarize content from various sources, including YouTube, Hacker News, and blog posts. This process can be quite resource-intensive, often requiring prompts that range from 10,000 to over 100,000 characters.

In my quest to minimize costs, I’ve considered running a large language model (LLM) locally. After researching, I discovered a tool called Ollama that simplifies this process.

Ollama can be installed on personal computers, granting access to numerous open models, including Llama 3.1, Google’s Gemma 2, and Microsoft’s Phi 3, among others. Impressively, some users have managed to run Ollama on a Raspberry Pi.

I have contemplated deploying this on my Virtual Private Server (VPS), although it would necessitate a more expensive plan. My concern lies in the quality of these models; when utilizing APIs, companies often enhance the models through tuning and post-processing. After watching videos of various Ollama models, I noted some performance issues.

One common issue encountered with these models is the occurrence of broken words, a problem I have not experienced with APIs before. However, there is one model highlighted in a review that performs admirably: Llama 3.

Despite its strengths, Llama 3 is not without flaws. I tested it at meta.ai, and while it performed decently, it lagged behind models such as Claude 3.5 Sonnet and GPT-4o. For instance, when I inquired about the nesting of <code>&lt;item&gt;</code> tags in RSS parsing, both ChatGPT and Claude provided the correct answer, while Llama responded incorrectly.

This leads me to question Zuckerberg’s assertions. Perhaps Llama 3.1 will present improvements, as Meta claims significant advancements have been made, including a 50% enhancement in math capabilities and a 50% increase in GCPA, alongside a 100% improvement in Gorilla performance metrics.

Moreover, Meta has increased the maximum token count to 128,000, a significant jump from Llama 3's previous limit of 8,000 tokens. This increase expands the potential for handling larger text summarization tasks effectively.

There have been previous attempts to modify Llama 3 for greater token capacity, with reports suggesting capabilities of over a million tokens. Such modifications are possible due to the model’s ‘open’ nature—though not open source, its weights are publicly available for modification. This has led to the creation of variations like ‘Llama 3 Uncensored,’ which, while intriguing, may compromise quality.

The cost of operating Llama 3.1 is also a crucial consideration. With 405 billion parameters, each requiring storage space, Meta utilizes 8-bit quantization, translating to significant RAM requirements—roughly 386 GB, translating to substantial monthly rental costs for the necessary infrastructure.

Meta is also collaborating with various companies to develop a broader ecosystem around Llama 3. These partnerships aim to facilitate fine-tuning and model development services on major cloud platforms, fostering a community that could position Llama as a standard in AI.

In terms of pricing, the recently published GPT-3.5 Turbo offers a competitive cost structure, charging 15 cents per million input tokens and 60 cents per million output tokens, making it a more economical choice compared to models like Claude 3 Haiku and Gemini 1.5 Flash.

This raises a critical question: why should developers opt for Llama 3.1 when alternatives such as GPT-4o Mini are available at comparable or lower costs?

Zuckerberg argues for the necessity of an efficient and affordable model for developers, suggesting that Llama 3.1 enables inference on personal infrastructure at approximately half the cost of using closed models like GPT-4o. However, I remain skeptical about the assertion that Llama 3.1 will drive a shift towards open-source models.

Zuckerberg's analogy of comparing Open AI to Unix, suggesting that open models will foster innovation, does not resonate with me. Modifying Llama requires substantial technical knowledge and resources, possibly deterring widespread contributions to LLM development. The current landscape already features considerable openness, with companies like OpenAI and Google sharing extensive research.

The decision by Meta to make Llama accessible could stem from a perceived lack of competitiveness against other leading AI models or a strategic choice to avoid monetization.

Ultimately, I believe that while open-source code has its merits, it is not a panacea. The potential loss of innovation incentives through open models is a significant concern. I previously discussed this in a post about the GPL's impact on Linux development.

In contrast, I see merit in Google's approach—offering both closed and open models while ensuring research is publicly accessible.

Zuckerberg presents himself as a champion for open software, sharing his frustrations about restrictions imposed by platforms like Apple. He expresses a desire for more freedom in AI development, believing it will lead to superior services.

However, I doubt that Llama 3.1 will catalyze a shift toward open models in the near future. While it may be a capable model, the existing closed models are already sufficient and cost-effective, making it unlikely that Llama 3.1 will significantly alter the current dynamics in the industry.