A Brief Overview of NLG Commercialization

In 2010, I collaborated with sports data entrepreneur Robbie Allen and a small team of developers to create a platform that generated human-like content from extensive data sets, primarily in sports. At that time, we were unaware of the term NLG; we were simply focused on enhancing sports data. This endeavor led to the formation of Automated Insights, which was later acquired by private equity in 2015.

I developed algorithms to analyze data and determine the appropriate narratives to convey, paying close attention to patterns, trends, and significant milestones. Within the realm of sports and eventually in sectors like finance, marketing, and healthcare, my algorithms produced remarkable and engaging content. Thanks to Robbie and the programming team, we were able to generate thousands of unique articles per second, each resembling the work of a skilled professional.

In 2011 and 2012, we shifted our focus from sports to a wider range of industries, creating content in areas where human writers were scarce. For instance, we produced millions of fantasy matchup recaps for Yahoo Fantasy Football and hundreds of quarterly earnings report articles for the Associated Press, some of which were enhanced by human editors.

These initiatives continue to thrive today.

Were We Displacing Writers?

Automated Insights received venture capital backing and expanded rapidly. As we showcased our technology, we frequently faced questions from journalists about whether we intended to replace their roles.

The straightforward answer was a resounding no. Our aim was not to automate journalism but to automate data analysis.

This question became so common that we eventually added a ticker on our website:

"Number of Journalists Automated Insights Has Replaced: 0"

The joke was that it never changed.

I consistently battled the misconception that what we were doing was akin to writing. Similarly, ChatGPT, at this stage, is not writing but rather synthesizing and analyzing data to produce narratives—albeit ones that are remarkably well-crafted.

The seemingly magical capabilities of ChatGPT today are not dissimilar to the perceived wonders of our technology back in 2010.

So, what does this mean for entrepreneurs? What is the potential gold mine?

Just Because You Can, Doesn’t Mean You Should

One crucial lesson we learned at Automated Insights was the importance of not automating every process. After mastering our technology, we decided to demonstrate its capabilities by launching over 800 websites, one for each professional and collegiate sports team in the United States, generating content multiple times daily.

While this was impressive, we encountered a significant issue—every major team had at least one human journalist covering it. Despite the brilliance of our technology, human reporters had the advantage of gathering quotes and providing nuanced insights.

Great technology, poor application. Throughout my entrepreneurial journey, I've realized that this phrase often leads to the downfall of promising startups.

However, an intriguing development occurred.

Identifying the Optimal Use Case

We began to notice that smaller college conferences were actively promoting our content on social media and issuing press releases when one of their players received recognition from us.

This was a revelation: these smaller teams were largely overlooked by traditional media. We discovered an unfulfilled demand and a new audience.

By repurposing our existing technology, we successfully addressed this unmet need. This realization spurred our project for Yahoo Fantasy Football, which proved to be a lucrative venture before its official launch, based on the revenue generated from advertising against our content. We decided to fully commit to producing narratives from large data sets in finance, marketing, and demographics—insights that would take a human weeks to compile could be generated in seconds.

What does this imply for ChatGPT? To return to the main question, I currently do not foresee a glaring optimal use case on the horizon. Despite the impressive nature of the technology, there isn’t a pressing need that ChatGPT can immediately address.

Nonetheless, numerous possibilities exist. I've been inundated with suggestions from enthusiastic entrepreneurs eager to capitalize on ChatGPT. My responses have been firm: ChatGPT should not write your blog posts, nor should it nag colleagues for reports. It is not a substitute for human interaction.

This raises the question: Where do I envision a viable use case emerging?

The Web is Overloaded

Alongside our mission to produce written narratives from data, we recognized that the internet itself was overwhelmed by the sheer volume of information being generated and aggregated. I firmly believe that in 2023, the web is fundamentally flawed.

Initially branded as an "information superhighway," it has devolved into a congested mess. Search engines prioritize paid results, social media platforms are rife with misinformation, and professional media often resorts to clickbait and dwindling revenues. Many younger users are turning to Reddit for reliable information.

To emphasize this point, Reddit is increasingly becoming the go-to source for truth.

Can AI and NLG Be the Solution?

Possibly. I believe that addressing the information overload is the ideal use case for ChatGPT at scale. However, initial outcomes have been less than promising (looking at you, Bing and Google).

We may require another catalyst.

Automated Insights could only meet the demand for machine-generated, data-rich narratives once cloud-based processing power was effectively harnessed. The most cost-efficient way to produce content at scale involved spinning up numerous servers for short bursts and then shutting them down. We frequently pushed the limits of AWS and collaborated with them to optimize our operations.

ChatGPT will need a similar breakthrough. While its potential is immense, it currently serves best as a tool for entrepreneurs rather than a foundation for building a product.

If you found this discussion informative and insightful, I invite you to subscribe to my newsletter at joeprocopio.com for updates on new content.

For personalized startup and business advice, consider a free 15-day trial of Teaching Startup.