Hi

It’s fashionable to talk about the “knowledge economy” as something distinct from the broader economy, but the reality is that every company is in the knowledge business, and every business owner is becoming a knowledge worker.

This is increasingly evident as advanced AI capabilities and enhanced tools and techniques are made available to every business owner. How well companies embrace their knowledge dividend will come down to how well they harness their data. 

It’s fair to say that no company can have an AI or business strategy without having a data strategy. Because without good, clean data that is easily (and responsibly) accessible across the business, it will be impossible to generate business, operational, and AI value.

Data cannot be an afterthought in generative AI. Rather it's the core fuel that powers the ability of a business to capture value from generative AI. But businesses that want that value cannot merely manage data; they need to understand how to use data to lead the business.

As the saying goes: if your data isn't ready for generative AI, your business isn't ready for generative AI. The challenge for today's business owners and data leaders is to focus on the changes that can enable generative AI to generate the greatest value for the business.

In determining a data strategy for generative AI, business owners will need to develop a clear view of the data implications of the businesses overall approach to generative AI. Most entrepreneurial businesses will need access to models that they fine tune on their own data rather than just consuming pre-existing services and publicly available databases.

A key function in driving this approach will be communicating the trade-offs needed to deliver on specific use cases and highlighting those that are most feasible and create the most value to the business.

The big change when it comes to data is that the scope of value has got much bigger because of generative AI's ability to work with unstructured data such as emails, chats and videos. This represents a significant shift because data architecture has traditionally worked with only structured data. Business owners will need to map out all unstructured data sources and establish tagging standards so models can process the data and find the data they need. 

Further, most data will need to be pre-processed, for example, by converting file formats and cleansing for data quality so that generative AI can use the data properly.

Business will increasingly want to develop vector databases to create numerical representations of text meanings in order to streamline access to context, the complementary information generative AI needs to provide accurate answers. Effective prompt engineering (the process of structuring questions in a way that elicits the best response from generative AI models) relies on context, which can be determined only from existing data and information across structured and unstructured sources.

Vector databases allow generative AI models to access just the most relevant information, so that instead of providing a thousand-page PDF, for example, a vector database provides only the most relevant pages.

The generative AI world today has many similarities to the opening up of the Wild West in America in the 19th century. There are more unknowns than knows and businesses are learning their way forward. 

It is crucial that businesses set up systems to actively track and manage progress on their generative AI initiatives to understand how well their data is performing in supporting their businesses goals. 

Just as they do to measure their financial performance, businesses must set up effective metrics to measure both core KPI's and operational KPI's (the underlying activities that drive KPI's) to help track the progress of their AI initiatives and identify the causes of any issues. The value of these metrics will only be as great as the degree to which business owners act upon them, hence the need to establish data performance metrics that can be reviewed in near real time and used to make critical decisions.

I'd love to hear about how you are using generative AI, whether you are building your own databases of unstructured data, and how your prompt engineering is progressing. Please drop me a line and let me know.

Noel Guilford