Decoding AI: Part 3, Making data speak human

Siddhartha Chaturvedi

Miri Rodriguez

October 10th, 20230 2

Welcome back to Part 3 of our Decoding AI: A Government Perspective series. In our last installment, we explored the nuanced landscape of anomaly detection and how it intersects with generative AI in shaping the future of public governance. As promised, today’s dialogue will pivot toward semantic search and its role in converting complex data into easily interpretable, actionable insights. – Siddhartha Chaturvedi, Miri Rodriguez

Semantic search: A cognitive leap beyond keywords

For a long time, traditional search engines have been revered for their impressive ability to sift through and parse millions of documents in the blink of an eye. This remarkable feat of processing power and speed has revolutionized the way we access information. However, despite their advanced capabilities, these engines often stumbled when it came to understanding the nuances of human language. Specifically, they struggled with comprehending the context or sentiment embedded within a user’s query. This limitation highlighted the complexity and subtlety of human communication, which goes beyond mere words and involves a deeper understanding of context, tone, and emotion.

This is where semantic search swings in, adding a layer of comprehension to the raw computing power of generative AI. Utilizing Natural Language Processing (NLP), it can decipher not just the query’s literal text but the intention behind it.

This image shows how a semantic search engine interprets and responds to a natural language query. The top part of the image illustrates the query analysis, where the search engine breaks down the query into semantic components and relationships. The bottom part of the image illustrates the data retrieval, where the search engine accesses and displays relevant data sources that match the query. Semantic search engines aim to provide more natural and interactive search experiences for users. – Generated by Bing Chat (GPT4)

Imagine the transformative potential this capability can have in the public sector. For instance, immigration officers could sift through vast legal documents, not merely for generic terms like “visa statuses,” but pinpointing nuanced conditions such as “temporary work visas granted under economic hardship.” This level of precision could significantly enhance efficiency and decision-making in immigration processes. It’s a shift from data retrieval to knowledge discovery, streamlining governmental operations like never before.

Potential use cases

Semantic search lends itself to multiple applications within government:

Climate resilience and adaptation

Agencies focused on environmental protection and climate adaptation can employ semantic search sift through meteorological data, scientific research, and crowd-sourced data, then translated to the right format leveraging LLMs. With engines like Microsoft Planetary Computer, we can even bring in Geospatial data, but more on that in the next blog. This can yield an intricate understanding of climate-related risks and potential adaptive measures, simplified for understanding and access, aiding in policy formulation and community engagement, supercharging investments to build a climate ready nation.

Citizen services

Imagine a citizen wanting to understand tax codes or healthcare benefits. A semantically intelligent portal could dissect the context of the user’s query, tapping into underlying databases and generating human-readable summaries or action plans, thereby accelerating service delivery.

Integrating generative AI: The ‘translate’ function for data

Semantic search is not an island; it works best when integrated with generative AI. Consider a scenario where an anomaly is detected in public healthcare records. The semantic search first narrows down the possible causes from medical literature and guidelines. Then generative AI kicks in, translating the jargon-filled text into layman’s terms complete with suggestions for policymakers.

This image shows how a generative engine can produce new and realistic content based on a user’s input. The user can enter any natural language query or prompt, and the generative engine will create content that matches the query. The image illustrates the data flow between the user and the generative engine, Generative engines are a type of artificial intelligence that can learn from existing data and generate novel data that reflects the characteristics of the underpinning training data. – Generated by Bing Chat (GPT4)

The confluence of technologies: Framework for synergy

So, what’s the ideal architecture for this? While the architecture of integrative AI is still evolving, we suggest a way of thinking that has three parts. The first part is about people. We want to make AI that helps people and understand them. The second part is about nature. We want to make AI that fits well with the things that people already do and use. The third part is about process. We want to make AI that can change and grow with the needs of people. Semantic search is a type of AI that helps with the first part. It can understand what people mean when they search for something. Generative AI is a type of AI that helps with the second and third parts. It can create new things that match what people want. It can also work well with different situations and scales.

Let’s get started:

Prepare your semantic search
- Begin with a semantic search engine like Azure Cognitive Search, optimized for government databases.
- Solution accelerator for knowledge mining available for Azure Government – Azure Government (microsoft.com)
- Read more: What is Azure Cognitive Search?
Incorporate generative AI
- Integrate your search results with a generative AI model using Azure OpenAI Services.
- Read more: Use OpenAI GPT-4 with Azure Cognitive Services
Validate and fine-tune
- Use dashboards and logs to understand how often the AI system is used and how accurately it interprets semantic queries. Iterate.
- Read more: Microsoft Learn on AI accuracy

In our next installment, we will delve into multimodal sensing, feeding diverse data streams, text, voice, and images, to have a higher fidelity of information and knowledge.

The horizon of integrating AI and human-centric design is dazzling, and we couldn’t be more excited to explore this frontier with you.

Your continual engagement fuels our pursuit of clarity in this complex landscape. Don’t hesitate to share your thoughts as we continue this intellectual journey together. Stay tuned!