About Me

My photo
I am an MCSE in Data Management and Analytics, specializing in MS SQL Server, and an MCP in Azure. With over 19+ years of experience in the IT industry, I bring expertise in data management, Azure Cloud, Data Center Migration, Infrastructure Architecture planning, as well as Virtualization and automation. I have a deep passion for driving innovation through infrastructure automation, particularly using Terraform for efficient provisioning. If you're looking for guidance on automating your infrastructure or have questions about Azure, SQL Server, or cloud migration, feel free to reach out. I often write to capture my own experiences and insights for future reference, but I hope that sharing these experiences through my blog will help others on their journey as well. Thank you for reading!

How to Add More Documents to Your Knowledge Base Index Using Azure AI Foundry

Introduction

So, you've set up a powerful knowledge base using Azure AI Foundry, and now you have a fresh batch of documents—PDF reports, Word docs, text files—that you need to add. How do you get this new information seamlessly into your index so your AI applications can start using it?

This guide walks you through the different methods to add documents to your knowledge base index within the Azure AI Foundry portal. Whether you prefer a direct upload, connecting to cloud storage, or using the broader Azure Portal, we've got you covered.

Method 1: Using the Azure AI Foundry Portal (Recommended)

This is the most straightforward method, designed specifically for AI Foundry workflows. It's perfect for quick updates and managing your data sources directly.

  1. Go to Azure AI Foundry

  2. Navigate to Your Index

    • Click on Data or Indexes from the left-hand sidebar.
    • You should see your existing knowledge base index listed here.

  3. Add a Data Source

    • Click on + New Index or Add data source.
    • Choose your source type from the options:
      • Azure Blob Storage: Ideal for uploading files (PDF, Word, TXT, etc.) in bulk.
      • Azure Data Lake: For large-scale data lakes.
      • Upload files: The quickest way for a handful of documents via direct upload.

  4. Configure Indexing

    • Select your search service (e.g., aifoundry1478search).
    • Choose the existing index (e.g., knowledge-base) or create a new one.
    • Configure chunking and processing options (the defaults are usually optimal to start).

Method 2: Using the Azure Portal (Search Service)

If you're more comfortable in the main Azure Portal or need access to advanced data sources, this method is for you.

  1. Go to Azure Portal

    • Open the Azure Portal.
    • Search for your Azure AI Search service (e.g., aifoundry1478search).

  2. Import Data

    • Within your search service, click on Import data at the top of the overview page.
    • Choose your data source:
      • Azure Blob Storage
      • Azure SQL Database
      • Azure Cosmos DB
      • Upload a JSON or CSV file

  3. Configure the Index

    • Select your existing knowledge-base index.
    • Map the fields from your source data to the index fields (e.g., id, title, content, category).

  4. Create and Run the Indexer

    • Create an indexer, which is the process that pulls the data.
    • You can set a schedule (e.g., run once, hourly, daily) for recurring updates.

Method 3: Upload Files to Blob Storage (The "Set and Forget" Method)

This is an excellent method for automating document ingestion, especially if you regularly add files to a designated cloud folder.

  1. Upload Files to Azure Blob Storage

    • Go to your storage account (e.g., aifoundry1478storage).
    • Create a container (if needed), such as documents.
    • Upload your PDF, Word, and other documents to this container.

  2. Connect in AI Foundry Portal

    • Back in the AI Foundry portal, go to Data → + Add data.
    • Select Azure Blob Storage and connect to your storage account.
    • Select the documents container you just used.
    • AI Foundry will automatically handle the chunking and indexing of all files in that container.

Quick Reference: Direct Upload Cheat Sheet

For a super fast, one-time upload, follow these steps:

StepAction
1Go to https://ai.azure.com → Your project
2Click Indexes or Data
3Click + New index
4Select Upload files
5Drag & drop your PDF, Word, or TXT files directly into the browser
6Configure search settings (or use defaults)
7Click Create and let the platform index your content!

Conclusion

Adding documents to your Azure AI Foundry knowledge base is a flexible process. For most users, Method 1 (AI Foundry Portal) is the quickest and most intuitive. If you're building an automated pipeline, Method 3 (Blob Storage) is the way to go.

By following these steps, you can easily keep your knowledge base current and powerful, ensuring your AI applications have access to the most relevant and up-to-date information.

Happy indexing

No comments:
Subscribe to: Comments (Atom)