Who remembers when the term citizen data scientist was the buzzword of the moment, and you could play bingo with it in every data strategy meeting? Well, it didn’t exactly take off in practice, despite all the great ideas and intentions. The biggest culprit? Impractical interfaces and clunky tools - what was missing was an easy-to-use platform as the foundation. But now that Databricks is in the picture, it’s time to raise the difficulty level and start talking about citizen GenAI opportunities!
Databricks has recently launched its own GenAI assistant, AI/BI Genie, designed with business users in mind. AI/BI Genie offers business teams a natural language interface to interact with their data. Powered by generative AI customized to your organization’s language and data, Genie evolves with user feedback, enabling you to ask questions just as you would a knowledgeable colleague - and get precise, relevant answers from your enterprise data.
Traditional dashboards have limitations in terms of flexibility and interactivity, making it hard for business users to easily access and analyze data. With Genie, users can explore beyond dashboards and find answers without relying on complex tools or experts. Leveraging the speed and scalability of Databricks SQL, Genie delivers instant responses while maintaining governance and security through Unity Catalog.
This is how Databricks advertises Genie, but let's put it to the test and see how it truly performs!
Easy to find, simple to start
You can find Genie in the left-hand menu of your Databricks Workspace under its own name. Clicking it opens a view where you can explore existing Genie spaces or create new ones. Creating new spaces is a quick and straightforward process.
When setting up a new space, remember to:
Provide a clear and descriptive name.
Write a detailed description to guide users.
Select the appropriate SQL warehouse (Genie operates on top of these).
Choose the necessary tables for Genie to query data from. It's good to keep in mind that data access is governed with the viewers Unity Catalog permissions.
Add example questions for users to see on the welcome screen.
Keep in mind that Genie relies on metadata for tables, so make sure the selected tables have well-written descriptions at both the table and column levels to maximize usability.
Configurations makes all the difference
Under the Instructions tab, you can provide general guidelines for how Genie should behave. Prompt engineering is making a comeback, so be as clear, precise, and easy to understand as possible. Keep in mind that Genie hasn’t seen your data before - it relies heavily on metadata and the instructions you provide here to perform effectively.
The exciting part here is the ability to provide example SQL queries and functions. Think of it as creating a "golden record" SQL query that Genie can use as a reference. Keep in mind, Genie supports table-valued functions (TVF), so it's crucial to ensure your functions are built correctly to ensure reliability. Genie evaluates each user request to determine whether to utilize the function, but if the function isn't working correctly, Genie won't be able to fix it on the fly. Plan and test carefully for the best results.
You can also add new datasets via the Data tab or edit existing ones in the Settings tab - these are the same configurations you set when creating a Space. Unfortunately, the LLM model itself cannot be modified or replaced with your own. While this limitation is usually not an issue for standard use cases, it does create a bit of a "black box" feeling when it comes to Genie.
How Databricks Genie makes citizen GenAI possible
Finally, it’s time to put Genie to work! You can start asking questions, but keep in mind that the scope is limited to the selected datasets (so no asking for fun jokes!). Genie will automatically generate the necessary SQL queries to fetch the requested information or activate the selected functions independently. It’s quite fast, especially considering that the data needs to be fetched every single time. To enhance transparency, users can review each SQL query used, which is fantastic (I’m not a big fan of black boxes ⬛). All in all, quite simple and efficient tool!
As always, a feedback loop is crucial for Genie as well. For every response, you can provide feedback with a thumbs up (the response was good) or a thumbs down (the response missed the mark). This feedback is invaluable as training data, helping Databricks refine Genie to better meet user needs. These options are standard across the platform, but there’s a new functionality available: the review feature. If you’re unsure about a response, you can flag it and send it to Genie space authors for validation. This is an excellent feature that allows subject matter experts (SMEs) to get involved. Sometimes, business users have highly specific questions that require deep expertise, and now there’s an incredibly simple tool to facilitate that!
Native integration to AI/BI Dashboards
Dashboards have long been the go-to tool for analysts and business users alike. Personally, I’ve often struggled with their static nature throughout my career, frustrated by how challenging it can be to drill down into the numbers. As everyone knows, charts can be visually tailored to show almost anything, but the numbers behind the charts tell the real story. I almost jumped for joy when I discovered that Genie is now natively integrated into AI/BI dashboards. With the click of a button, you can activate the Genie chat window and start asking questions about the data behind the dashboard - no coding required! This allows you to dive deep into the numbers behind the charts, making analysis far more rewarding. This is a truly groundbreaking feature that is set to reshape the way we work in the future.
Quality & Monitoring
Genie offers a variety of tools and approaches for quality assurance. We’ve already discussed the general feedback mechanism for responses (thumbs up/down), but let’s take a closer look at the author validation feature. If a user is unsure about the accuracy of a Genie response, they can "flag" it with a comment. These flagged responses appear in the "history" tab, where Genie space authors can review them. Validation is as simple as the click of a button, with an option to add comments for further context. This fosters direct interaction between end users and SMEs, eliminating the need for lengthy ticketing processes that can drag on for months.
Another intriguing capability is the benchmarking feature, accessible under its dedicated tab. Here, you can input sample questions along with the corresponding ground truth SQL queries. This makes it easy to validate how Genie performs with benchmark queries. It’s highly recommended to run this operation during the initial setup of a space. If Genie struggles to generate correct responses (i.e., the SQL queries don’t work as expected), consider the following steps:
Double check that all needed datasets are selected for the space in question.
Verify that the metadata is sufficiently detailed (including dataset and column-level descriptions).
Provide more detailed guidance in the general instructions and example SQL queries.
These steps will help optimize Genie’s performance and accuracy.
Genie’s SQL queries are also logged into system tables, enabling quality control directly through query monitoring. However, the data is limited to SQL queries - neither the prompts nor the final text responses are accessible through this logging. Perhaps this will change in the future?
Wrapping it up
Databricks AI/BI Genie is designed for business users and analysts, bringing the power of citizen GenAI and enabling them to work without writing a single line of code, thanks to its text-to-SQL functionality with enrichment. While it’s still in its early stages for more advanced use cases, it already performs exceptionally well for foundational tasks. Imagine the efficiency boost when business users can independently analyze data on demand, without needing data professionals to translate their ideas into SQL queries. This not only saves resources across the board but also provides access to critical insights much faster by eliminating bottlenecks. The quality and SME feedback features make Genie an even more practical tool, seamlessly supporting everyday workflows.
The dashboard integration is the icing on the cake, making analysis even more user-friendly. If Genie doesn’t feel like the perfect assistant straight out of the box, smart configuration can unlock even greater potential—though this does require a solid understanding of how it works under the hood. Genie is still in its early stages, but it already shows great potential!
Ps. If you got interested, here's a bit more detailed tips and tricks article provided by Databricks solution architects: Best Practices for AI/BI Genie Spaces on Databricks
Written by Aarni Sillanpää
From bottlenecks to data democratization
Related Posts
