Summarize Short Answer Survey Feedback

Decorative In this activity, we’ll explore how GenAI can help you summarize short answer survey responses, to help you quickly understand at a high level the key issues being brought up by respondents for a specific survey question. If you asked multiple questions you’d need to upload the responses for each question in individual documents. We’ll also review how to customize these summaries by adjusting the tone, style, and language level.

NOTE: Like all other Generative AI tools, NotebookLM will sometimes make mistakes, although because you are providing it with training data, it will likely make fewer mistakes than general purpose Generative AI tools. Always research any factual claims it makes when summarizing survey question responses.

If you have any questions or get stuck as you work through this exercise, please ask the instructor for assistance.

Learning goals

By the end of this activity, you will be able to:

  • Load survey or qualitative text (comments, reflections, open-ended responses) into a NotebookLM notebook in a format that NotebookLM accepts.
  • Ask NotebookLM to identify themes, patterns, and representative quotes.
  • Turn messy text into concise summaries, tables, or reports for different audiences.
  • Use citations and excerpts to ensure that themes are actually supported by the data.

Getting Started

Let’s go back to the notebook with the three journal articles we’ve already created, and add a YouTube video to the notebook so that there will be four items for NotebookLM to draw information from:

  1. Download the following two documents, each with survey question responses, to use as training data for your notebook in this activity. When you do this for your own survey data, please note that you’ll need to export the responses for each survey question you want to analyze into separate documents in a format that NotebookLM accepts (like TXT or PDF formats). Please make note of where your web browser is saving the survey response PDF files for this exercise (usually in your Downloads or Desktop folders):
  2. Create a new notebook by clicking on the blue Create new button on the left side of the screen. NotebookLM Create new button
  3. Upload the two documents you downloaded by clicking on the blue choose file link in the middle of the page. You’ll need to either select the two documents, and add them together, or do as the video below shows and add them individually:
    Add Survey Documents
  4. Good job! You’re now ready to explore ways to summarize the survey responses!

Initial Exploration of Survey Question 1 Responses

  1. Let’s limit our first summary to Survey Question 1 Responses:
    • De-select survey-question-2.pdf file by clicking on the checkbox on the right of the file name.
      NotebookLM Create new button
    • You will now only be asking questions about the responses to Survey Question 1.
  2. Let’s ask NotebookLM to summarize the responses to Survey Question 1 by typing the following into the chat box and pressing Enter: Please analyze the following short-answer survey responses from participants who attended a university-level workshop. Summarize the key feedback by identifying recurring themes, specific suggestions for improvement, and notable positive aspects. Highlight actionable insights and any patterns in participant opinions. Present the findings clearly, categorizing them into strengths, areas for improvement, and additional comments or suggestions. Ensure the summary is concise and easy to understand.
    • Reflection: Do the themes and suggestions questions look reasonable based on reading the comments in the Survey Question 1 PDF document?
    • Reflection: In my response, two instructors are highlighted for being particularly expert and helpful. Do we know if that is because these two instructors teach most of the workshops, and equally expert and helpful instructors teach only a handful of workshops so aren’t highlighted by NotebookLM? You don’t have the data to evaluate this question, but it would be something to explore if this was your research project.
  3. NotebookLM provides linkable citations for the factual claims or interpretations it makes. Let’s see if these citations can help validate NotebookLM’s claims:
    • Click on the first citation link which is a 1 in a little grey circle.
    • The Source guide with appear on the right side of the NotebookLM interface. Please read the highlighted passage to see if it supports the claim that it’s supposed to support. Does it seem accurate?
      Citation Check
    • Click on the second citation link which is a 2 in a little grey circle. Does the highlighted passage support the claim that it’s supposed to support?
    • Feel free to check out further citations in the document to see how it’s done.
  4. Now close the Source guide by clicking on the little icon in the top right corner of the Source guide area (see image below). Citation Check
  5. Let’s check our Survey Question 2 responses:
    • Click on the Sources checkbox to select survey-question-2.pdf and deselect the survey-question-1.pdf document. Deselect document 1 and select document 2
    • Let’s ask NotebookLM to summarize the responses to Survey Question 2 by typing the following into the chat box and pressing Enter: Please analyze the following short-answer survey responses from participants who attended a university-level workshop. Summarize the key feedback by identifying recurring themes, specific suggestions for improvement, and notable positive aspects. Highlight actionable insights and any patterns in participant opinions. Present the findings clearly, categorizing them into strengths, areas for improvement, and additional comments or suggestions. Ensure the summary is concise and easy to understand.
    • Note that the summary is shorter for the responses to Survey Question 2. This isn’t surprising as about half the number of people responded to this survey question compared to Question 1.
    • Try looking at the citations to see how reasonable they look.
    • You might want to try a follow-up question like: Please expand on the Actionable Insights section.
      • Reflection: What do you think of the quality of the summaries it created?
      • Reflection: Could this help you in your research workflow? If so where, and would it be worth your time and effort to work on your prompting skills to make the most of NotebookLM features?
      • Reflection: What potential drawback can you think of from over reliance on GenAI document summaries for your research?

Studio Tools

  1. Let’s get ready to check out some of the Studio Tools that come with NotebookLM:
    • Click on the Select all sources checkbox on the bottom right of the window.
  2. Reports:
    • Reports are a quick way to summarize and understand the contents of the survey feedback information in your notebook in a general way. NOTE: NotebookLM does not provide inline citations in this type of document like it does when it responds to your prompts, which makes verifying factual claims more time-consuming.
    • To create a briefing document, simpley click on the Reports button on the right side of the page. You will be given options to choose from, as well as the ability to use one of a few predefined prompts, or type in your own prompt. It will take 30-60 seconds to generate the briefing document.
    • Once it is finished creating your briefing document, simpley click on the document link directly below Studio Buttons on the right side of the page, and scan through the summary
      • Reflection: What do you think of the quality of the Summary?
      • Reflection: Are there any potential drawbacks or problems you can think of with this type of GenAI-created summary?
  3. Data Table: Create a customize table of data based on your prompt.
    • To create a data table, simpley click on the Data Table button on the right side of the page.
    • Provide a prompt for your data table. For example Create a table with feedback for specific instructors. Include: workshop, instructor name, and a quote of the feedback given.
    • It will take 30-60 seconds to generate the briefing document.
    • Once it is finished creating your briefing document, simpley click on the document link directly below Studio Buttons on the right side of the page, and scan through the summary
      • Reflection: What do you think of the quality of the data in the table?
      • Reflection: Are there any potential drawbacks or problems you can think of with this type of GenAI-created table?
  4. Audio Overview:
    • To create an audio overview, all you need to do is click on the blue Audio Overview button . Select the options that make the most sense for you (including chaning the language if you like), as well as provide a prompt describing what you want, and who your target audience is.
    • It typically takes between two and five minutes to generate the podcast, and the podcasts seem to be about 10 to 18 minutes long depending on how much information it needs to summarize.
    • Once the audio overview has been generated, you can listen to it by clicking on the light blue play button.
    • To download the audio overview to your computer to share with other people click on the three vertical dots to the right of the play button and then click on the Download button.
    • If you don’t want to wait for NotebookLM to create the podcast you can listen to the version it created for the DSC: Podcast Summary Audio (12-min).
      • Reflection: What do you think of the quality of the audio overview it created?
      • Reflection: Can you think of any ways that this type of audio overview summary could be useful?
      • Reflection: Are there any potential drawbacks or problems you can think of with this type of GenAI-created audio overview summary?
  5. Video Overview:
  6. Infographic:
  7. Slide Deck:

NEXT STEP: Creating Presentations