How to Automate Customer Sentiment Analysis Reports: A Guide for Product Managers
Understanding customer sentiment is critical to product success. If you’re a product manager, you can use AI and customer data from various sources to shape your product and how to market it. This guide shows you how to automatically generate customer sentiment analysis reports using powerful tools from Webz.io and OpenAI. Depending on your skill sets, you may need technical help to complete the steps outlined in this guide. The steps show you how to create a Python script that puts these powerful tools together.
So, buckle up and prepare to gain actionable insights from your customers’ voices!
What you’ll need to run the script
- Customer Reviews Data — You should obtain customer reviews data from a reliable source. For this guide, we’re getting the data from the Webz.io eCommerce Reviews API. It provides product information and customer reviews from 900+ eCommerce and marketplace sources.
- OpenAI API — You’ll use OpenAI’s API to leverage the GPT-4 and DALL·E models. GPT-4 analyzes and summarizes the text from customer reviews, while DALL·E generates a main image for the report.
- Python — We’re using Python to automate the report creation process. You’ll need to ensure you can run Python code on your machine.
Set up your development environment
Setting up the environment to create automated reports requires the following:
- Get a Webz.io API key — You need an API key to use the Webz.io eCommerce Reviews API. To get a key, contact Webz.io. This guide includes a free NDJSON file with sample reviews data if you would like to experiment with the script without using the API.
- Get an OpenAI API key — You also need an API key for the OpenAI API. Create an account or sign in at OpenAI to get a key. OpenAI uses pay-per-use pricing for its language and image models. You can see the price points on the OpenAI website.
- Install Python — If you don’t already have a development environment with Python installed, you’ll need to set one up. If you’re using a Windows operating system, you can find a tutorial on how to get started using Python on Windows on the Microsoft website. Next, install the Python packages using pip, the standard package installer and manager for Python.
Create your automated customer sentiment analysis report
Now that you’ve set up your development environment, you can move on to automating the report generation process. The below steps will let you automatically generate a detailed customer sentiment analysis report for a specific product.
Gather relevant data
Leverage the Webz.io eCommerce Reviews API to collect customer reviews for the product you want to analyze. Filter the reviews by star ratings for sentiment categorization (positive/negative). Use the API to create an external NDJSON file that contains the sorted and filtered reviews. Place the NDSON file in a directory and specify that path in the script file.
You could opt to use customer reviews data from another source, but you would need to modify the script and adjust the mapping for the data file. You can import an NDJSON or JSON data file.
Unlock and summarize hidden insights
Unleash the magic of OpenAI’s GPT-4 model to analyze the data gathered by the Webz.io eCommerce Reviews API. Use this powerful large language model to uncover key customer sentiments about your product and summarize the reviews for your report. The script includes a global variable that you can use to set the number of reviews included in the report. Also, make sure you’ve set up your OpenAI API key in your development environment before completing this step.
Craft a compelling report
Utilize Python’s Docx package to create a professional Word document for your report, structured with:
- Product details
- Positive feedback analysis
- Negative feedback analysis
- Recommendations and conclusion
Boost visual appeal
Use OpenAI’s DALL-E model to generate impactful visuals that highlight customer sentiment trends and insights.
Finalize and share the report
Compile all the textual and visual elements into your report and review the contents carefully. Go through and polish the report for clarity and coherence. Once satisfied with the finalized report, share it with key stakeholders to inform product development, marketing, and customer service.
Have fun experimenting with Webz.io and OpenAI
You’re all set to start automating customer sentiment analysis reports. By following this guide and leveraging tools from Webz.io and OpenAI, you’ll gain invaluable customer insights regarding your products. You can use these insights to drive informed product decisions that resonate with your audience. Let the automated reports production begin!
Download the example code and files
The full Python script
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 |
import json import glob import docx import requests from openai import OpenAI import os import openai from docx.shared import Pt from bs4 import BeautifulSoup import io from docx.shared import Pt from docx.enum.text import WD_ALIGN_PARAGRAPH from docx.oxml.shared import OxmlElement, qn from docx.opc.constants import RELATIONSHIP_TYPE openai.api_key = os.getenv("OPENAI_API_KEY") NUM_OF_REVIEWS = 50 client = OpenAI() def add_image_from_base64(doc, image_url): response = requests.get(image_url) # Check if the request was successful if response.status_code == 200: image_stream = io.BytesIO(response.content) doc.add_picture(image_stream, width=docx.shared.Inches(6)) else: print(f"Failed to download image. Status code: {response.status_code}") def html_to_word(doc, html_content): soup = BeautifulSoup(html_content, 'html.parser') for element in soup.find_all(['strong', 'ul']): if element.name == 'strong': # Add bold text as a heading doc.add_paragraph(element.get_text().strip(), style='Heading 2') elif element.name == 'ul': for item in element.find_all('li'): # Add list items doc.add_paragraph(item.get_text().strip(), style='List Bullet') def add_hyperlink(paragraph, url, text): """ A function that places a hyperlink within a paragraph object. :param paragraph: The paragraph we are adding the hyperlink to. :param url: The URL the link points to. :param text: The text displayed for the link. """ part = paragraph.part r_id = part.relate_to(url, RELATIONSHIP_TYPE.HYPERLINK, is_external=True) hyperlink = OxmlElement('w:hyperlink') hyperlink.set(qn('r:id'), r_id) new_run = OxmlElement('w:r') rPr = OxmlElement('w:rPr') # Set the style for the hyperlink (color, underline) c = OxmlElement('w:color') c.set(qn('w:val'), '0000FF') # Blue color rPr.append(c) u = OxmlElement('w:u') u.set(qn('w:val'), 'single') rPr.append(u) new_run.append(rPr) new_run.text = text hyperlink.append(new_run) paragraph._p.append(hyperlink) def insert_titles_in_text(text, reports): # Placeholder for inserting the titles placeholder = "[]" # Extracting the titles from the reports and formatting them with new lines titles = "\n".join([report['title'] for report in reports]) # Replacing the placeholder with the titles updated_text = text.replace(placeholder, titles) return updated_text def generate_article_image(name): print("Generating post image") image_url = "" try: response = client.images.generate( model="dall-e-3", prompt=f"Create a professional and informative cover for a Customer Sentiment Analysis report. The image should feature a diverse range of emoticons, including happy, sad, and neutral faces, symbolizing different customer emotions. Include a large, digital-style bar graph in the center, illustrating varying levels of customer satisfaction, with each bar colored according to the sentiment it represents (green for positive, red for negative, yellow for neutral). The overall color scheme should be clean and professional, with a balance of bright and muted colors to convey a sense of data-driven analysis and insight. The image must not show any text. ", n=1, size="1024x1024" ) image_url = response.data[0].url except Exception as e: print("An error occurred generating the image:", str(e)) return image_url def call_gpt_completion(prompt): return client.chat.completions.create( model="gpt-4-1106-preview", max_tokens=4096, messages=[ {"role": "user", "content": prompt}, ] ) def extract_points(reviews, sentiment): print("Extract Points: " + sentiment) points = [] for review in reviews: review_text = review['title'] + "\n" + review['text'] prompt = f"The following is a {sentiment} review of a product, summarize in one bullet point the main {sentiment} feedback:\n{review_text}" summary = "" try: response = call_gpt_completion(prompt) for choice in response.choices: summary += choice.message.content except Exception as e: print("An error occurred:", str(e)) points.append(summary) if len(points) == NUM_OF_REVIEWS: break return points def generate_intro(product_name, product_description): print("Generate post intro") prompt = f""" Write a paragraph introducing a customer sentiment report about: Product name: {product_name} Product description: {product_description} The report is created automatically by using Webz.io eCommerce Reviews api and ChatGPT. The report is generated by calling the Webz.io eCommerce Reviews API for the reviews about {product_name}. It then splits the product reviews into positive and negative reviews. Following this step, it summarizes up to {NUM_OF_REVIEWS} reviews from both negative and positive reviews using ChatGPT to create a comprehensive list of posts. It then gives those lists to ChatGPT to create a comprehensive report highlighting both positive and negative feedback and provide a report based on the feedback. """ intro = "" try: response = call_gpt_completion(prompt) for choice in response.choices: intro += choice.message.content except Exception as e: print("An error occurred:", str(e)) return intro def generate_title(product_name): print("Creating a title") prompt = "Create a title for a customer sentiment report about the following product:\n" + product_name title_text = "" try: response = call_gpt_completion(prompt) for choice in response.choices: title_text += choice.message.content except Exception as e: print("An error occurred:", str(e)) title_text = title_text.strip(" ").strip('\"') if title_text.startswith("Title:"): # Sometimes ChatGPT prefix the title with Title: return title_text[len("Title:"):] return title_text def create_negative_report(feedback, product_name): print("Generating Negative Report") prompt = f"""Create a customer sentiment analysis report that includes the following sections. Use <UL> and <LI> tags for listing items and <strong> for the titles of each section. <HTML> <strong>Analysis of Feedback</strong> <UL><LI>Summarize recurring and common negative issues mentioned in the reviews.</LI></UL> <strong>Recommendations</strong> <UL><LI>Based on the analysis, suggest actionable measures the company can take to address the issues raised in the feedback.</LI></UL> <strong>Conclusion</strong> <UL><LI>Summarize the key findings of the report.</LI></UL> </HTML> The following is the list of the negative feedback about the product you will base your report on: {feedback} """ try: response = call_gpt_completion(prompt) report = "" for choice in response.choices: report += choice.message.content except Exception as e: print("An error occurred:", str(e)) return report def create_positive_report(feedback, product_name): print("Generating Positive Report") prompt = f"""Create a customer sentiment analysis report that includes the following sections. Use <UL> and <LI> tags for listing items and <strong> for the titles of each section. <HTML> <strong>Detailed Analysis</strong> <UL><LI>Summarize common positive feedback mentioned in the reviews.</LI></UL> <strong>Recommendations </strong> <UL><LI>Propose marketing strategies that leverage the positive aspects highlighted in the reviews. </LI></UL> <strong>Conclusion </strong> <UL><LI>Summarize the key findings and the overall sentiment of the customers towards the product.</LI></UL> </HTML> The following is the list of the positive feedback about the product you will base your report on: {feedback} """ try: response = call_gpt_completion(prompt) report = "" for choice in response.choices: report += choice.message.content except Exception as e: print("An error occurred:", str(e)) return report def create_word_doc(file_name, title_text, image_url, product_details, intro, negative_report, positive_report, one_star_count, two_star_count, three_star_count, four_star_count, five_star_count): print("Saving to word document") total_count = one_star_count + two_star_count + three_star_count + four_star_count + five_star_count doc = docx.Document() # Add a title title = doc.add_paragraph() title.style = 'Title' title_run = title.add_run(title_text) title_run.font.size = Pt(24) # Set the font size title_run.font.name = 'Arial (Body)' # Set the font title.alignment = WD_ALIGN_PARAGRAPH.CENTER # Center align the title if len(image_url) > 0: add_image_from_base64(doc, image_url) title = doc.add_paragraph(style='Heading 1') title_run = title.add_run("Product Details") title_run.font.name = 'Arial (Body)' # Set the font p = doc.add_paragraph() add_hyperlink(p, product_details['url'], product_details['name']) p.add_run("\n* Price: {}\n* Total Reviews: {}\n* Aggregated Rating: {}\n* 1 Start Count: {}\n* 2 Start Count: {}\n* 3 Start Count: {}\n* 4 Start Count:{}\n* 5 Start Count: {}\n".format( product_details['price'], product_details['review_count'], product_details['aggregated_rating'], one_star_count, two_star_count, three_star_count, four_star_count, five_star_count)) title = doc.add_paragraph(style='Heading 1') title_run = title.add_run("Introduction") title_run.font.name = 'Arial (Body)' # Set the font doc.add_paragraph(intro) # Positive feedback title = doc.add_paragraph(style='Heading 1') title_run = title.add_run("Positive Feedback Analysis") positive_percentage = round(100 * ((four_star_count + five_star_count) / total_count)) doc.add_paragraph(f"Around {positive_percentage}% gave the product a very positive feedback (four and five star rating). The following analysis summarizes the recurring themes and findings in the reviews for the product. ") title_run.font.name = 'Arial (Body)' # Set the font html_to_word(doc, positive_report) # Negative feedback title = doc.add_paragraph(style='Heading 1') title_run = title.add_run("Negative Feedback Analysis") negative_percentage = round(100 * ((one_star_count + two_star_count) / total_count)) doc.add_paragraph(f"Around {negative_percentage}% gave the product a very negative feedback (one and two star rating). The following analysis summarizes the recurring themes and findings in the reviews for the product. ") title_run.font.name = 'Arial (Body)' # Set the font html_to_word(doc, negative_report) for paragraph in doc.paragraphs: for run in paragraph.runs: run.font.name = 'Arial (Body)' # Save the document doc.save(file_name) def read_ndjson_file(file_path): """Reads an ndjson file and returns the content as a list of dictionaries.""" with open(file_path, 'r', encoding='utf-8') as file: return [json.loads(line) for line in file] def main(): # Path to the reviews folder reviews_folder = 'reviews' # Read product information product_info = read_ndjson_file(os.path.join(reviews_folder, 'Products.ndjson'))[0] # Initialize an empty list to store all reviews positive_reviews = [] negative_reviews = [] # Counters for each star rating one_star_count = 0 two_star_count = 0 three_star_count = 0 four_star_count = 0 five_star_count = 0 # Read all reviews files in the reviews folder and add those with substantial text into separate lists for review_file in glob.glob(os.path.join(reviews_folder, 'Reviews_*.ndjson')): reviews = read_ndjson_file(review_file) for review in reviews: if review['rating'] == 1: one_star_count += 1 elif review['rating'] == 2: two_star_count += 1 elif review['rating'] == 3: three_star_count += 1 elif review['rating'] == 4: four_star_count += 1 elif review['rating'] == 5: five_star_count += 1 if len(review['text']) > 100: if review['rating'] <3: # 1-2 stars rating is negative negative_reviews.append(review) if review['rating'] >3: # 4-5 starts reating is positive positive_reviews.append(review) image_url = generate_article_image(product_info['name']) title = generate_title(product_info['name']) intro = generate_intro(product_info['name'], product_info['description']) positive_bullet_points = "\n".join(extract_points(positive_reviews, 'positive')) negative_bullet_points = "\n".join(extract_points(negative_reviews, 'negative')) positive_report = create_positive_report(positive_bullet_points, product_info['name']) negative_report = create_negative_report(negative_bullet_points, product_info['name']) create_word_doc("customer sentiment analysis report.docx", title, image_url, product_info , intro, negative_report, positive_report, one_star_count, two_star_count, three_star_count, four_star_count, five_star_count) print("done") if __name__ == "__main__": main() |
Free sample data (NDJSON format)
- Download a free product data sample
- Download free product reviews data: Sample #1, Sample #2, Sample #3, Sample #4
Download an example auto-generated Sentiment Analysis Report
You can download an example of an auto-generated report in PDF format here.
To run the script
- Ensure that Python and the required Python libraries are installed on your machine.
- Set your OpenAI API key in your development environment.
- Place the NDJSON file with the product and review information in the specified directory.
- Run the script.
Ready to automate customer sentiment analysis for your organization? Talk to one of our experts today.
Not subscribed to our Dark Web Pulse updates?
By submitting you agree to Webz.io's Privacy Policy and further marketing communications.
