Researching and analyzing menu cards of up to a thousand restaurants in a given geographical area is a tantalizing task. You drive over to a competitor’s restaurant and photograph the menu card, drive back, manually enter the contents and structure of the menu to a spreadsheet, construct a usable database, and finally you are able to use the data to your advantage. Do this a hundred or thousand times over and you will have a valuable amount of data. The man hours it costs to do this and the amount of time it will take before you have a solid amount of data is not efficient and certainly not cost-effective. Well, what if you take a different approach? We live in a modern age, there must be a way of having a computer do all the work for us, right? Right! In this blog, we will explain how you can automate the process of menu card scanning by using optical character recognition (OCR) and machine learning.
Jump to:
How was it done before?
How should it be done?
What is menu card OCR?
What can you use menu card OCR for?
What is the technology behind menu card OCR?
Use cases for menu card OCR
Frequently Asked Questions
Plan a meeting with Klippa
How was it done before?
To have a clear and accurate picture of what competitors in your branch of the market are doing, you need to consistently analyze what their pricing is, what new meals they are offering or any other information that might improve your chances over theirs. Gathering data on competitors or other restaurants in your neighbourhood would involve the following steps:
- Take a photo of a restaurant menu card. This photo can be taken in haste or subject to low lighting.
- Data on the photo is manually transferred into a spreadsheet, including sections such as name, course, ingredients, price, allergy warnings, and more. This information is combined with the name and address of the restaurant.
- The spreadsheet is then processed by hand into a database of multiple restaurants.
- This database is scoured for useful information on which conclusions about for example pricing can be made.
If you wish to keep ahead of the competition, you would need to do this on a regular basis. This means this painstaking process starts over every month, or sometimes even every week, which makes it a costly and time consuming endeavour. This type of high volume and frequency work will expand your crew requirements, which might tempt you to look at back office outsourcing to low-wage countries. Understandable, but maybe there is another way.
How should it be done?
Automating your approach would be the best way to start. You would still need to provide a photo or other form of image of the menu card, but that would basically be it. An AI could take it from there. You provide the input, the AI provides the output. The following steps should be taken:
- You upload a photo or PDF of a menu card.
- The photo is automatically clarified and corrected.
- The image is then read with the help of OCR, and converted into raw text.
- This text is converted into a format that allows it to be contextualized and categorized using machine learning algorithms.
- You receive the output, which can be deployed in data management software and is easily and automatically transferred into research plans.
This process is fully automatic, accurate, fast and most of all modern. The API is supported by an AI that has been trained with practical examples and continues to learn over time. It allows you to keep everything within your own control, without being dependent upon error-prone manual entry.
What is menu card OCR?
Menu card OCR is all about automating the process of scanning numerous menu cards by uploading photos or scans of menu cards to an API and having the menus digitized and processed into usable data. We’ve seen what used to be done and what should be done. The following is an overview of the steps taken:
- You upload a photo of a menu card to the API.
- The API scans the photo and clarifies its content with blur and glare detection and correction.
- The image is then read with the help of OCR, and extracted into a raw TXT-file.
- This TXT-file is converted into a JSON-format, which allows the data to be contextualized and categorized.
- You receive the JSON output, which can be deployed in data management software and is easily and automatically transferred into research plans.
The JSON output is automatically structured into predetermined data presets, which allows you to contextualize all data on the menu card. This means that you don’t have to place each and every item into the context slot on a spreadsheet.
What can you use menu card OCR for?
The result is data that you can use for your own specific aims. It is perfect for the following purposes:
Market research
Quickly contextualize what restaurants and bars within a specific target group offer for what price and how that might change over time. It can be for exploratory, descriptive or casual purposes and have the goal of determining a business strategy, competitiveness or simply for research purposes.
Competitor analysis
Forming an accurate overview of menu offerings, pricing or changes and offers of direct competitors, so that you can quickly adapt and stay one step ahead of the competition at all times.
Geographical pricing analysis
By singling out a specific area or region, you can use the data to set up research into pricing trends of restaurants and bars in the vicinity. You can adapt your own pricing scheme with accurate, contextual data.
Large scale menu digitization
Apart from research, you can also simply use OCR to digitize physical menu cards for use online. Especially when you are a meal delivery platform, parsing and structuring all menu cards of restaurants can simplify the onboarding process for new restaurants a lot.
What is the technology behind menu card OCR?
With each step taken in the menu card OCR process, a piece of technology is triggered into action. It is this collaboration between different types of software that automates the process and makes it effective. Here’s how each piece of software works together to form the output you need:
- An easy-to-use GUI (Graphical User Interface) gives you access to the camera within the app, allowing you to take and upload a picture or upload a PDF. When needed, documents are automatically corrected on perspective and blur- and glare detection is used.
- The API (Application Programming Interface) takes it from here. This is the service that does all of the work and moves between the back end, where the work is done, and the front end, which is presented to you with the GUI. The API links the upload to the trained neural network that will determine what characters and data are on the image.
- Optical character recognition (OCR) is used to determine what pixels on a picture constitute readable text. Any text that is discernible in the photo is extracted into a simple digital TXT-format. The photo has now been transformed into digital data.
- The brain behind this process is a neural network in the form of an artificial intelligence (AI), which has been trained not only in identifying the text on an image, but is also able to make a reasonable assumption about what the context of specific text is. If a menu has a specific layout with multiple subsections, then the AI is able to automatically determine which item belongs to which section and what price belongs to it. This form of deep learning allows menu card OCR to go way further than simply turning an image into text.
- Using the contextualizing capacities of the neural network, a JSON-format is formed. JSON is suitable for data contextualization and can automatically group and link data points, both with existing database entries or simply as idiosyncratic data. This makes it ideal for all research purposes.
Although this process may seem complicated and cumbersome, and on the backend it actually is, it only takes a couple of seconds to complete. The only thing the user sees is the GUI through which a photo can be uploaded and the JSON-output is produced in a jiffy. This means that this process can be repeated en masse and will extend your researching capabilities way further than manual spreadsheet entries.
Use cases for menu card OCR
So who exactly benefits from menu card OCR? Although you’ll find that the possibilities of using this data are near endless, we’ll provide you with three use cases:
Restaurant owners
Large restaurant chains need to keep track of what competitors in the area are doing. This is especially true when you drive a format that is meant to be competitive and consequently has many adversaries in a local area. Whether you are challenging the market in terms of pricing, originality of menu, variety or any other format that involves what is on the menu card, a constant eye on what the competition is doing is paramount. It will give you competitive advantage over the rest.
The best way is to plan a monthly photo of relevant competitor menu cards in the vicinity. If you immediately take the photo in the app, you can instantly upload the image and it is processed directly. Say you wish to process a thousand menu cards per month in a large urban area and you need to quickly access the data. Every time an employee uploads a picture, it is converted into JSON data within seconds. Your database can thus be kept up to date at any time. This means you can for instance safely boast the cheapest menu in the area or country, for you will know it to be true.
Market research agency
If you aim to serve the restaurant industry with accurate data, you need to instigate an effective way of gathering that data. Whether it concerns a simple SWOT analysis or menu optimization for a restaurant, you should be able to offer your clients accurate and swift advice based on a thorough investigation. Of course, you have employees that should be able to manually gather data, or you have chosen to outsource this back office work to low-wage countries. Still, gathering data is more of a computer’s job, wouldn’t you say?
Using an API to take over this work results in a more cost efficient, more accurate and faster process. Transforming a menu card into usable, segmented data is done by menu card OCR within a matter of seconds, whereas an employee will take longer to enter the data into segmented spreadsheets.
Food delivery platforms
Especially in times of COVID-19, the predominance of delivery platforms has grown. If you wish to compete in this sometimes saturated market, it is clear that you need an easy way of onboarding. However, it is just as important to keep your platform up to date. To compete, you need to offer the delivery of hundreds of restaurants and bars, all of which adapt their menus with increasing regularity. You would need a fast way of processing all restaurant menus, so that none of the information on your platform is outdated.
A quick way is to have participating restaurants upload a photo of each new menu card to our API. With menu card OCR each menu is scanned and read, extracting all relevant data into a JSON-format. This format allows you to swiftly and automatically update any restaurant menu listing on your delivery platform. It is an excellent way to keep both participating restaurants and of course your customers happy.
Frequently Asked Questions
Does it work on all menu cards?
Given that nearly all menu cards follow a similar format, our OCR API can distinguish all information on a menu card and transfer it into a JSON format. You can think of the division into aperitif, main course, dessert, and so on, but also vegetarian or vegan and pricing schemes.
Does it work with low-quality photos?
Especially when you quickly have to snap a photo of a menu card in all haste when passing a competitor restaurant, the quality of the photo will be compromised. Thankfully, our API includes image pre-processing capabilities that will improve and rotate bad quality images for better processing. Besides that, Klippa offers a scanning SDK that can be implemented in mobile apps. This SDK includes image processing capabilities like perspective correction and glare- and blur detection to clarify the content of photos. This way, you will only receive photos that contain valid information for further processing.
Which languages can it process?
Klippa’s API works best with all western languages, so common languages we work with are English, Dutch, German, French, Spanish, Portuguese, Swedish, Norwegian, Danish, Finnish and Italian. Any other language can be added on request.
Plan a meeting with Klippa
Would you like to see our product in action or are you in need of more technical information? Our experts would be pleased to guide you through our solutions in a 30-minute demonstration. You can find a time schedule below. We don’t mind customizing our products, so even if you have a unique inquiry, feel free to contact us.
