RecipeSnap- Your Digital Cooking Companion!

Cookbooks are picture books for adults. And I love myself pretty pictures of good tasting food and the challenge of trying to recrease them myself! Sometimes cookbooks aren't the most user friendly, so I built RecipeSnap to provide an easy to follow digital cooking experience for your cookbooks. All you have to do is take a picture of the recipe you want to cook and RecipeSnap will create a digital experience to help you cook your recipe!

Sometimes a recipe isn't a complete meal. RecipeSnap uses AI to generate suggestions for how you can complete your meal!
Do you need to double the serving size? RecipeSnap allows you to change the serving size and will update the recipe ingredients and instructions accordingly. So you don't need to do the math yourself!
Never miss an ingredient before you start cooking. RecipeSnap gives you a list to easily track ingredients you have and what you're missing.
Track where you are in the recipe with the step by step instructions you can click through so you don't have to read through paragraphs of text and maybe miss a step.

How does it work?

RecipeSnap uses a finetuned LayoutLM model to extract the recipe name, servings, ingredients and instructions from the recipe. The model is trained on hundreds of recipes with weekly updates to improve the user experience.

Once data is extracted from the image, ChatGPT is used to summarize the instructions. Many times cookbooks have lengthy paragraphs of instructions that can be hard to follow. The instructions are often printed in paragraphs to save space on the page at the cost of usability. ChatGPT summarizes the instructions that the user can easily follow and not have to read through long paragraphs of text.

ChatGPT is also used when users change the serving size. When the user changes the serving size, ChatGPT will do the calculations for updating the ingredients and instructions for the new recipe serving size.

Whats next.

Right now my primary focus is to make sure the existing features work really well. Weekly updates are shipped for the model itself so that it's constantly improving. Cookbooks present recipes in many different formats, so I'm working to make sure the model is robust to as many different representations as possible. The GPT prompts also need a little more refinement to reduce the frequency of inconsistencies in the generated results.

Aside from improving the baseline performance of existing features the main addition I'd like to add is the ability for users to save their recipes and easily search them. One problem with cookbooks is I'll find a recipe I like and never be able to find it again because cookbooks aren't easily searchable. Since RecipeSnap is already extracting the important parts of the recipe to display in the app, saving and making that information searchable provides another benefit to the user.

Go try out RecipeSnap and let me know what you think!