10 Computer Vision Ideas
Computer Vision is a subset of Machine Learning. Many times new users ask how they can get started with computer vision. Do they take a class? Read a book? Find a tutorial? There are so many paths one can take to get started it can be overwhelming. While it comes down to ones preference and learning style. My take is that the best approach is to just dive in and make something. Find something that you are passionate about and start building!
To get your creative juices flowing I’ve put together a short list of computer vision projects you can leverage if you need a starting point. Below are some awesome projects that highlight a fragment of what’s possible with computer vision. I hope by reading through this list you will be inspired by one or more of them. Sparking a creative idea on how you can adapt one of these or coming up with your own idea altogether.
Overview of computer vision projects
1. Background Removal
2. For Fun
3. Animals
4. Hands Free
5. Visualizations
6. Play Games with your body
7. Body Pose Detection
8. Food Calorie Detection
9. Rubik’s Cube Solver
10. Augmented Reality
1. Background Removal
Removing backgrounds from an image really has a wide variety of applications. It makes it great for adding images to photoshop or other image editing tools. There were many times I have needed to isolate a logo or object and add it to a meme or presentation. There are quite a few open projects utilizing this idea. Check out the available github repos below. One favorite of mine was the slack bot that performs this action based on the “edward scissor hands emoji”
Github Project: https://github.com/topics/background-removal
2. For Fun
Computer vision projects don’t always have to have a purpose or reason why it was made. Sometimes it’s just fun to build things. I am really excited to see how computer vision gets implemented into more fun games, toys and even art. I really believe that we will see the future of art and games change in the near future as computer vision gets blended into our reality. This idea is based off of a video tutorial where instead of being able to read sign language the user has instead made it follow his finger tips.
Video Tutorial — https://www.youtube.com/watch?v=01sAkU_NvOY
Reddit Post: https://www.reddit.com/r/computervision/comments/ouk0lv/when_you_get_bored/
3. Animals
Identifying animals is a classic use case of machine learning. From keeping an eye on your pet when you’re not home to making sure your pet doesn’t get into trouble. Many researchers and students also leverage custom computer vision models to help save animals in the wild or for performing wildlife research. What ways could you envision leveraging a custom model to help animal conservation?
The dataset for this example doesn’t seem to be around anymore but there is a similar one on Roboflow Universe for example an Aquarium Dataset
4. Hands Free
Everything seems to eventually become hands free or wireless. This type of CV project is just the natural evolution of that. There are some pretty cool examples of creating hands free computer vision programs to control a variety of items. From musical instruments to games. What will be next? A hands free operating system?
Project Website: https://handsfree.js.org/#installing
5. Visualizations
This project is a really cool combo of what’s possible with sports and art, blending the two together. This example tracks ping pong balls across a table. However it can be applied to many other sports as well. From soccer, to football, basketball or hockey, plus many more. What’s really cool about this too is the insights you could actually gain as well. By seeing where the ball has gone one could learn how to reposition themselves in a game allowing for better spikes, steals and interceptions, just based off of where the ball is statistically likely to go during a game or against a certain team or opponent. If you’d like to see another example of this type of computer vision project. Make sure to check out this blog post which details how XXXXX visualized football game data
Ping Pong Project (paid course): https://note.com/tdsw/n/n7445bdbc907aFootball Data
(free): https://blog.roboflow.com/ai-football-coach-playbook/
6. Play snake with your head
The gaming industry is changing rapidly. How we control and interact with video games is evolving all the time. From consoles to computers to handhelds to VR systems and next I believe it will be with computer vision. Imagine you being the controller. Imagine Will boxing but with no controller in your hand. Nothing to hold you back or tie you down. How do you think the controller will evolve? Let me know your thoughts.
Github: https://github.com/paruby/snake-face
7. Body Pose Detection
Body pose is another amazing way humans can gain insight and better control of themselves. There is lots of room for innovation in this segment of computer vision. As an example I would love to see a slouch detector, so I could improve my posture while sitting down. This is also the type of technology that I wish I had when I was playing sports growing up.
Imagine practicing your sport but then being able to go back and review not just the footage of yourself. But all the times when your form was off or not. Weight lifting injuries could be severely reduced by ensuring you have good and proper form. Imagine a baseball player who is practicing pitching and hitting. They could get details with examples of when their form is off resulting in less power overall. Or a swimmer who could after or during practice get real time feedback on their strokes form. Can you think of any sports where getting feedback on one’s form would be helpful? How about a sport where tracking player movements could improve their positioning? What insights would you want to learn?
Github: https://github.com/tensorflow/tfjs-models/tree/master/pose-detection/src/posenet
8. Food Calorie Detection
Calories, they can be so delicious but also unhealthy. I’m not sure if you’ve ever tried to go on a diet. But in my experience, diets sure aren’t fun, but that’s besides the point. The unexpected part and what really sucked, almost more than not getting to eat what you want was the constant tracking that you had to do. Track this track that. Write down what you ate for breakfast, lunch and dinner plus any snacks. It’s exhausting and super time consuming. However, it’s critically important to do. If you don’t keep track of what you eat and how many calories you’ve eaten for the day, then you won’t be able to lose weight. As Thanksgiving is approaching an awesome project to build would be a calorie estimation app. Snap a photo of a plate full of food, detect the food types and then look them up in a database to get the calories for each group. You could even take it a step further by estimating the number of servings for each type as well increasing the accuracy of the output. This is a closed source project I found on reddit but I have linked to a dataset you could use to build your own.
Reddit Post: https://www.reddit.com/r/computervision/comments/or2s1y/i_trained_a_model_to_do_single_photo_calorie_and/
Github Dataset: https://github.com/google-research-datasets/Nutrition5k
9. Rubik’s Cube Solver
Solving and playing games make for a great computer vision project. In this example user Herbert Kociemba utilizes computer vision to solve a Rubik’s cube. Which is a popular puzzle game invented in 1974 by Ernő Rubik. If you’re interested in learning more about how Herbert’s python program make sure to check out his github project linked below.
Github: https://github.com/hkociemba/RubiksCube-TwophaseSolver
10. Enhance Augmented Reality (Hardware)
While not exactly computer vision I thought this project was too cool not to mention. Gram over at the Triton Project has been working on an open source hardware wearable. It uses existing technology to create an augmented reality headset that users can wear. What’s really cool about this open source project is that Gram has 3D printing files available as well as assembly instructions and the software guide too! Currently the project uses computer vision to detect the wearers hands and when they press augmented buttons that are present when wearing the headset. I’m excited to see where this goes as the technology improves and becomes more common place.
Github: https://github.com/atlee19/TritonProjectGram's Twitter: https://twitter.com/GrahamAtlee
I hope you have enjoyed this list of computer vision projects and datasets. Hopefully it has inspired you or lead you to a tutorial on how to make something that interests you. If you have ideas for other projects not listed here please let us know in the comments or on twitter @once_upon_a_dev. Perhaps another member of the community knows of such a project or dataset. If not feel free to start developing your own!
