Experiment #204

Autonomous Robotic Arm with Machine Teaching, Cognitive Services and IoT

The Experiment #204 researches about the possibilities of using Custom Vision Object Detection and Reinforcement Learning in a Machine Teaching approach to building an autonomous robotic system.

Project Date: January-March 2020

Services: Azure Custom Vision, Reinforcement Learning, OpenAI Gym, Windows 10 IoT Core

Technology used: Machine Teaching – Cognitive Services
– IoT

Automatic teaching
Idea &

For this Proof of Concept, we will try to build a robotic arm. The main purpose of the project is to create an Artificial Intelligence capable of detecting objects automatically and manipulating the arm autonomously.

First, we will print some 3D pieces and assemble the servo motors and electronics on them.

Then, we will create an Azure Custom Vision project to build and train an Object Detection model. After that, we will use Machine Teaching and Reinforcement Learning to build and train a model to manage the autonomous system. Finally, we will put all things together in a Windows IoT Core application and deploy it on a Raspberry Pi device attached to the robotic arm.

Calculate the best path with the fewest number of instructions

The system optimizes the movements with the consequent energy saving and speed increase. Also, a well-calibrated robotic arm is more accurate than humans in manual and repetitive tasks.

Do you have a project in mind?