social-robotic system design with Sara and Milan
PRODUCT BRIEF
Makerspaces are becoming increasingly popular in K-12 schools, universities, libraries, community centers, and even hospitals. However, there are many problems present in these spaces. One of the largest issues is staffing the space with people who are knowledgeable and willing to help train people who are unfamiliar with using certain tools and techniques. Ensuring that all makerspace users are being safe is one of the largest challenges facing makerspaces right now. Additionally, finding ways to attract all kinds of different maker-people, regardless of experience level or background, is another huge problem present in the makerspace community.
That’s where the iMake comes in! Designed to be a both a greeter, informer, and teacher about all things Makerspace, the iMake can help welcome people into any maker/hackerspace, provides instruction and training for new tools and processes, and ensures that everyone is being safe in the space. Equipped with 360 degree cameras, omnidirectional wheels, a touchscreen display, wireless cloud connections and machine learning technology, the iMake will meet all your makerspace needs.
Note: The iMake is an adaptation of the Jibo robot. Learn more about Jibo here.
BACKGROUND
As makerspaces increase in popularity, the demand for solutions specific to makerspace environments is also increasing. For an in-depth look at makerspace history and user needs, check out the full background.
USER PHYSICALITY AND PSYCHOLOGY
With the image processing, iMake would recognize users once people walk in. Users would hear iMake say Hello and see a welcoming face. Users could interact with iMake via voice control, touchscreen, and their facial expression. By using different gestures and touching, users could see different reactions of iMake such as tapping on the head to pet it. As users moving, iMake would follow users as much as possible.
Unlike other standard robots, iMake is more sensitive and friendly. It could sense when users need help and approach the problem step by step. Users will not have any pressure by asking simple questions anymore. With the robot character, it might be easier for the users to communicate and ask for more help. Comparing to the human staff, it also provides better solutions and detailed explanation if users are curious.
At first, users might feel unfamiliar to the 'robot teacher,' but the cute and sincere smile of iMake would make a difference gradually. iMake is good at interacting with people. It has different responses to who is having trouble, hesitating or happy with the prototype, which make it more initiative and friendly.
The iMake is designed to help users who are:
Unsure about whether or not they are welcome in a MakerspaceNew to making and are hesitant to try new tools/techniqueUnskilled and need help learning a tool/techniqueOverworked/busy makerspace managers who need help monitoring tools, training users, and keeping everyone in the space safeOverconfident and may be in harms way when trying to use a tool/technique that they don't truly understand
SOLUTION
Solution Overview: iMake is an adaptation of the Jibo robot. Using the base features and same core computer as Jibo (the leader in social robotics technology), the iMake machine contains additional software and hardware capabilities documented below.
The goal of the Jibo Robot is three-fold:
1. Welcome people into the space: iMake can act as a greeter and more effectively welcome new users into and inform people about the Makerspace. Makerspaces strive to be inclusive to people of all backgrounds and experience levels. iMake will help ensure that everyone feels like they too can be a maker. By taking on a multitude of diverse user profiles, iMake's emotive display can show people of all abilities, ages, and backgrounds that the makerspace is a space for them.
2. Training: iMake can help train/teach new makers about various machines and processes. Using a combination of video, audio, image processing, and wireless machine control the iMake will walk users through doing a task or using a machine for the first time.
3. Machine management and safety: iMake acts as a machine/device management tool. Using the iMake cloud, the use status, health, and other pertinent information regarding each machine in the space is automatically transmitted to the iMake. The iMake can then alert the human makerspace manager to provide maintenance when needed, shut off machines in case of an emergency, and collect data regarding tool usage and performance.
iMake Modes
Sensors, Actuators, and Processors
Sensors:
iMake Cloud: The iMake cloud connects the iMake robot to every machine in the makerspace through a wireless internet connection. This allows the iMake to control certain aspects of the tools, monitor the health of the machines, and conduct safety measures such as emergency stops.
Panoramic Cameras: Detect the actions and faces of people in the makerspace and people walking by. iMake can recognize human emotions (like fear and frustration) and conduct image processing on the projects in the makerspace to provide useful hints and notice when members are misusing/struggling with a certain tool or technology.
Microphone: Detects audio commands, questions, and other human sounds. The iMake machine uses these audio cues to determine the needs of the users in the space and enter the appropriate mode to meet those user needs.
Touch Screen: Allows users to select options, manipulate 3D models, and interact with the iMake in a more direct way.
Actuators:
Screen: Displays iMake's various modes: welcome mode, training mode, and safety mode. iMake switches seamlessly between modes based on user audio commands/questions.
Welcome Mode: Displays humanoid expressions and makerspace information
Training Mode: Displays tutorial videos and and animations
Safety Mode: Displays control panels indicating the status of the tools and machines in the space
Speakers: Provide iMake users with audio instructions, information, and other audio cues.
Omnidirectional Wheels: Enable iMake to move quickly and easily through the makerspace
Swivel Head: Allows the iMake to see what is happening at every point in the makerspace
Telescoping Base: This allows the head/screen of the iMake robot to raise up to human eye level and see what is happening on tables/in machines throughout the makerspace
Processor:
Jetson TX1 AI Developer Supercomputer: GPU: 1 teraflops, 256-core Maxwell™ architecture-based GPU offering best-in-class performanceCPU: 64-bit ARM A57 CPUsVideo: 4K video encode and decodeCamera: Support for 1400 megapixels/secondMemory: 4GB LPDDR4; 25.6 gigabytes/secondStorage: 16GB eMMCWi-Fi/Bluetooth: 802.11ac 2x2 Bluetooth readyComprehensive SDK for embedded visual computing and machine learning.
INTERACTIONS
The iMake uses several strategies for interacting with users. Combining a touch screen display with video and audio interaction maximizes positive user outcomes and enhances the machine learning so that the iMake becomes increasingly useful overtime.
Machine Learning: iMake learns as it experiences the makerspace. Not only does iMake learn from every interaction in its own makerspace, but using the iMake cloud, each iMake can pull information from the experiences of other machines. This means that the longer you have the iMaker, the better it will be at meeting the needs of your makerspace.
Touch: iMake's touch screen allows users to select options from a menu as well as for makerspace managers to change modes manually if needed/access various formats of information.
Sensing: iMake senses the sounds and images of whats happening in the makerspace and uses that to decide which mode to enter, how to best welcome/help someone in the space, and ways to enhance the overall making experience while keeping all makers safe.
Audio: iMake speaks to users, asks/answers questions, plays video tutorial audio, and can generate alarm signals if needed.
Visual: iMake displays different humanoid emotive screens (faces), videos, images, and selection menus. To help welcome people into the space, the iMake takes on different real life user persona's to show newcomers different maker persona's that match their own. The phrase "you can't be it if you can't see it" rings true for makerspaces. The iMake adopts a diverse persona profile to show people from all walks of life that they too can be makers.
DECISIONAL/EMOTIONAL TASK ANALYSIS
The decisional/emotional task analysis for the iMake system is shown below:
USER WALKTHROUGHS
Meet Jessica! She is a 19 year old freshman in college from Medford. As a child she often wondered why leaves change colors in the Fall before falling off in the winter. She would wander in the woods in summer looking for bugs and counted their legs, noted them down on her diary with some sketch before setting them free. Her bedroom wall had animal murals with dandelion and hibiscus flowers in the background, because her parents knew she loved plants and animals from tender age. Her love towards living beings led her to pursue biology as her major in college. In her Introductory Biology class as a freshman she has to present on DNA as an assignment. Being the hard-worker she is, she decided to put in some extra effort to build a 3D model of DNA to clearly communicate its form and function to the class. She learned about the makerspace from her friend in engineering who was trained to use 3D printer. He had proudly showed off a LEGO brick printed with elastic material. Jessica got the idea of building a physical model of DNA from that interaction.
On a cold Monday morning, Jessica fought with second thoughts to get out of bed early to pay a visit to makerspace. She had checked makerspace opening hours the night before going to bed but checked it again to be certain. She walked with a laptop in her backpack and a sketchbook. The makerspace door was propped open and she could see people walking around as she entered the space. There were about 10 people mostly working in pairs. Nobody noticed her entering, or maybe didn’t bother to entertain her. As she was turning around to leave thinking it was a bad idea she heard a robotic but warm voice. “Hello”, iMake said with an image of a twinkling set of eyes on it screen. Jessica bent a little feeling slightly silly while returning the greeting. “Hello!” iMake knew this was a new face so it went into welcoming mode. “I am Mike and I am here to help you. What’s your name?” iMake saved Jessica’s face with her name and her credentials in the cloud for future reference. He proceeded to ask if she needed any help. Jessica didn’t feel silly now. She started to feel comfortable. She asked if anyone could show her how to use a 3D printer. iMake suggested her to take a tour to the 3D printing station and led her to the digital fabrication lab. On their way iMake greeted few other familiar faces and introduced Jessica, causally. It was slightly awkward for Jessica, others were used to it. iMake played a video on its screen while narrating its different features. It demonstrated different projects that she could do with a 3D printer, showed softwares to create models, online sites to find 3D models, other online resources. It also played a video on how to use a 3D printer. At the end, it asked Jessica if she understood how to operate 3D printer or she needed more help from a human. Jessica was confident she could figure out printing on her own. So she said she was comfortable. She took out her computer and followed iMake’s instructions to print her first 3D model. iMake sat there to make sure she didn’t feel left out or get frustrated.
Evan is a 20 year old electrical engineering major. He loves coding and working with microcontrollers. He loves biking and playing football. On weekends he bikes for hours and that worries his mother. A die hard Harry Potter fan, his caricature of Snape’s mannerism and voice is spot on. He secretly wishes he had a scar on his forehead and a hairy, large man like Hagrid would come and tell him that he is not a muggle. Not usual for a 20 year old. But, oh well! Fantasies aside he does a lot of cool work with microcontrollers. He programmed his blinds to automatically open in the morning. He also built a automatic watering system for his mother’s garden using moisture sensors and an Arduino. His non-engineering family thinks he is not very far from being a magician. He doesn’t keep his pride a secret while shouting spells every now and then. Evan is a regular visitor of the makerspace. He has been working on his personal project to build a light system for his bike. He wants to make a turning signal on his bike and a tail light that lights up when he presses his brake. He also wants to send his location status to his mother’s cell phone as she is constantly worried about his biking adventures. He wants to build a printed circuit board that would be a motherboard of his system.
Today Evan is sitting in Digital Fabrication Lab in front of Othermill, a CNC machine to mill PCB boards. He has used it once before. He carefully designed a .brd file in Eagle. It took him a week to run simulations and get everything right. He puts double sided tape on a PCB board and connects his computer to the machine. Suddenly, he realizes he doesn’t know what bit size to use first. He thought he knew that. But since it has been more than a year he last used Othermill he is not super confident. He spots a student worker. But hesitates to ask because he thinks it is such a simple question and also the worker seemed busy. He googles instead. No luck! iMake with its panoramic camera has been keeping track of everything that is going around in makerspace. Reading people’s faces and updating his database on who is using what tool it constantly tries to see if someone needs help. iMake correctly diagnosed Evan’s struggle. He approaches Evan and asks “Hello Evan! What are you working on?” Evan- “I am trying to make a PCB for my project … ”. And explains his project. iMake- “Oh cool! Do you have any questions you would like to ask?” Evan tells iMake how he is getting confused on what bits to use. iMake suggests it could show him some videos on how to choose bits correctly. Evan decides to watch the video and surely enough he gets his answer. Evan loads the bits and makes his PCB. iMake stays near Evan to make sure he gets further support. When Evan completes his cut, he gives positive feedback to iMake by choosing “iMake helped me” option on the survey form on his screen. iMake updates the feedback on cloud.
FUTURE DIRECTIONS AND LIMITATIONS
For future development, we are expecting to be able to leverage emerging technology, such as holographic projection, 360-degree VR cameras, and a network of multiple iMake devices.
Equipped the holographic projection, iMake could create an immersive experience and direct interaction with users. Users would have a distinct and accurate perception of the makerspace in the virtual or augmented world. When users need an overall introduction or instruction for a specific situation, the holographic projection will lead users through the overall design process, transform the troublesome technical part into an understandable 3D view for study.
With multiples cameras, makerspaces will not need as many human workers. All the devices are connected and cooperate with each other. In this way, there will be no blind zone with our maker space monitoring system. It will eliminate the situation which employees occasionally commit mistakes
For a more smooth experience, we could load users' essential information to the system, such as their educational background, age, and overall skill set. With the assistant of public information database, we could also have those public social medium information. Then the conversation could be more personal and engaging.
iMake does not have a hand or clamp to grab or reach an object, which might make it difficult to hand over a tool. However, as an instruction robot, the primary purpose of iMake is to teach and lead the users to do it by themselves. It could move smoothly to direct the users and skilled in teaching.
As a new robot face to users, iMake needs time for people to build trust and relationship. AI and robots still have a long way to go toward being entirely trustworthy and free of bias. However, people can come to trust robots by getting to know them.
Comments