In recent years, the application of visual image sensors in intelligent robots and home security systems has been very extensive. At the same time, more pioneering technology application developments have also brought other multi-pixel sensors into the field of home automation and smart home. Thermal imaging sensors are a good example of how to prevent fire hazards in the home, act as a cooking assistant for users, and monitor and care for the elderly and infants in the home. Let's look at a few specific cases.
01 Safer, smarter kitchens
Why use thermal imaging in your kitchen? The short answer is to ensure the safety and health of users and provide convenience. In fact, according to the National Fire Protection Association (NPFA), cooking is the main point of ignition for home fires, accounting for about 49% of such fires. Not all smoke detectors in households with fire accidents fail, on the contrary, when the detector detects smoke, it is already too late to immediately eliminate the cause of the fire. Thermal imaging minimizes such accidents by identifying high temperatures that could lead to fires before risk factors occur.
One way to adopt thermal imaging technology in the kitchen is to integrate sensors into the range hood to achieve a "cooktop + monitor" mode. As shown in Figure 1, Cooksy-Pro is the world's first smart cooking assistant product. In the picture, the copper-colored device installed under the range hood is the Cooksy-Pro, whose thermal imager can continuously monitor the temperature distribution of the entire stove or stove. Many cooking ingredients are sensitive to processing temperatures. Overheating may not only reduce the nutritional value of ingredients, but even convert them into harmful compounds. The thermal imager above the cooktop can continuously "read" the temperature of the cooking meal and display it to the user, helping the user to cook a healthier meal. Cooksy Corp. Steven Cartwright, Chief Financial Officer, said: "This reminds us of the concept of a cooking assistant that helps users make healthy and delicious dishes every time. "Essentially, Cooksy enables users to share details such as their own or others' cooking recipes and the cooking time and food temperature spent during the cooking process. Cartwright explains: "Cooksy combines vision and thermal imaging camera sensors in the device above the stove. The device can transmit the user's stove situation to the user's mobile app in real time, allowing the user to "see" the temperature of the cooking ingredients in time and take follow-up actions based on this information. ”
Figure 1: Cooksy-Pro is the world's first smart cooking assistant
The advantages of the cooktop monitor are not limited to the remote cooking function, but also in the field of kitchen safety protection. Monitors, for example, can detect unattended gas stoves overexposed or overheated stoves, overheated pots and pans (oily or oilless), vessel boiling and spillage, and other dangerous situations. Figure 2 shows a kettle that is boiling, monitored by a thermal imager at all times. Although the temperature is not yet close to 100 degrees Celsius, the boiling point of the water, thermal image processing and analysis applications can detect when the kettle water begins to boil, when it begins to overflow, and when the gas appliance needs to be turned off. In the case of a whole-house smart home network, identifying such risk events can not only trigger home alarms, but also regulate or stop the power or gas supply if necessary to prevent the accident from expanding or catching a fire.
Figure 2: Cooktops using thermal imaging analysis techniques (Meridian Innovation)
The stove monitor isn't the only place in the kitchen where thermal imaging sensors can be found. Figure 3 shows an early prototype of a microwave oven that supports thermal imaging.
Figure 3: Smart Thermal Imaging Microwave Oven (Meridian Innovation)
The smart microwave oven in Figure 3 integrates thermal imager technology, which automatically heats food to the right temperature, rather than simply heating a fixed power for a fixed period of time. In addition, with the application of AI capabilities on TinyML's embedded processors, smart microwave ovens automatically select the optimal temperature by identifying what the user is putting in the microwave oven when heating.
02 Air conditioners that care about the comfort of users
HVAC systems are another example of integrated thermal imaging technology positively impacting product design. Most air conditioners commonly used today have a simple temperature sensor embedded to control the room temperature, providing unique feedback from the environment, and such a system can be considered the first generation of air conditioning sensing systems. In contrast, the second-generation air conditioning sensing system with integrated thermal imaging technology can detect whether there are people in the room and where these people may be located in the room, and more effectively grasp the situation of the space unit. For small apartment or hotel rooms, a thermal imaging camera of a few hundred to a thousand pixels may be enough coverage, while for larger spaces, a sensor of several thousand pixels is required, plus a wide-angle lens of more than 90°. Figure 4 shows a product on the market that applies the concept of intelligent thermal imaging air conditioning. Note that thermal imaging technology can provide insight into not only spatial utilization, but also the temperature distribution of the entire space. Based on this information, the air conditioning unit can automatically adjust the temperature, direction and speed of the airflow to save energy and improve user comfort.
Figure 4: Smart Camera Air Conditioner (Photo: Meridian Innovation)
Air conditioning and HVAC systems are used to improve the user's comfort, rather than simply lowering or raising the temperature of the room to a preset temperature. This leads us to what we call the concept of a generation 3 HVAC system. At that layer, the heating or cooling unit uses higher-resolution thermal imaging — about 5,000 to 20,000 pixels — and it is possible to obtain more detailed thermal images, directly measuring the average skin temperature of the occupants and estimating their level of comfort. This information enables environmental control that targets true thermal comfort rather than temperature. The second generation of air conditioning units is already on the market, thanks to the recent batch supply of cost-effective 1,000 to 5,000 pixel thermal imaging cameras. It is predicted that the rapid development of thermal imaging technology will bring Generation 3 capabilities to high-end air-conditioning models within five years.
03 Smarter home care
Baby monitors with cameras have been on the market for a while and are a great device for users to share baby records with friends and family and remotely care for babies at home. However, traditional cameras cannot provide insight into a baby's physiological state, but thermal imaging technology can do just that. Figure 5 is a baby monitor that integrates a vision and thermal imaging camera and supports the camera using a rich AI computing engine. Since fever is a common symptom of the disease in infants and young children, every parent wants to be detected as soon as possible, even in the middle of the night. In order to solve this need, users can start to use the infant monitor to track the baby's normal body temperature when the infant's signs are healthy, and then the monitor can alert the parents or caregiver as early as possible when the baby and child have abnormal body temperature symptoms according to the data situation.
Figure 5: Smart Baby Monitor (Courtesy of 5GenCare)
Health monitoring through thermal imaging can also be extended to elder care. Based on two important current global human trends: life expectancy and population ageing, the development of elderly guardianship technology is also showing its importance, and thermal imaging can play a role in many ways, from simple body temperature and dynamic vital sign monitoring to complex behavioral analysis. A key need in elderly care is to prevent the elderly from falling. While this can be partially achieved by installing a camera in the living room, it is not possible to install a camera in a high-risk bathroom. With this in mind, a research and development team at LSCM Hong Kong pioneered the thermal imaging camera solution, which was built on a relatively low-resolution LWIR sensor with fewer than 5,000 pixels. The camera has enough resolution for attitude estimation for continuous, real-time thermal image processing — but most importantly, it protects the privacy of the elderly. The product, which can detect the occurrence of accidents or falls and alert elderly caregivers or family members, won a gold medal at the 47th International Exhibition of Inventions in Geneva in 2019.
Thermal imaging sensing technology is expected to enhance the functionality and value of many smart home devices, such as smart health monitoring, home care and fire prevention, while also providing energy efficiency and comfort in people's daily living spaces, and even improving human cooking! These advances are due to the mass production of a new generation of thermal imaging cameras with resolutions of thousands or even tens of thousands of pixels. At the same time, a large number of hardware designs and software solutions have greatly facilitated the adoption of the technology and accelerated the development cycle of innovative products that include thermal imaging cameras.
Content and image source: EETIMES, Yishang Network, Meridian Innovation Limited