This November, Mr. Ryu Narusawa, a Factory Team Leader within Sony Semiconductor Solutions Corporation followed up from Maruyama-san's interview, to discuss the present-day challenges affecting IoT development.
The following interview discloses these challenges, in-depth, and gives the reader a new perspective on digital transformation within the manufacturing industry, as well as how AITRIOS™ is supporting the future of work and automation.
The following interview, translated from Japanese, has been published in association with PEAKS MEDIA. Read the original article in Japanese, here: https://www.peaks-media.com/8947/
The Difficult Challenges of IoT Development and the Future of Manufacturing Digital Transformation Envisioned by AITRIOS | AITRIOS and Manufacturing Digital Transformation [Part II]
The second half of the roundtable discussion on the complexity of IoT system development and the role of edge AI will be discussed by front-runners at the forefront of manufacturing DX. In the first half, we discussed the challenges of promoting DX in the manufacturing industry.
In the second half, we will explore the current state of IoT development, which requires a wide range of technologies, from sensors to the cloud, and solutions using Sony's edge AI sensing platform "AITRIOS". They also talked about building an ecosystem and the future of the community and covered the challenges of DX in the manufacturing industry and its breakthroughs.
PROFILE:
From right: Hiroshi Onishi, CEO of edge AI development startup Futsuper Co., Ltd., Ryu Narusawa, Factory Team Leader, Sony Semiconductor Solutions Corporation, Daisuke Hayashi, manufacturing DX consultant focit Co., Ltd.
Difficulties in IoT System Development
Mr. Hayashi: From here, we will focus on development. From the perspective of the manufacturing industry, what is the difficulty of developing IoT systems?
I would like to ask Mr. Onishi for this.
Mr. Onishi: I think it's because of the breadth of hardware involved. It is often said that "IoT is the mixed martial art of technology", but it is not enough to say that software or hardware alone is not enough, but the reality is that it is really wide.
I had been involved in the development of IoT since my previous job, such as using AI to control water control in the monitoring of factory plants. I've been writing my own programs since I was a student, so if I thought I could do something about it, I couldn't do it at all. At first, I didn't even know what to do with the sensor, how to connect it, or even a PLC*.
In the end, when you think that you can finally see the data on the PLC screen after working with someone who is qualified in electrical work, you can send it to the cloud and how to display it on your smartphone... The communication methods are different, and the sensors are all different (laughs).
Everyone: Laughter
Onishi: There are so many different types that you must study them all, which is difficult. In this way, it is connected to the cloud AWS, and the data can finally be viewed remotely and can be called IoT. In other words, the volume until it is uploaded to the cloud is extremely large. It's impossible for one person to do it alone, and it's at a level where a team of four or five people can work on it and finally do one thing, so I don't think we will be able to progress unless it is made into a platform.
Narusawa: It's like playing all positions in baseball except catcher by yourself.
Mr. Hayashi: There is a limit to how much passion and teamwork alone can confront such difficult difficulties, and I think that is the difficult part of DX in the manufacturing industry. Mr. Narusawa, what kind of ingenuity and mechanism are available on the AITRIOS side?
Mr. Narusawa: I have a similar experience. I want the smartwatch to alert me when a specific error comes out from the PLC," and I was asked, "Isn't it easy because the distance is close?"
In fact, starting with a PLC and using AWS's IoT Greengrass, the data uploaded to the cloud can be sent to a smartphone via the Internet, and the smartwatch connected via Bluetooth can finally be notified that the production equipment has stopped. Even just displaying a simple message is a huge process.
It doesn't solve all of these problems, but AITRIOS can make the most difficult part of the problem easier.
Chipsets for AI processing also require a heat sink because they usually generate a lot of heat, but IMX500 does not require it. The combination of IMX500/AITRIOS is enough to create such a handling area, and only a small amount of data can be transferred to the cloud and sent to the Internet. Even if you don't use the cloud, you can complete it in the factory, or check it here and there on your smartphone. If you have one person who is familiar with IoT, you can do it.
Mr. Hayashi: With AITRIOS, you are saying that "we cover the most difficult part of handling data" until it is sent to the cloud. If the data is up to the cloud, the hurdles are not that high, and there are people who can do it.
Mr. Narusawa: When I worked on a medical system before, we used a 24M image sensor for image processing, and we were handling hundreds of gigabytes of data overnight. If you have hundreds of GB, it's not easy to back up. By using AITRIOS for AI processing on the edge side, the flowing data is about a few kilobytes, so backup can be easily done with the copy command.
It is also important that AI processing is the hardware of the image sensor.
In a development environment that uses a graphic board, there are many open-source tools associated with it, and it will not work unless it is properly upgraded. It is also very stressful that development resources are taken up just to upgrade the version. These parts are hardware, and all you must do is create an AI model and send it in. The processed output data is also text data, so it is easy to handle.
Mr. Onishi: I mentioned earlier that IoT development is difficult, but half of the difficulty may be version upgrades. For example, if you want to install software on multiple devices with the same shell script with the same procedure, the first one will work but the second and subsequent devices will fail. This may be because the version of the software you download from the Internet changes depending on when you install it. This complicates dependencies and is prone to unexpected errors.
What kind of community is needed?
Mr. Hayashi: You mentioned that IoT development is difficult because of the wide range of technologies you handle, but how are you solving the current installation problem, for example? I don't think there is a community in IoT that says it's okay if you include it, such as AWS's JAWS-UG*, but what do you think about it from the perspective of an IoT developer?
* A non-profit community of AWS-affiliated cloud computing users
Mr. Onishi: It takes time, but basically, we contact the official office. Also, I feel that most information is gathered on a general-purpose platform. When I asked our engineers, they also told me that about one-third of their programming time is spent writing questions on ChatGPT. Eventually, it may be integrated into generative AI as a point of contact.
Hayashi: You mentioned ChatGPT, but what kind of image is the community that AITRIOS is aiming for?
Mr. Narusawa: Basically, I think it is important to proceed with development while interacting with them with passion for the purpose of obtaining feedback on needs and usability. On top of that, I think that the troubleshooting-related content gathered will fall into the generative AI as Mr. Onishi says. Developers don't like the time lag in their queries and need immediate answers. It's hard for me to pick up information buried in threads one after another and follow them topic by topic, so I ask Copilot.
The speed at which we try to create new things in a program is much faster than in the past, so I think this is the final form of interaction associated with inquiries. For example, if you try to process an image with AI, if you ask a question about how well the AI results can be obtained with this image, you should immediately get an answer such as "please brighten it" or adjust the angle of view, and ideally you will get an answer that you should tune the AI in this way.
Mr. Hayashi: It seems that AI will answer simple inquiries and judge you in advance. If you ask a more complex or difficult question, they will introduce you to someone who knows more. "Please ask Mr. Onishi about that" (laughs).
Onishi: No, I think that's really happening. Recently, there has been an increase in the number of people who say "I learned about it through ChatGPT" in our inquiries.
I thought it was a lie, but when I tried to ask ChatGPT the question, "I want to optimize staffing in the manufacturing industry, what should I do?", our company's "Skill Puzzle", an AI-based optimal service for staffing, was introduced, and inquiries came through it. Since ChatGPT is recommending it, I think that something like the LLM version of SEO will become even hotter in the future.
Mr. Hayashi: It is said that the difficulty of the community is how much enthusiasm the community manager has and the amount of enthusiasm of the people inside, but we need to put some of their tasks into generative AI and automate management, while also preparing complex skills such as referral services and marketplaces. As a community, we may not be able to make it interesting or unique. People in the manufacturing industry will also want an answer immediately, so the community that AITRIOS aims for may be aiming for that area.
Onishi: One of the current issues is that only text information remains in ChatGPT.
Photos of the site and images recording the connection method are also uploaded, but such images are not picked up by ChatGPT. Recently, to make it easier for images to be caught, we believe that related images will also be caught in the search by adding text data that relevant information is written on a specific page of PowerPoint by using RAG (Search Extended Generation) data to improve search accuracy.
If there are LLMs and communities that gather this kind of information, I think IoT will evolve even further. Since the software code itself is on the web, highly accurate answers are returned, but in the IoT field, only important information on the site is completely missing from the Internet. In the future, AI will need to be able to pull such data. I think it would be very convenient if you could pull up what is only written in the paper manual or even the diagram that has been converted to PDF.
AITRIOS' Ecosystem
Hayashi: There are several aspects of the ecosystem, but first of all, please tell us how you are trying to create an ecosystem related to development. In particular, what do you think would be ideal to create an ecosystem where the manufacturing industry in Japan can work together?
Mr. Narusawa: There are four stakeholders: AI development, camera vendors, system integrators (SIers), and end-user factories, and I think it is fundamental to create something that is easy for those people to use.
For example, if you are a camera vendor, you will not be able to use it unless you provide a register map* of the sensor and how to use it, so I want to make it easy to understand. Currently, only vendors with a certain amount of experience can make them, but if we can make them easier, I hope that a world will come where it is easy to make a small camera if it has a sensor, even if it is not a very ostentatious camera.
* This is the place where the microcontroller temporarily holds data. It's like a small plot of memory on your computer
There are not so many AI vendors that can create small AI models, so I think it would be good if we could build a relationship where we can put the models developed by such small AI vendors in our chips and sell them.
The last two are SIers who take them to the field and the end-user's factories, but I think it is necessary to improve the ease of use of the SDKs that we provide, such as those that are easy to use from the perspective of being used by production engineers in the factories in the field. Apart from that, from the standpoint of an SIer, it is often better to have a service or product that does not require heavy outside support. I would like to find a vendor who will prepare a certain amount of packaged things and package them so that it is easier for distributors to handle.
Mr. Hayashi: These four things, AI development, camera vendors, system integrators, and factories, will not mix unless nothing is done. Because it exists as a platform for properly gluing them together, it depicts a world where four things can go around if they are in the ecosystem.
Mr. Narusawa: Since we are an image sensor company, our main customers have been camera vendors, and it has been difficult to grasp how cameras are used after they become cameras. When we launched AITRIOS, we talked to our customers, who are actually end users, and found out that they were using it in this way and seeing what they were looking at. For the first time, we are connected to the question of how difficult it was to adjust the angle of view, whether such a jig is used, and if so, whether it should be prepared on the camera side. I believe that our value is that we are becoming able to provide such a place.
Source: Sony Semiconductor Solutions Group
Onishi: I think that's the ideal type. We are an AI vendor, and our strength is to provide one-stop services. We do everything from camera selection to installation, and finally we can automate the process. However, if the cooperation between various companies goes more smoothly, for example, it may be possible to automate everything in-house while promoting it by people in the manufacturing industry. I think communities and platforms have the potential to do that. Small and medium-sized customers who want to leave everything to us find it easier to use a one-stop service like ours, so we expect that demand will continue to grow.
Mr. Hayashi: What do you think is the biggest obstacle to building the ecosystem that you mentioned?
Mr. Narusawa: I feel that trust is important in the ecosystem. I've been doing new businesses for a long time, but I've always felt that it's easy to get into a company because the name of the company is known. When connecting various things, such as an ecosystem, Sony's accumulated trust becomes a weapon. I don't think this can be done overnight.
Onishi: It's not just about name recognition, is it? There is a sense of trust that the quality is properly considered, and that you can entrust it to us in our relationship.
Mr. Hayashi: Finally, I would like to talk about the expansion of business. In the past, I was involved in the development of sensors that can read the air like a human. The idea was to combine a lot of sensors to detect the situation like a human. Sony has the world's strongest sensor. From the perspective of sensor fusion, what kind of worldview do you want to create in 20 or 30 years?
Mr. Narusawa: Sony's strength is its technology for capturing light. I'm trying to capture not only the light that we see, but also a wider range of light. For example, capturing non-visible light, such as infrared and ultraviolet rays, which we usually cannot see, may open up new possibilities. Sony has developed a variety of sensors with very unique characteristics, and it motivates us to think about what we can do by incorporating these sensors into our solutions.
The IMX500 is an AI processing of RGB sensors, but the Pre-processing stage does not have to be RGB. It can be captured at a wider wavelength or at a distance, so I think that putting it into the AI processing in the later stage is also a type of sensor fusion. Of course, it is also important to say that interesting things can be done by combining it with other sensors other than light. In the long term, I would like to think about what we can do by connecting with sensors from other companies. However, it is not us who develop it, but our partner companies and end users. I think it's important to have a world where they can choose from a variety of options and create one.
Mr. Hayashi: Do you have anything from a developer's point of view?
Onishi: The best thing is to be able to make what you want to make in a short period of time. The difference in sound has already been combined with the image as a difference in vibration. By quantifying images from level 1 to level 10 and including sound data in chronological order, it is possible to understand things that were previously unknown by making it possible to handle them in parallel. The amount of data is so huge that it cannot be seen by humans, but it can be processed by AI. The more elements there are, the more things you can do, so I hope you can make it easier. It is not necessary to complete everything with AITRIOS alone, and I think it is good to connect with other platforms at the upper layer. No matter how far you go, I think that the eyes are still killer content, so I'm looking forward to seeing "seeing" delved deeper.
Toward the development of DX in the manufacturing industry
Mr. Hayashi: Could you give us a message that you would like AITRIOS to use in the context of manufacturing DX and that you would like us to help you here?
Mr. Narusawa: We want to expand our community and stakeholders. Compared to China, labor costs in Japan's manufacturing industry have declined and competitiveness has increased. In addition, in China, there is a way of thinking that "the warranty is only three years because the production equipment will be changed after three years," but in Japan, on the other hand, it will be used for 10 or 20 years, so there is a culture of making higher quality. As a situation, I think that the number of people is decreasing, and it is necessary to find out what can be done. Japan people are good at coming up with ideas and how to use them, and how to approach them when there are issues, so we want to help reduce the time and effort required to implement what comes to mind. We want to help you become a more open-minded manufacturing company.
Onishi: I think the manufacturing industry supports Japan. There are tens of thousands of parts for automobiles, and even small and medium-sized companies will have no more people and will not be able to make them, which will be a problem for BCP. There is an image that overseas companies make decisions faster for automation, so I think Japan should also move up one or two gears. Because there are fewer people, we have a strong desire to promote automation at all costs.
Hayashi: I think IoT is a technology that is suitable for Japan. It may be difficult to do it with software alone, but there is technology to create hardware that is bundled with software. In that regard, I am looking forward to the development of AITRIOS and its future developments.
Listening to everyone's stories, I felt that the Japan manufacturing industry is becoming more open-minded, and that it is trying to enliven the manufacturing industry as a whole by providing a system to support it and providing it horizontally. Thank you very much for your time today.
AITRIOS and its logo are registered trademarks or trademarks of Sony Group Corporation or its affiliates.
PEAKSMEDIA EDITORIAL TEAM
PEAKS MEDIA is a website that disseminates news articles and useful information that supports the transformation of the manufacturing industry and open innovation.
We will introduce a variety of "PEAKS" such as outstanding technologies, materials, people, and companies in each project, and deliver information that will support innovators in companies who are promoting reform.