The IoT devices, including smart phones and wearables, can be applied in a plethora of applications ranging from building automation and industrial systems to self-driving vehicles and health services. The distributed and growing usage of the connected devices deliver the users more responsive and intelligent support for decision making in a given environment.

The foundation of AI is based on data fed to algorithms for machine learning (ML). They require a lot of processing power due to the amount of data and recursive/concurrent nature of calculation. Until recently this has been accomplished mainly in the cloud environment, where the raw data is uploaded into. This exposes all the data, even private and sensitive data, to the transmission phase and processing system. In conjunction with IoT there is a possibility to perform ML closer to the origin of data concerning local intelligence. It means that only the results of local or edge ML are transmitted to cloud for more general aggregation of AI. Local systems do not need to send the raw data anymore, which helps on prevailing the privacy and security of the data. This type of ML is referred to as federated/collaborative learning (FL).

This study focuses on finding the existing and/or recommended solutions for up-to-date AI close to the devices. At first, the definitions of devices are reviewed in order to find out classifications of their capacity to contribute for the computation and scalability. Secondly, the other computing and serving options between the devices and the cloud are studied. Thirdly, the facts learned are being applied in two use cases in order to support the discussion and applicability of AIoT in practice.

The main conclusion is that there are no silver bullets for solving all the requirements. Instead there are multiple options from mutually connected devices via middle layer support to cloud services, and distributed learning, respectively.