Hello, everyone. Welcome to our webinar. Thanks for joining us today. My name is Vladamir Tsaganov. I'm a Solutions Architect Alibaba Cloud. I'm focusing on big data and artificial intelligence technologies in industry practices. Today, we're gonna talk about how we deal with analytics at scale, specifically how to empower your big data analytics with real-time data from IoT devices. Today's webinar agenda will be divided in three parts. First, we will discuss how big data and IoT technologies impact enterprises, and what is the key trends driving these changes.

Next, I will introduce a holistic IoT big data portfolio that we offer to our customers at Alibaba Cloud, and how it can help you address today's challenges. Lastly, we're going to discuss architectural principles and best practices, designing analytics solutions with IoT data. This session will be accompanied with demo of real time IoT data acquisition, storage processing, and visualization. Without further ado, I would start from the topic number one, which is big data and IoT, a new paradigm for the enterprises.

Today's enterprises have to survive in very complicated environment. They have various challenges come from ever-changing demand structure that is under major transformation of consumption behavior towards more personalized goods preference. Or another source of the challenges is the workforce. Aging workforce, new industry skill gap, major demographic changes, all these make a big impact to the modern workforce. 

Of course, there are a lot of concern that is relevant to the industry itself, such challenges as resource scarcity, access to capital, as well as new technologies that highly impact the modern industries. The new regulations protectionism and low efficiency of government at the certain countries or regions might create quite complicated conditions for the enterprises to operate. Another topic is the social environment challenge, and the social activities, climate changes, natural resource utilization, and all these limitations that is bring by the environment.

The last things about the challenges that come from the massive digitalization or business operations, the significant impact from disruptive technologies, such as cloud computing, big data, AI, IoT, and the other technologies make a great threats to the modern enterprises. So what is the key trends that drive the changes that necessary for the modern enterprises to transform and to adapt to the very aggressive environments. And so these three topics that more and more experts talking or discussing different forms, obviously is the implementation of the cloud technologies that were in transition of the new digital world, where we can obstruct the physical world into the digital one. Or the implementation of new concept of the internet of things, where we can perceive the physical world through digital world. And of course the fusion of the cloud and internet of things will change the human production practices.

So what is the internet of things or more commonly referred as the IoT? The conventional understanding is that the IoT is a network of physical devices, vehicle, home appliances, and other items embedded with electronics, software sensors, activators in connectivity which enables this objects to connect and exchange data. In more practical way, we can understand the concept of IoT as a technological approach to sense or perceive physical world translating an out data digital forum.

Such approach is really about getting access to information that wasn't previously available. The information is always around us, but most of the time it's not available as a data. So the one of the main goal of IoT is helping us to go from the information into the insight. So as we think about the industrial applications that I've already list of quite mature use cases that we can address as we start building IoT applications. There are many manufacturers that want to improve overall efficiency of their product line. They might suffer a lot if there is any downtime in their machineries. So they want to use data and apply analysis on those data to monitor performance, predict failures, provide convenient digitalization services and alarm operators in real time to weigh any such downtime.

Supply chain optimization and the real time assets striking is also a key use cases. Because you can use that not only avoid inventory stock out, but you also can predict demand. So you can make sure that there is enough inventory as the customer demand increases. And of course, per the quality energy material and cost efficiency optimizations are the key areas where a lot of our customers and partners have a big interest as well. IoT is already everywhere in our today's life with billions of devices, including our cars, our homes, airplanes, parking meters, any type of the variable devices or so-called smart factories with heavy machinery that is connected to the internet.

The IoT has potential to be the most disruptive technological advance in recent ages. According to research from IDC about certain billion things will be connected by 2020. Helping enterprises and governments drive efficiencies and launch new products and services. Today we've already see how IoT transformed from the high-level concept to the real life implementation. In various industry verticals, such as intelligence transportation, where we can remotely monitor and dispatch the interim operations of public transportation systems, or smart plants in factories, where we can optimize the production processes, leveraging the data generated by sensors or monitor performance of critical parts in real time.

In many, many more other use cases, including smart building, smart campus, environmental solutions, solutions for the water, energy, savings, hospitality, healthcare, engine of vehicles, and any other, a new concept that we are witnessing as of today. The next section of my presentation will be about how can we build such big data platform in order to process the IoT data at Alibaba Cloud? So with so many things connected IoT will drive an explosion of data that will need to be processed stored, managed and analyzed.

In some particular use cases we need to serve such data in real time. IoT will generate far greater volume, and variety of data than most information leaders are currently familiar with. Requiring volumization of information infrastructure, to realize such value. In Alibaba Cloud, as we think about this such transformations and the new use cases that come together with IoT technologies advancement, there are several key areas which we need to solve in order to address those use cases.

One is how to address the massive amount of data across different sites, and then be able to process it at scale and derive the key insights and business value from those learnings. And in Alibaba Cloud, we have a number of services that can help you build exactly that. Today we offer an end to end solution that enable our partners and customers to connect various devices to IoT platform and bring the data securely to the cloud, where they can apply various data analysis tools and consume it at their business applications.

Alibaba Cloud IoT platform is the one-stop solution for developing IoT applications. It helps our customers and partners self pinpoints of developing such applications and provide various features. For example, the platform supports stable and little latency by directional communication between devices and the cloud using using the IoT Hub. The other example is we can provide multiple protection mechanism to guarantee cloud security for devices. The platform also provides the device management functions like functional modeling, political analysis, and remote debugging to allow users to operate and maintain devices remotely IoT platform supports full device life cycle based on the open IoT services, helping the developer to simplify the process of application development.

If you think about the IoT data that you get from the sensors or devices, each of them might generate very small data, couple of bites per second or per few seconds. But when you look at it from the cumulated perspective, you generate a massive volume of data. And so we really need to solve the problem with the data ingestion skill. And to do that we propose to use IoT hub that can ingest data at scale. And essentially what it does is if we have IoT devices, which can publish the data, either directly or via gateway to a particular topic at IoT hub, then IoT hub can help consume that data from the topic for further storage processing analysis and other cloud services following the rules that we can simply configure at the rule engine.

Once we've ingested the data, the next things we need to do is really process it. And in case of IoT applications, often real time analysis, and the real-time insights become really, really critical. So we offer two great services for real time data processing. First, you can choose a StreamCompute, which allows you to constantly receive streams of data from various sources and continuously analyze it. You can use SQL to transform your data before you store it.

Other solution you might choose is designed based on the open source Apache Kafka, and Spark Streaming that is provided within E-MapReduce. E-MapReduce is our version of Hadoop in the cloud. If you need to process data in batch, we can offer a MaxCompute. MaxCompute is very powerful big data engine designed to process petabyte level of data. It comes together with online ID called DataWorks that can help you to ingest data to MaxCompute, manage your queries, assist with the governance framework and many, many more.

Also live Alibaba cloud offer high performance analytic DB for extremely fast data processing of massive amounts of data. For example, if we can use to establish new real time reporting practices. We also offer a solution to decouple computer sources for data processing and storage service to implement data lake framework. For that the best part like is obviously Data Lake Analytics that can connect to various type of the data sources and know its queries without the needs to establish very comprehensive data housing solution.

Lastly, you can enjoy managed services provided by Alibaba Cloud, it's relevant to E-MapReduce that has already told Hadoop in the cloud, and of course ElasticSearch. Apart from the cloud offerings that we have today, more solutions can be found in our marketplace. So we have the marketplace domestically that we provide in China. We also have on the marketplace for the international customers that can be accessed through the international marketplace.

So as of today, the marketplace offer or for thousands of their products in all of the more than 1200 ISVs available on the marketplace. The next section of my presentations, we will focus on architectural principles and best practices to design analytics with IoT data. Here, we provide a typical IoT application architecture. When you're developing such application, we would recommend to consider that kind of, what kind of sensors and devices you're going to use for your applications.

How those devices will be connected to the network, will they use IoT gateway? Can it relicense or alliances bad network? Also, as we've discussed today, that the importance of IoT connection and age is crucial. And today Alibaba Cloud already provides its own comprehensive one stop solutions. For that, the name is Alibaba Cloud IoT platform. When those basics established, we could define how we process and store our data. How it will drive the insights from that data. And lastly, how it will feed these data into our applications.

For example, for a full detections versus optimizations, predictive maintenance and the other services. Once we clarify our IoT application architecture, we can consider how we will implement it. On these slides, we provide an overview of the services that can be used to implement such applications. The following diagram you can see the devices can be connected to IoT platform directly or through the gateway. Then the data that they publish can be forwarded to IoT Hub to various cloud services. Let's start from the real time data processing.

For real time data, we can utilize at the hub. And forward such data, for the real-time process in StreamCompute. The clean data can be transferred to MaxCompute for batch processing and further machine learning analysis. Also, we can use data V for real time data visualization. We also provide serverless running environment for the data processing. For example, we parse IoT data before we send it for further real time or batch processing. Such use cases can be implemented using function compute for the data for processing.

Sometimes it is required to store petabytes of time series data from IoT devices and monitoring systems and process thousands of queries per second. Then table store for such storage is the best way to address that requirements. It's a big data analysis using SQL directory function, and efficient incremental streaming interface provide easy way for offline data analysis and real time streaming computing. The all data can be stored at our object story service, which can provide various types of the data storage, like standard storage and frequent access or archiving.

The combination of such storage types can help to optimize your data storage cost. It depends on type of the data and how often you would require it, you can choose different type of the services. Then these data that is stored in or excess can be transferred to MexCompute using data works. MaxCompute can be used in this use case as a main big data processing engine. Their all data will be uploaded to MaxCompute from, or asses or it can be loaded directly from the IoT hub or IoT platform, and processed using various specific functions, such as SQL, MapReduce, Graph and UDF. UDF is the user defined functions, and if you're not satisfied with the default functions, you can use various type of the is the case to develop your own.

In order to analyze data using machine learning, we can offer platform of AI. So data suggestions that the generated based on the analysis conducted inside the PI. Then processed data will be stored at the MaxCompute. The processed data can be visualized using the QuickBI or frameworks inside Alibaba Cloud environment using our online ID that called DataWorks. After we studied the typical IoT cloud applications, I would introduce several typical use cases. One of the first use case is bicycle sharing solution.

In this example, what we've done to our customer is we help them to connect millions of bicycles, shared bicycles around the city into the cloud. So you can see that using IoT is the key, they can configure a business logic, and send the data into the clouds through the IoT platform. Then this data will be sent to Alibaba Cloud analytics products for big data analysis, storage's, real-time computing and data visualizations. Also the services is connected to the business applications like bike positioning, remote unlocking after sale services, metering and billing, bike reservations, et cetera.

Another example is the energy monitoring. When the power station uses the IoT platform to collect data from in Virtus and other devices. The data is then processed through the ETL and stored and visualized before using machine learning techniques to predict the performance of these devices. The last typical use cases that I would introduce today is the smart city solution. As in previous use cases you can see that different type of devices for infrastructure, urban digitalization, telemetry, unified sensor network, communication open compatibility, all can be connected to the cloud using Alibaba Cloud IoT platform. Then leveraging the newly limitless capabilities of cloud computing we can develop various type of software as a service applications, and deploy it in various cities, districts, or even neighborhoods.

After we studied the typical use cases, we can consider how to really implement IoT projects. So from the more strategic point of view, the implementation practices for in the IoT project is very similar to the implementation of the standard IoT projects or where they come in to start from this site survey and requirements analysis, in order to clarify business needs and project objectives, to clarify the hardware and software specifications, functional requirements, data and a network security.

Then we need to move into solution design and development where we need to consider how devices will be connected. What kind of approach we're gonna use for the data collection. How are we gonna compute and store the data? How are we gonna build data applications for the analytics and modeling? Then we have to design the solutions for data consumptions through visualizations and optimizations that can enable business users to make a proper decisions based on the data that they have on their hands. Lastly, leveraging such new data sources as the IoT devices, we can conduct certain actions for automatic control, according to the quality results, manual operations by the optimized scale. And this tab will look in our circle when you need to run it over and over in order to polish your IoT project.