Fogs on a Cloud

Fogs on a Cloud

Fogs and Clouds 02/12/2020 Fog Applications on Cloud 1 Motivation Fog Architecture that allows connected IoTs to the cloud Wearable sensors or sensors embedded in environment Data collection, storage, condition monitoring E-health, traffic monitoring, industrial environment, home environment monitoring, IoT assisted living Fog stands between IoT and the Cloud is the ideal environment for the Fog Fog implementation range from a small device or server to

another virtualized infrastructure Probably a cloud eg. SIXSQs nuvlabox Bring anything you like at edges of the network 02/12/2020 Fog Applications on Cloud 2 Fog Applications IoT assisted living using motion sensing devices Rehabilitation for patients who suffered injury or operation: data are collected by sensors installed at home, analyzed and evaluated on cloud, provides feedback to patients by physicians Patient monitoring without presence or caregiving personnel Advantages: Safer environment for patients and caregivers, patients feel safe,

protection from accidents (i.e. elderly falls), reduced hospitalization and costs, increase patient autonomy and confidence Personalization of health care Independent living Combined with cloud advantages (scalability, low maintenance costs) 02/12/2020 Fog Applications on Cloud 3 Generic Architecture 02/12/2020 Fog Applications on Cloud 4

Reference Architecture REStful, Event-based architecture, FIWARE 02/12/2020 Fog Applications on Cloud 5 Service Roles Producers: sensors, sensor owners (patients) that generate data Consumers: end-users that subscribe to data (caregivers) Back-End (BE) : cloud services, authentication, data context subscription, event management, data storage, data analysis, system control, application-business logic, message handling and data exchange

Front-End (FE): gateways (mediators between sensors and network) that collect data from sensors, local data context management (filtering), data encryption, protocol adapter, data exchange 02/12/2020 Fog Applications on Cloud 6 Front-End (FE) Services IoT Connectivity (FE): connects sensor with application components, protocol adapter Protocol Adapter: Converts protocol specific data (e.g., BLE, RFID etc.) to common format (e.g. XML or JSON) Sensor Data Collector (FE): Collects, stores (temporal store) and forwards sensor data to cloud when connection is available Data Filtering: not all data are sent to cloud to save bandwidth and

cost Event processing: Emergency handling, message handling Connectivity Service (FE): establishes connection with cloud (BE), sends and received data from cloud, data encryption In Native Android, iOS code or convent with PhoneGAP 02/12/2020 Fog Applications on Cloud 7 Back-End (BE) Services Identity management: user authentication/authorization Application Logic: application specific, business logic e.g., decision making Publish/Subscribe Context broker: receives sensor measurements from Application logic, connects data providers with data consumers on a publishsubscribe way Complex Event Processing: analysis of complex conditional events (e.g., if pulse rate > 120 and SPO2 < 92 generate alert), works with Context Broker who decides

flow of data based on user subscriptions Cloud Storage: stores, retrieves data for users, sensor measurements, rules and events, users (permanent storage), keeps log files, handles stream data, big data Data Analytics: big data analysis, uncover hidden patterns in data, unknown correlations, market trends, customer preferences and other useful business information (allows users or companies take better business decisions) Connectivity Service: Establish connections with Front-End 02/12/2020 Fog Applications on Cloud 8 Wearable Sensors Wearable sensors for continuous and fast communication of vital data (blood pressure, heart rate, SP02 , temperature, blood glucose): vital measurements are sent to cloud, they are analyzed, abnormal values trigger actions (e.g. Take your medicine, contact

physician) Motion sensors capture body motion or hand gestures that produce signals signals to cloud signals to commands or actions or assist peoples communication (people with speaking disabilities Victory" GESTURE LEAP MOTION SENSOR 02/12/2020 INTERACT Fog Applications on Cloud: https://www.leapmotion.com RESULT 9 Motion Sensors

Sensors that can track the human body or parts of it (skeletal tracking) Asus Xtion Pro Leap Motion 02/12/2020 Fog Applications on Cloud MS Kinect 10 Kinect Sensor Originally designed as an addition to Xbox360 Combines input from

Color VGA video camera: for face recognition, motion detection Depth sensor: sees room in 3D regardless of lighting conditions Multi-array microphone: isolates voices of the players in the room 640x480 pixel resolution, 30fps, 1.8m play space Detects and tracks 48 points on each players body mapping each body to skeletal structure including facial details Each body creates endless combinations of points/features actions by classifying skeletal movements of each model emphasizing the joints and distances between the joints Supported by machine learning for training the system to react, map data to models representing people of different ages, genders, body types, even if they are partially hidden 02/12/2020 Fog Applications on Cloud: https://developer.microsoft.com/en-us/win dows/kinect

11 Hospitalized Patient Monitoring Kinect monitors left and right shoulders and positions of wrists, set thresholds on shoulders/wrist positions The patient is trying to get off the bed: if limit is reached effort to get off from either left or right side of bed application notifies personnel The patient requires assistance: once position of wrists reaches the threshold alert is generated as a request for assistance

02/12/2020 Fog Applications on Cloud: http://www.intelligence.tuc.gr/~petrakis/p ublications/Cloud4ELE.pdf 12 Rehabilitation A user is suffered a knee injury or operation he is now at rehabilitation phase Use Kinect to perform an exercise without being present in the physiotherapy center but in his own space Can be connected with physiotherapist elsewhere

Patient lifts leg off the ground kinect records maximum height notifies patient, caregivers review motion attempted provide feedback 02/12/2020 Fog Applications on Cloud: http://www.intelligence.tuc.gr/~petrakis/p ublications/Cloud4ELE.pdf 13 Gateway-Fog service for Management of Sensor Signals Gateway services (FOG services) that support data management and FOG node capabilities Running on tablets, smartphones (Android, IOS) Portable i.e. generic access for all BLE sensors and dynamic sensor

registration (Sensor = XML schema) Automatic discovery of BLE sensors Transient storage Data encoding (encryption, compression) Data filtering (for reducing data sizes, processing on cloud) Real time data transmission to cloud through internet Modular services for all key functionalities, SOA architecture 02/12/2020 Fog Applications on Cloud: http://www.intelligence.tuc.gr/~petrakis/p ublications/SARNOFF2016.pdf 14 Healthcare and IoT

Transmitting and processing vital user data (from BLE sensors) Remote patient monitoring in real time Data analytics (offline) 02/12/2020 Fog Applications on 2016 Cloud IEEE Sarnoff 15 IoT data collection Real time data collection Generic and universal device recognition for BLE devices

On-the-fly and fast data collection and transient storage 02/12/2020 Fog Applications on 2016 Cloud IEEE Sarnoff 16 Bluetooth Low Energy (BLE) Bluetooth low energy is a new, open, short range radio technology Different to Bluetooth classic Optimized for ultra low power Enable coin cell battery use cases

< 20mA peak current < 5 mA average current 02/12/2020 Fog Applications on 2016 Cloud IEEE Sarnoff 17 Bluetooth Low Energy (BLE) The standard to support low latency/low energy and short-range networks (less than 50m) BLE defines GAP mechanism and two protocols (GATT, ATT)

Generic Access Profile (GAP) defines the mechanism for BLE devices to communicate with each other Makes a device visible and allows other devices to connect with other Controls connections and advertising and discovery process Once discovered the peripheral will stop advertising itself, can be connected to one central device at a time Generic Attribute Profile (GATT): defines how data are formatted and exchanged (two-way communication) GATT uses a Generic Attribute Protocol (ATT) that defines services, their characteristics (e.g. sensed values) and ids (16-bit ids) in a table 02/12/2020 Fog Applications on Cloud 18 Generic Access Profile (GAP)

Controls connections, advertising, makes devices visible and determines whether two devices can interact with each other Defines roles for devices Central: more power, smart phones, tablets Peripheral: small, low power (e.g. sensors) A peripheral devices issues a discovery process, sets time interval (power consumption?), broadcasts id every time time interval passes, may send data to more than one device if listening device is interested, connection is established, advertising process will stop and will start using GATT services 02/12/2020 Fog Applications on Cloud 19

BLE Network topology 02/12/2020 Fog Applications on Cloud 20 Services and Characteristics High level, nested objects defining: Profiles: collection of services defined by designers (e.g. heart rate profile) Services: data breakdown into entities e.g. Heart Rate Measurement, Body sensor location, Heart Rate Control Point

Characteristics: Numeric IDs called UUIDs e.g. values in the form of (X,Y,Z) values from accelerometer or GPS 02/12/2020 Fog Applications on Cloud 21 Roles and Functionalities Producers: users and their devices Front-End (FE): data generator point and related functionalities on gateway like sensor data collection from BLE devices, connection to cloud, transient storage Back-End (BE): public cloud services, system logic and related functionalities on public cloud like storage, connection to gateway,

context services, user authentication/authorization, permanent storage Consumers: users entitled to access data in the cloud (e.g. Caregivers) and interact with FE users Administrators ie., users entitled for system set-up, register users and sensors, define context services 02/12/2020 Fog Applications on Cloud 22 Reference Architecture 02/12/2020 Fog Applications on Cloud

23 Flow of information 02/12/2020 Fog Applications on 2016 Cloud IEEE Sarnoff 24 Sensor Registration Each sensor is described by and XML schema stored in

Sensor Data Base, declared once, once declared it can be reused Protocol adapter establishes connection with sensor, converts sensor data to XML, JSON according to XML Schema in sensor database Administrators can access, define, modify XML sensor schemas Nonin Pulse Oximeter: saturation of O2 in blood, pulse rate Polar H7 heart rate sensor 02/12/2020 Fog Applications on Cloud 25 NONIN and Polar H7 XML Schema 02/12/2020

Fog Applications on Cloud 26 XML Tags : This tag includes the BLE devices that are registered into the system : A specific sensor which is characterized by its category (e.g. Polar H7). : Indicates the measurement type and the measurement units : It is the format of produced values (e.g. FORMAT_UINT8 that is an 8-bit integer) & : In some cases, BLE sensor manufacturers define complicated function for decoding their features 02/12/2020

Fog Applications on Cloud 27 Front-End 02/12/2020 Fog Applications on Cloud 28 System Functionality (front-end) User Interaction: He/she only has to wear a portable BLE sensor and carry a mobile device. The sensor is automatically paired to mobile device. Then the service can monitor sensor data in real time. Data forwarding to the Cloud: Device Name identifier, Measuring

Time, Triggered by rules: When a rule is violated, the service activates the automated measurements per second. Monitoring measurements in real time to the Cloud Messaging to user: GCM forwards messages to the user in real time Rules for mobile devices: The service transfers the rules from backend to Front-End based on an HTTP post request Uses an appropriate identification code to get rules and sensors related to the specific user. 02/12/2020 Fog Applications on 2016 Cloud IEEE Sarnoff 29

Database in the Front-End SLQLite database engine in Android It includes data encryption and decryption is a component that runs on top of local storage for decoding/encoding data based on the Advanced Encryption Standard (AES). SQLITE database tables user device user rules sensor measures asset folder 02/12/2020 Fog Applications on 2016 Cloud

IEEE Sarnoff 30 Local storage service Insert/update/ delete /query data Pushes data to application logic Database Manager Data forwarding after encryption / Datadecryption

Encryption /Decryptio n Pushes data to sensor data collector User Device s User Rules User Sensor Data

SQLITE-DB Sensor XML Schema Updates sensor XML schema Asset folder 02/12/2020 Fog Applications on 2016 Cloud IEEE Sarnoff

Responsible for storage and retrieval Encoding / decoding of sensitive critical data. Forwarding information relevant to other sub modules. 31 Protocol Adapter Implements BLE functionality

GAP advertising & discovery process GATT/ATT exchange of information 02/12/2020 Fog Applications on Cloud 32 Sensor Data Collector Collects the sensor data Converts data into the desirable format (e.g., JSON, XML etc.) and schema so it can be processed in the back-end without causing interoperability problems

02/12/2020 IEEE Sarnoff 2016 33 Data collector Protocol Adapter Service Scans for devices Device Scan Activity Convert it to Java object

DOM Parses Gets sensors XML schema from local storage service Connects to sensors MAC address Establishes GATT connection BLE Service Pushes sensor measurement to application logic ( in JSON format)

DOM parsing converts Sensor Data to Java Objects Transfers the data in JSON format to the module logic of the application system 02/12/2020 Fog Applications on Cloud 34 Connectivity Service Establishes a connection between the frontend and the back-end Transfers the sensor data received from the Sensor Data Collector to the Back-End Encryption/Decryption of information flow Transfers the data from the device to the cloud(and vice versa) via HTTP requests to the server

Data are encrypted and decrypted (AES 128) 02/12/2020 IEEE Sarnoff 2016 35 Notification service Real time information exchange between front / back end Update when situation changes (Toast messages) (i.e. no network thus it enables Local Storage). Messages may be broadcasted, or the messages may be personalized. Application logic mechanisms Management of rules Processing records coming from users (i.e. date, timestamp,

patient status) Change data forwarding - ON TIME INTERVAL (default), ON DEMAND, VIOLATION RULE 02/12/2020 IEEE Sarnoff 2016 36 BACK-END (CLOUD) 02/12/2020 Fog Applications on Cloud 37 02/12/2020

Fog Applications on Cloud 38 Back-End (cloud) Registration of a new sensor: An XML schema describes each sensor Registration of a new user: The system administrator uses the authentication mechanism to register new users to the service. Request to the Google Cloud Messaging (GCM) service. User and device registration in Cloud: The service uses local storage (JSON format) for storing data associated with a user Sensor assignment and user rules: in a healthcare scenario, medical personnel can define sensor category (e.g. Nonin pulse oximeter ), and set lower and upper values for sensor measurements, thus to be notified when these values are violated using GCM service.

02/12/2020 Fog Applications on 2016 Cloud IEEE Sarnoff 39 Identity Management Used for user authentication and access authorization. Users and Developers authorized to access applications and services

02/12/2020 IEEE Sarnoff 2016 40 Application Logic Application specific module. Encapsulates the business logic of the FIapplication. Handles and processes sensor data by using the Complex Event processing, Cloud storage and Publish subscribe modules 02/12/2020

IEEE Sarnoff 2016 41 Publish Subscribe Responsible for publishing the results of the sensor data processing. Publishes context to context subscribers. Example: 02/12/2020 IEEE Sarnoff 2016 42 Complex Event Processing Used for decision-making thorough the analysis

of complex conditional events. Processes custom event patterns Based on user defined conditions, decides the flow of the data. Combines data from multiple sources for event management 02/12/2020 IEEE Sarnoff 2016 43 Cloud Storage Responsible for storing or retrieving sensor data using a database. Functionality should be

offered as a RESTful service (JSON Storage GE). Developers should be able to create or retrieve user data from the database 02/12/2020 IEEE Sarnoff 2016 44 System Evaluation The experimental configuration includes a gateway service that is installed in a smart phone device (running Android) The back-end services are deployed on OpenStack and on the FIWARE platform Methods: The method user registration to register a new user and

his/her gateway device to the Cloud. The method get rules retrieves rules from the Cloud using the appropriate HTTP post call. The message notification method sends messages through GCM to the device. 02/12/2020 IEEE Sarnoff 2016 45 Measurements Average of 100 method exec. 02/12/2020 Fog Applications on Cloud

46 Storage VM performance We deployed the MongoDB in OpenStack using a virtual machine (VM) with 4 CPU Cores, 4 GB RAM and 40 GB hard disk. We measured the utilization levels of the VM with regards to the CPU percentage, memory percentage and disk usage percentage. We calculated the average run for 1 hour. We measure Mongo DB performance for 100 users with 2 devices each that produce 3 records per second for medium flavor OpenStack VM 02/12/2020 IEEE Sarnoff 2016 47

VM Measurements 80 70 60 50 40 30 20 10 0 02/12/2020 CPU VMem percent Disk usage percent Swap percent IEEE Sarnoff 2016

48 Summary Intelligent gateway on native android for patient monitoring Easy to use cloud service deployed in OpenStack Generic BLE sensor connection Fast data forwarding to the cloud It supports a high number of concurrent users while a medium flavour VM can manage the storage needs for the 2 selected sensors 02/12/2020 IEEE Sarnoff 2016 49

References https://learn.adafruit.com/introduction-to-bluetooth-low-e nergy/ Theodoros Soultanopoulos, Stelios Sotiriadis, Euripides G.M. Petrakis, Christianna Amza," Data Management of Sensor Signals for High Bandwidth Dat a Streaming to the Cloud", The 37 th IEEE Sarnoff Symposium 2016, New Jersey Institute of Tech nology Conference Center, Newark, New Jersey, USA, Septe mber 19-21, 2016. Theodoros Soultanopoulos, Stelios Sotiriadis, Euripides G.M. Petrakis, Christianna Amza, " Internet of Things Data Management in the Cloud for Blueth 02/12/2020 Applications on Cloud ooth Low Energey (BLE)FogDevices",

Workshop on Adaptive Re50

Recently Viewed Presentations

  • Contributing Factors - Units 3 &amp; 4 Psychology

    Contributing Factors - Units 3 & 4 Psychology

    The power of being labeled as 'schizophrenic' was so powerful, that any behaviour by the participants was seen as part of the . psychopathology. of their illness - the scientific and systematic study of abnormal experience, cognition and behaviour; or...
  • DNA, RNA and Protein

    DNA, RNA and Protein

    Translation How the Genetic Information Is Used to Build a Protein DNA Information Flow From DNA initial transcript RNA transcription Protein translation replication processing mature RNA Properties of the Genetic Code Triplet 3 nucleotides code for 1 amino acid Non-overlapping...
  • CHAPTER 6 OXYGENATION NEEDS LANCASTER HIGH SCHOOL MRS.

    CHAPTER 6 OXYGENATION NEEDS LANCASTER HIGH SCHOOL MRS.

    CHAPTER 6 OXYGENATION NEEDS LANCASTER HIGH SCHOOL MRS. CARPENTER care of the person with a chest tube prevent the tubing from becoming kinked observe chest drainage and report increased amount bright red drainage bubbling activity increase, decrease or stopping care...
  • 2012 Commissary Council Update ALA/DeCA Roundtable May 2012

    2012 Commissary Council Update ALA/DeCA Roundtable May 2012

    Outreach Initiatives. Child Development Center Tours. Consumer Education/Recipes. Single Service Member Focus. Produce Roundtable Initiatives. Inventory Procedures. USDA Inspections. Produce Mgr Turnover. 2012 Council Committee Initiatives . 40-15/40-16/40-17 forms . Matrix Deep Dive. Gathers Matrix input. Works input with DeCA....
  • Unit 2 Displaying Univariate Data

    Unit 2 Displaying Univariate Data

    Means & Outliers. Are there any values that appear to be outliers in our fuel economy data? If so, find the mean without the outlier. The single outlier adds 2 mpg onto the mean highway mileage of 2004 motor vehicles....
  • 355 Geoscience for Elementary Educators

    355 Geoscience for Elementary Educators

    355 Geoscience for Elementary Educators Author: Sciencea2z Last modified by: Sciencea2z Created Date: 3/17/2008 6:43:22 PM Document presentation format: On-screen Show Company: Science From A To Z Other titles: Times New Roman Arial Calibri Constantia Wingdings 2 3_Flow 2_Flow Lets...
  • Delivering NATS ATM CO - The Airport Operators Association

    Delivering NATS ATM CO - The Airport Operators Association

    NATS Environmental Targets. NATS PRIVATE. 2008 - NATS is first ANSP to set a specific ATM CO. 2 Target. Reduction of ATM carbon dioxide (CO. 2) by anaverage 10% per flight by 2020(versus 2006 baseline). Equates to 2.49 million tonnesCO....
  • Accelerated Reader Questions &amp; Answers

    Accelerated Reader Questions & Answers

    Accelerated Reader Questions & Answers Power Academic and Performing Arts Complex What is Accelerated Reader (AR)? A computer program designed to encourage student reading AR tests comprehension using multiple choice quizzes Computer software provides information about student performance to teachers...