Please visit my website and subscribe to my youtube channel for more articles
Real time Data Streaming
Ability to process and analyze data in real-time is essential to do things such as continuously monitor your applications to ensure high service uptime and personalize promotional offers and product recommendations
Batch Processing vs Real time Stream Processing
Consider the monthly billing scenario. By analyzing a customer’s usage data as it is generated, an organization can enable valuable features, such as notifying users that they’re approaching a pre-defined billing limit. If the IT application logs can be analyzed in real-time, a system administrator can be notified immediately to investigate and take corrective action.
Amazon Kinesis Firehose Amazon Kinesis Firehose is the easiest way to load streaming data into AWS. It can capture, transform, and load streaming data into Amazon Kinesis Analytics, Amazon S3, Amazon Redshift, and Amazon Elasticsearch Service, enabling near real-time analytics with existing business intelligence tools and dashboards that you’re already using today
Amazon Kinesis Agent The Amazon Kinesis Agent is a stand-alone Java software application that offers an easy way to collect and send data to Kinesis Streams and Kinesis Firehose.
Data Transformation In some scenarios, you might want to transform or enhance your streaming data before it is delivered to its destination. For example, data producers might send unstructured text in each data record, and you need to transform it to JSON before delivering it to Amazon Elasticsearch Service. To enable streaming data transformations, Kinesis Firehose uses an AWS Lambda function that you create to transform your data.
Kafka vs Kinesis: Comparing Real-time Data Streaming Tools
Managed streaming data capabilities offered by Amazon Kinesis, or deploy and manage your own streaming data solution using Apache Kafka. Click here for more details
