Amazon EventBridge Pipes: Simplified point-to-point Integration for Event-Driven Systems

In modern event driven microservice based applications where services are loosely coupled, communication between the decoupled services requires integration code. Maintaining the integration code is a challenge. Amazon EventBridge Pipes is a new feature of Amazon EventBridge that makes it easier to build event-driven applications. It provides a simple & and cost-effective way to create point-to-point integrations between event producers and consumers. Amazon EventBridge Pipes helps to reduce the amount of integration code that needs to be maintained for event-driven applications.

An EventBridge pipe setup consists of 4 steps – Source, Filtering, Enrichment & Target. Filtering and Enrichment are optional.

The source can receive events from various AWS services like SQS, Kinesis, DynamoDB, etc… In the filtering step filter patterns can be configured to filter the events that are passed on to the enrichment or target step. This helps to reduce the cost by filtering out unnecessary events that need not be processed. In the enrichment step, data from the source can be enhanced before sending it to the target. Built-in transformations are available to enrich data or AWS Lambda, API Gateway, or Step Functions can be used to perform advanced transformations on the data. Enrichments are invoked synchronously. Pipes can send the event/data to targets such as AWS service (AWS Lambda, API Gateway, ECS Cluster, CloudWatch Log, Kinesis Stream, etc..) or an API destination. Transformers can be written in the Target step to define how data needs to be sent to the targets. EventBridge supports invoking the targets synchronously or asynchronously based on the target.

Pipes can be activated or deactivated to process the events based on the need. CloudTrail and CloudWatch can be used to monitor the EventBridge Pipes. CloudTrail tracks the EventBridge Pipe invocations & their details. The health status of the Pipes can be monitored using various metrics supported by CloudWatch. Overall, EventBridge Pipes provides a simple, fast, cost-efficient way to configure advanced integrations with enhanced security, reliability, and scalability out of the box.

WHAT TO EXPECT FROM AWS RE:INVENT 2021

What To Expect From AWS reInvent 2021

AWS re:Invent 2021 is an Amazon Web Services annual technology conference scheduled for November 29 through December 3 in Las Vegas, Nevada. This is the largest annual Amazon Web Services (AWS) conference for partners and customers. The event is scheduled to include 1,500 technical sessions, a partner expo, training and certification opportunities, and multiple keynote announcements. Also, AWS re:Invent 2021 is dedicated to cloud strategies, IT architecture and infrastructure, operations, security, and developer productivity with a focus on AWS products and features. Irrespective of whether you are an engineer, a business leader, or just embarking on the cloud journey, this is an opportunity to discover everything the event has to offer. As an AWS Advanced Consulting Partner and Managed Service Provider (MSP), Idexcel is much excited and looking forward to this event.

Key Note Sessions

Keynote Session 1: Global Partner Summit – November, 29 (3:00 PM – 5:00 PM PST)

Doug Yeum – Head of AWS Partner Organization: Yeum is currently AWS’ head of worldwide channels and alliances. His responsibilities included engaging with c-level executives about their digital transformation and innovation strategies, developing business through channels & alliances, and managing business operations.

Sandy Carter – Vice President for Worldwide Public Sector Partners and Programs at AWS: She is responsible for driving next-generation partnering and evolving partner models to intensify partner innovation and AWS Cloud adoption.

Stephen Orban – General Manager of AWS Marketplace and Control Services: Orban formerly served for three years as general manager of AWS Data Exchange, a service launched in 2019 that makes it easy for AWS customers to find, subscribe to, and use third-party data in the cloud through Marketplace.

Keynote Session 2: AWS Customers, Products, & Services – November, 30 (8:30 AM – 10:30 AM PST)

Adam Selipsky – AWS CEO: Adam would take the stage to share his insights and latest news about AWS customers, products, and services.

Keynote Session 3: Databases, Analytics, & Machine Learning– December, 01 (8:30 AM – 10:30 AM PST)

Swami Sivasubramanian – Vice President at AWS: Heading up all Amazon AI and Machine Learning services and leading on all aspects of Machine Learning, from ML frameworks and infrastructure to Amazon SageMaker and AI services.

Keynote Session 4: AWS & Cloud Infrastructure – December, 01 (3:00 PM – 4:30 PM PST)

Peter DeSantis – Senior Vice President of AWS Global Infrastructure and Customer Support: Leading the AWS teams responsible for designing the data centers, servers, and network that underpin AWS services and for deploying and operating this infrastructure worldwide.

Keynote Session 5: Future of Software Development – December, 02 (8:30 AM – 10:30 AM PST)

Dr. Werner Vogels – Chief Technology Officer at Amazon.com: Dr. Werner is responsible for driving the company’s customer-centric technology vision and is one of the forces behind Amazon’s approach to cloud computing.

Offerings of re:Invent 2021

Here is a lineup of some of the expanded offerings of the event that you may want to check out:

Presentations

  • Keynotes from AWS on cutting edge-technology and industry trends
  • Leadership Sessions on a range of hot topics
  • Breakout Groups and Q&A Sessions
  • Expert Learning Lounges
  • Builders’ Fair with Presentations and Question & Answer Sessions
  • AWS Product Announcements

AWS Training & Certification

  • Exam Readiness Sessions
  • Remote Certification Examinations
  • Hands-on Labs & Bootcamps

Hands-on Learning & Fun

  • AWS DeepRacer Competitions
  • Jams, Game Days & Hackathon
  • Virtual Play Events

Performances & Entertainment

  • Cooking Demonstrations
  • re:Play – a Live Concert by a National Act

Our AWS re:Invent 2021 Predictions for ML Services

  1. Release of new generation ec2 instance for faster Machine Learning Training and Inference, which will offer a better Price-Performance Ratio.
  2. Amazon Textract will soon penetrate the market by providing extraction solutions that are domain-specific, covering specific types of document extraction solutions. We may see examples of specific types of documents that will be extracted.
  3. As we are in the midst of guiding ourselves through the pandemic, we will be aware that 2020 was a big year for medical image processing, particularly for chest x-rays. In this regard, we are expecting Amazon to announce a service similar to “Lookout For Vision” which will be used for anomaly detection, specifically focused on radiology-based images.
  4. An enhancement to Amazon Health Lake is expected to augment patient tracking and management.
  5. With the recent acquisition of Wickr, improvements in Lex are likely to be out later this year or early next year.
  6. A range of Automation options within AWS Service is likely to be announced.
  7. Furthermore, we look forward to big announcements of several incremental improvements to Amazon Rekognition service offering and additional flexibility in Machine Learning Training Models for images and videos.

Schedule a meeting with our AWS Cloud Solution Experts and accelerate your cloud journey with Idexcel.

How To Build Business Intelligent Chatbots with Amazon Lex


Enabling Business Intelligence in Chatbots with Amazon Lex

In this fast-paced digital age, organizations need a fast and efficient way of gathering information. Especially in a customer-driven market like fintech, “time is money ”. Decisions will have to be made accurately and fast. Incorrect decisions can lead to severe consequences or lost customers. In several fintech applications, information is made available through reporting solutions, presentations, charts, etc. What customers find difficult is digging out the specific report or data needed through a multitude of mouse-clicks and then spending a lot of time analyzing them. There is a critical need for one central point from which a variety of data can be delivered to the user in an efficient and effective process. AWS technology and tools open several avenues to make this possible.

Amazon Lex – Machine Learning As a Service

Amazon Lex is one service that enables state-of-the-art chatbots to be built. It has redefined how people in the industry perceive building chat-bots. Bots themselves have gradually evolved from typical question-answering bots to more complex ones that can perform an array of functions. Amazon Lex offers features that tackle several complexities faced while building the previous generation of chatbots. The intent fulfillment, dialogue flow, and context management features of Amazon Lex help to make conversation with a chat-bot as human-like as possible.

This blog discusses how information can be retrieved from databases with a simple question asked to Kasper (the name of our bot). The following components of this blog will give a clear understanding to the user, how everything is built, networked, and coupled with a custom user interface.

Solution Architecture

Kasper is a chatbot built specifically for a lending platform to retrieve various data points based on specific inquiries. Like all bots, Kasper is also built on intents, utterances, and slots. After adding intents, its corresponding utterances, and slots, a few slots need to be added as custom slots. For example, there was a query – “show clients where invoice amount is greater than 20000”.  In the utterance section of Kasper, it was recorded as below:

 

Here ‘cola’ and ‘operatora’ are slot variables under custom slots ‘columnname’ and ‘operator’ respectively.

Natural Language to SQL Conversion

All the responses that require output from the database are sourced with the help of a lambda function. The JSON response from the lambda function contains the input transcript, intent, and slots information. The back-end application then receives the response from the lambda function, segregates the JSON, and classifies information into the corresponding intent and slots. The application extracts the slots and intents and then proceeds to build the query.

Responses from Kasper

Responses from Kasper can result in different formats of data. There can be single value responses, images, tables, etc. The types of responses are automatically determined from the intents. A custom website with a chat window has been developed for interacting with Kasper. The chat window can take in both texts, as well as audio inputs. The following are the detailed sections explaining each response type, with their corresponding chat window.

 

Response type I – Single values

There are instances where users might want to know about a sum or count or any other single value response. For example, an inquiry might be “count the number of clients whose due date is within 2 weeks” or “sum of the invoice amount of all clients“. The responses of these queries will be just a single value eg. “10,000”.

Response type II – Images and Tables

1. Tables

Images and tables are the next type of responses Kasper delivers. Once the SQL query is constructed, it connects with the database and retrieves data and stores it in a pandas dataframe. This dataframe can be exported as an html table for previewing through the chat window. It can also be downloaded in the form of a csv file.

2. Images

From the pandas dataframe, different charts/graphs can be derived. When an image response is expected, charts are generated using python libraries, saved to a file, and then exported to the chat window. Two types of images are generated – one is a thumbnail and the second is the actual image. Kasper is equipped with a feature named Auto-visualization. According to the dataframe, the function will decide what type of graph or chart has to be plotted. There are numerous rules applied before making that decision. For example, the function determines whether a specific column features continuous or categorical values. The resulting graph is plotted based on such combinations.

Response type III – Fallback mechanism with response card

The third type of response are response cards – a response to clarify the intention of the user. Suppose the user asks an ambiguous question like this “what is the amount of Apollo Inc. “. The chatbot will find the query to be missing some keywords because the user did not specify the type of amount (either invoice amount or balance amount). Kasper then prompts back with a list of possible options, so the user can select the appropriate option and receive the accurate result.

Kasper is a chatbot that has evolved to its current operational capabilities because of maximizing Amazon Lex’s potential and accommodating other significant AWS services to its architecture. Currently, Kasper can solve important natural language to SQL problems and a few FAQ questions as well. It can also be modified for other domain problems to suit specific needs. Over time, more capabilities will be possible to add and could serve as a first-line substitute for human support personal, freeing up your support team to help address more critical issues more quickly. If you’re interested in how a chatbot might improve your operations, schedule a Free assessment with our Machine Learning team today.

Want to learn more?

AWS re:Invent 2019 – Global Partner Summit Announcements

AWS re:Invent 2019

1. Introducing AWS Retail Competency Partners: AWS Retail Competency Partners provide innovative technology offerings that accelerate retailers’ modernization and innovation journey across all areas in the enterprise. Read More

2. Introducing AWS Public Safety & Disaster Response Competency Partners: AWS Customers can quickly identify top-tier APN Consulting Partners who identify, build, and implement technology offerings aimed at improving organizational capacity to prepare, respond, and recover from emergencies and disasters, globally. Read More

3. New AWS Service Ready Program to help customers find tools that integrate with AWS services: AWS Partner Network (APN) announced AWS Service Ready Program, a new way for AWS customers to identify if a tool or application will integrate with AWS services running in their cloud environment. Read More

4. New APN Global Startup Program, helping startup APN Technology Partners grow their cloud-based business: AWS Partner Network (APN) announced the APN Global Startup Program, a dedicated go-to-market (GTM) program for eligible Startup APN Technology Partners. Read More

5. Introducing a new benefit for APN Consulting Partners, APN Immersion Days: AWS Immersion Day workshops provide a customizable AWS experience delivered by AWS Solution Architects and Account Managers to AWS customers. Read More

6. AWS Marketplace makes it easier for you to discover relevant third-party software and data products: AWS Marketplace, a digital catalog with over 7,000 software listings and data products, has announced Discovery API, a new API created for select partners. Read More

7. AWS Marketplace announces a simplified fee structure and the expansion of Seller Private Offers: Starting today, all registered sellers with a public listing in AWS Marketplace can extend a custom contract through Seller Private Offers. Read More

AWS re:Invent 2019

AWS re:Invent 2019

Event Details: At re:Invent 2019, you can expect deeper technical content, more hands-on learning opportunities, and more access to AWS experts than ever. Each year at re:Invent, we bring you over a thousand sessions, chalk talks, workshops, builders sessions, and hackathons that cover AWS core topics and highlight the emerging technologies that we are developing. This year, re:Invent will be no different.

[Know more about the Conference]

About Idexcel: Idexcel is a Professional Services and Technology Solutions provider specializing in Cloud Services, Application Modernization, and Data Analytics. Idexcel is proud that for more than 20 years it has provided services that implement complex technologies that are innovative, agile and successful and have provided our customers with lasting value.

Anand Allolankandy – (Senior Director, Cloud Services Practice at Idexcel) will be attending this event. For further queries, please write to anand@idexcel.com

7 Reasons Why You Should Choose AWS as Your Cloud Partner

7 Reasons Why You Should Choose AWS as Your Cloud Partner

Amazon Web Services, or AWS, is the world’s largest cloud platform, boasting roughly 1 million customers, and raking in $10 billion in revenue every year. Despite steep competition within the space, AWS Cloud has managed to grow at a consistent pace, now owning approx 33% of the cloud computing market.

Given its recent growth, AWS has quickly risen the ranks to become one of the top cloud providers in the market. Thanks its staggering growth, a lot of companies are migrating their businesses to the AWS cloud, offering them a slew of benefits.

Here are seven of the biggest reasons why your business should also move to the AWS and adopt them as your cloud provider.

1) Location matters… a lot: AWS is arguably the world’s leading cloud platform as its operations can be found in 69 global zones, as well as 22 geographical locations around the world. The company’s outreach is expanding quickly, truly making AWS a market leader. No matter what part of the world you may find yourself in, chances are AWS is already available for business in your neck of the woods. As it continues to expand its outreach, there are endless ways in which AWS helps businesses with their storage needs, regardless of location. The cloud is available anywhere and everywhere, due in large part to AWS.

2) Well defined disaster recovery: Business continuity is one of the most important factors for a business. On the off chance a disaster strikes, it is important for a company to have a recovery system that ensures its day-to-day operations progress without a hitch, preventing any losses from transpiring. Regardless of a company’s risk tolerance levels, no business wants to find itself facing any sort of downtime that creates a breach of trust with its customers over any period. Having said that, AWS has been built to provide a well-defined disaster recovery plan, which is focused on helping its clients keep their business rolling along as planned.

3) Reliability and consistency: Consistency and reliability are two keywords that aptly describe what AWS cloud is about. Despite having some high-profile outages, AWS has always been the most reliable service. In fact, 40% of the product’s downtime was caused by a single outage. This statistic suggests that outages are not very common within the world of AWS.

4) Scalability and flexibility: The list of AWS advantages does not stop at the aforementioned factors of reliability and consistency factors. Providing a business with the flexibility to scale up or down as required makes it an especially strong service for executives seeking to increase or decrease server space in a matter of miliseconds. There is no need for any unnecessary procedures or dealing with the morose task of setting up a physical infrastructure, as everything works as it should—in a sweet, simple and concise manner.

5) Simple, automated scheduling facilities: Scheduling and automation are common needs that companies expect out of cloud service providers nowadays. Guess what? The ability to schedule jobs not only allows things to be automated with ease, but also paves the way for a business to run smoothly. The whole idea is to ensure that there’s plenty of bandwidth available to run jobs without interruptions, especially when it comes to manual supervision.

6) Pay-as-you-go: Not many cloud providers have the pay-as-you-go option in their cloud packages. AWS, on the other hand, has, making it the go-to point that people turn to when seeking to save costs, while also helping them benefit from the excellent range of facilities offered within the AWS suite. Start and stop instances within the product according to your needs, and pay only the time spent on the cloud. Adjust your usage levels, scale up and down, and avoid the trouble of setting up a physical infrastructure while having your technical ends meet.

7) Security: No matter what kind of a company you own, whether it’s a startup or an enterprise level conglomerate, chances are security is of the utmost importance to you and your clients. AWS eliminates this fear, offering you world class security that follow the highest regulatory standards. It provides you with dozens of compliance programs built within AWS that pertain to its infrastructure, which are included per each service offered. There’s more, as the services also provide extensive security and support networks geared towards flagging suspicious activity on the accounts, and thereby raising revealing vulnerabilities within the systems.

Contact us to get your free AWS Well-Architected Framework Review

AWS Summit – New York

AWS Summit - NY

Event Details: The AWS Summit New York is a free event designed to bring together the cloud computing community to connect, collaborate, and learn about AWS. Summits are held in major cities around the world and attract technologists from multiple industries, segments, and learning levels who want to learn how AWS can help them innovate with speed and deliver services with scale, flexibility, and reliability.

Featured Topics: Choose from 120+ technical learning opportunities ranging from introductions to deep dives and includes small group hands-on workshops, lecture-style sessions, and open Q&A chalk talks. Topics will cover, but are not limited to, Machine Learning, Artificial Intelligence, Serverless, Compute, Databases, Security & Compliance, and more.

[Know more about the event]

About Idexcel: Idexcel is a Professional Services and Technology Solutions provider specializing in Cloud Services, Application Modernization, and Data Analytics. Idexcel is proud that for more than 20 years it has provided services that implement complex technologies that are innovative, agile and successful and have provided our customers with lasting value.

James Markham – (Regional Cloud Executive at Idexcel) will be attending this event. For further queries, please write to james.markham@idexcel.com