According to Amazon: EventBridge is a serverless event bus service that helps connect external SaaS providers with your own applications and AWS services. EventBridge enables you to decouple your architectures to make it faster to build and innovate, using routing rules to deliver events to selected targets.
However not all of these events need to be triggered, they can easily be scheduled. For example, a daily event. I currently have a goal to schedule a daily event that makes API calls which return Gzipped files, which I would then like to save into a database. Should I then have a DynamoDB table to keep track of which API calls I would like to make along with the required credentials? What if I need to make hundreds of these API calls. It seems like some sort of queue would be in order. Let's briefly go over how to set a scheduled event in EventBridge, and then review our AWS service options to find the best one to handle this workflow.
You can then name the rule and set it on a fixed schedule, or if you want something a little more complicated (IE every other thursday @ noon) you can use a cron expression.
This is where it gets interesting, we can select the event target, for example a Lambda function.
Lambda I am already very familiar with, we know for certain that Lambda could handle a lot of this. The question really is if any of these other services can do the job of Lambda here in a better/easier/dedicated way. So let's explore that.
Batch will automatically provision a set of resources to run large scale compute jobs synchronously. It looks like this is what we would use if we needed to train several machine learning models simultaneously. Which we may...
It looks like this is going to need a whole post, but this could be very useful. It's a GUI based system for complex state based functions. IE we could do different things depending on how our API call was returned, and plan out multiple steps clearly. Going to dig into this.
AWS Glue is a fully managed ETL (extract, transform and load) service that makes it simple to categorize your data, clean it, enrich it and move it reliably between various data stores.
So it sounds like this is the service that we are going to use once our data has been downloaded from the API and successfully saved into an S3 Bucket, we can extract it, clean it and transform it in the way that we need before feeding it into a machine learning model endpoint, and then back to S3 or DynamoDB again.
With Glue we will set up crawlers to define our data locations, and then workflows (which we can use Glue Databrew to help create) which are written with spark scripts, to transform the data and place it in a destination bucket. And we can use these workflows as steps in our Step Functions.
Seems like this is a cluster of databases for managing massive databases. Doesn't seem like I need this yet, but we will see. Also can get very expensive very quickly...
If we wanted to create a new machine learning model for each customer, instead of creating one model and feeding all the data through that endpoint, we could create a Sagemaker Pipeline, which is a sort of CI/CD for machine learning models. Automating the process of creating models based on a template.
In short this is a queuing system that is designed to mitigate high volume request spikes, such as read/write operations to a DynamoDB faster than the rate limit allows. This doesn't seem like something that I will be running into anytime soon, so i'm going to categorize this as a possible problem for "future me".
My takeaway from this is that I should actually be at step functions to help me manage some of the more complex operations and orchestrate the use of my microservices. This will also give me increased re-usability of my Lambda functions, like the one that refreshes the authorization token for the API.
Free desktop AI Chat client, designed for developers and businesses. Unlocks advanced model settings only available in the API. Includes quality of life features like custom syntax highlighting.
React is a declarative, efficient, and flexible JavaScript library for building user interfaces. It lets you compose complex UIs from small and isolated pieces of code called “components”.
Bidbear is a report automation tool. It downloads Amazon Seller and Advertising reports, daily, to a private database. It then merges and formats the data into beautiful, on demand, exportable performance reports.
JavaScript is a lightweight, interpreted programming language with first-class functions that is known for its role in web development. It is an essential part of web applications and allows for interactive elements.
Docker is a platform for developers and sysadmins to develop, deploy, and run applications with containers. The use of Linux containers to deploy applications is called containerization.
React is a declarative, efficient, and flexible JavaScript library for building user interfaces. It lets you compose complex UIs from small and isolated pieces of code called “components”.
Redux is a predictable state container for JavaScript apps. It helps you write applications that behave consistently, run in different environments (client, server, and native), and are easy to test.
Vite is a modern, fast front-end build tool that significantly improves the frontend development experience. It features a dev server with Hot Module Replacement (HMR) and a build command that bundles your code with Rollup, which is configured to output highly optimized static assets for production.
Next.js is an open-source React front-end development web framework that enables functionality such as server-side rendering and generating static websites for React based web applications.
Docusaurus is a project for building, deploying, and maintaining open source project websites easily. It's optimized for documentation and can leverage the power of Markdown and React to enable seamless documentation sites with built-in search capabilities.
Stripe is a technology company that builds economic infrastructure for the internet. Businesses of every size use the company's software to accept payments and manage their businesses online.
Sentry is an open-source error tracking tool that helps developers monitor and fix crashes in real time. The tool streamlines the error resolution process and aggregates errors across different programming languages.
D3.js is a JavaScript library for producing dynamic, interactive data visualizations in web browsers. It uses HTML, SVG, and CSS to bring data to life through a data-driven approach to DOM manipulation.
React Flow is a highly customizable library for building interactive node-based editors, diagrams, and flow charts. It provides a set of tools that make it easy to drag, drop, and connect nodes on a canvas.
TipTap is a headless, framework-agnostic text editor capable of handling a variety of complex text editing needs. It is built on top of ProseMirror and is extendable and customizable to fit different editing requirements.
JavaScript is a lightweight, interpreted programming language with first-class functions that is known for its role in web development. It is an essential part of web applications and allows for interactive elements.
Python is a high-level, interpreted, general-purpose programming language. Its design philosophy emphasizes code readability with its use of significant whitespace.
Amazon Cognito provides authentication, authorization, and user management for your web and mobile apps. Users can sign in directly with a username and password, or through a third party such as Facebook, Amazon, or Google.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. It acts as a front door for applications to access data, business logic, or functionality from your back-end services.
AWS Lambda is a compute service that lets you run code without provisioning or managing servers. Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second.
Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. It is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.
AWS Glue is a fully managed extract, transform, and load (ETL) service that makes it simple and cost-effective to categorize your data, clean it, enrich it, and move it reliably between various data stores.
AWS Step Functions is a service that lets you coordinate multiple AWS services into serverless workflows so you can build and update apps quickly. It provides a reliable way to orchestrate the components of distributed applications and microservices using visual workflows.
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware.
Amazon DynamoDB is a fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale. It is a fully managed database and supports both document and key-value store models.
Amazon S3 (Simple Storage Service) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics.
Amazon CloudWatch is a monitoring and observability service built for DevOps engineers, developers, site reliability engineers (SREs), and IT managers. CloudWatch provides data and actionable insights to monitor applications, understand and respond to system-wide performance changes, optimize resource utilization, and get a unified view of operational health.
Amazon CloudFront is a fast content delivery network (CDN) service that securely delivers data, videos, applications, and APIs to customers globally with low latency, high transfer speeds, all within a developer-friendly environment.
Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. It is designed to give developers and businesses an extremely reliable and cost-effective way to route end users to Internet applications.
Amazon EventBridge is a serverless event bus service that makes it easy to connect applications together using data from your own applications, integrated SaaS applications, and AWS services. EventBridge helps facilitate event-driven architectures by routing events between AWS services, integrated SaaS applications, and your own applications.
