Cloud Architecture

Recently, I was back at the Cloud Native London meetup, having been given the opportunity to present due to a speaker canceling at the last minute. This group has 7,000+ members and is, "...the official Cloud Native Computing Foundation (CNCF) Meetup group dedicated to building a strong, open, diverse developer community around the Cloud Native platform and technologies in London." You can also find them on their own Slack channel, so feel free to drop in for a quick chat if you like. There were over 85 attendees who braved the cold London evening to join us for pizza, drinks, and a bit of fun with my session having a special design this time around. I went out on a limb and tried something I'd never seen before - a sort of choose-your-own-adventure presentation. Below I've included a description of how I think it went, the feedback I got, and where you can find both the slides and recording online if you missed it. About the Presentation Here are the schedule details for the day: Check out the three fantastic speakers we've got lined up for you on Wednesday 10 January: 18:00 Pizza and drinks 18:30 Welcome 18:45 Quickwit: Cloud-Native Logging and Distributed Tracing (Francois Massot, Quickwit) 19:15 - 3 Pitfalls Everyone Should Avoid with Cloud Native Observability (Eric D. Schabell, Chronosphere) 19:45 Break 20:00 Transcending microservices hell for Supergraph Nirvana (Tom Harding, Hasura) 20:30 Wrap up See you there! The agenda for the January Cloud Native London Meetup is now up. If you're not able to join us, don't forget to update your RSVP before 10am on Wednesday! Or alternatively, join us via the YouTube stream without signing up. As I mentioned, my talk is a new idea I've been working on for the last year. I want to share insights into the mistakes and pitfalls that I'm seeing customers and practitioners make repeatedly on their cloud-native observability journey. Not only were there new ideas for content, but I wanted to try something a bit more daring this time around and tried to engage the audience with a bit of choose-your-own-adventure in which they were choosing which pitfall would be covered next. I started with a generic introduction, then gave them the following six choices: Ignoring costs in the application landscape Focusing on The Pillars Sneaky sprawling tooling mess Controlling costs Losing your way in the protocol jungles Underestimating cardinality For this Cloud Native London session, we ended up going in this order: pitfalls #6, #3, and #4. This meant the session recording posted online from the event contained the following content: Introduction to cloud-native and cloud-native observability problems (framing the topic) Pitfall #1 - Underestimating cardinality Pitfall #2 - Sneaky sprawling tooling mess Pitfall #3 - Controlling costs It went pretty smoothly and I was excited to get a lot of feedback from attendees who enjoyed the content, the takes on cloud-native observability pitfalls, and they loved the engaging style of choosing your own adventure! If you get the chance to see this talk next time I present it, there's a good chance it will contain completely different content. Video, Slides, and Abstract Session Video Recording Session Slides 3 Pitfalls Everyone Should Avoid with Cloud Native Observability from Eric D. Schabell Abstract Are you looking at your organization's efforts to enter or expand into the cloud-native landscape and feeling a bit daunted by the vast expanse of information surrounding cloud-native observability? When you're moving so fast with agile practices across your DevOps, SREs, and platform engineering teams, it's no wonder this can seem a bit confusing. Unfortunately, the choices being made have a great impact on your business, your budgets, and the ultimate success of your cloud-native initiatives. That hasty decision up front leads to big headaches very quickly down the road. In this talk, I'll introduce the problem facing everyone with cloud-native observability followed by 3 common mistakes that I'm seeing organizations make and how you can avoid them! Coming Up I am scheduled to return in May to present again and look forward to seeing everyone in London in the spring!

By Eric D. Schabell CORE

Kubernetes has significantly simplified the management and operation of containerized applications. However, as these applications grow in complexity, there is an increasing need for more sophisticated deployment management tools. This is where Helm becomes invaluable. As a Kubernetes package manager, Helm greatly streamlines and simplifies deployment processes. In this article, we will delve deeply into Helm and explore how it facilitates the easier management of Kubernetes deployments. The Challenges of Kubernetes Deployments Kubernetes is fantastic for automating the deployment and management of containerized apps. It's great for running microservices and any other stateless applications. However, managing deployments becomes a big challenge as your Kubernetes system gets more extensive and complicated. Here are the issues: Configuration Confusion: Managing configurations for different apps and services can get messy. It's even more complicated when you have different environments like development, staging, and production. Keeping Track of Versions: It takes a lot of work to track your apps' different versions and configurations. This can lead to mistakes and confusion. Dealing With Dependencies: When your apps get complex, they depend on other things. Making sure these dependencies are set up correctly takes time. Doing It Again and Again: Repeating deployments on different clusters or environments is a big job and can lead to mistakes. Introducing Helm Helm is often called the "app store" for Kubernetes because it makes handling deployments easy. Here's how Helm works: Charts: In Helm, a package of pre-set Kubernetes resources is called a "chart." A chart is a set of files that explains a group of Kubernetes resources. These resources include services, deployments, config maps, and more. Templates: Helm uses templates to create Kubernetes resources within a chart. These templates let you change how your app works. You can customize your deployments for different environments. Repositories: Charts are stored in "repositories," like app stores for Helm charts. You can use public ones or make your own private store. Managing helm deployments: Helm looks after "releases" and special chart deployments. This means you can track which version of a chart you deployed and what settings you used. The Advantages of Helm Helm has some significant advantages when it comes to handling Kubernetes deployments: Reusing Charts: You can share and reuse charts within your organization. This stops you from doing the same work repeatedly and ensures your deployments are consistent. Keeping Track of Versions: Helm helps you follow different versions of your apps and their setups. This is important for keeping your deployments stable and the same every time. Customization: Helm charts are very flexible. You can use values and templates to adjust your setup for different environments. Handling Dependencies: Helm sorts out dependencies quickly. If your app relies on other things, Helm will ensure they're set up and work correctly. Going Back in Time: Helm makes returning to an older app version easy, reducing downtime and stopping problems. Robust Support Network: Helm has a significant and active community. This means you can find and use charts made by other organizations. This saves you time when deploying common apps. Helm in Action Let's look at how Helm helps with deploying a web app, step by step: 1. Creating a Chart: First, you make a Helm chart for your web app. The chart has templates for the web server, the database, and other parts needed. 2. Changing the Setup: You use Helm's values and templates to change how your web app works. For example, you can say how many copies you want, the database connection, and which environment to use (like development or production). 3. Installation: With just one command, you install your web app using the Helm chart. Helm sets up everything your app needs based on the chart and your changes. 4. Upgrades: Change the chart version or values when updating your app. Helm will update your app with little work. Challenges and Important Points Even though Helm is great, you need to remember some things: Safety in Deployments: Ensure Helm deployments are secure, especially in multi-user environments, by implementing proper access controls and security practices. Best Practices: Focus on mastering the creation of Helm charts with best practices, ensuring efficient, reliable, and maintainable deployments. Dependency Management: Manage dependencies in Helm charts with careful consideration, including thorough testing and validation to avoid conflicts and issues. Chart Updates: Keep Helm charts regularly updated to benefit from the latest security patches, performance improvements, and new features. How Atmosly Integrates Helm? Atmosly's integration with Helm brings to the forefront a dynamic marketplace that makes deploying applications to Kubernetes smoother. This powerful feature provides a centralized hub for discovering and deploying a wide range of Helm charts. From popular open-source helm charts to private applications that are templated using helm, users can easily navigate and select the necessary helm charts to deploy applications across various clusters with ease without having to take care of access and permissions. Atmosly’s Marketplace Features The marketplace is thoughtfully designed to cater to both public and private chart repositories, enabling teams to maintain a catalog of their own custom charts while also leveraging the vast repository of community-driven Helm charts. This dual capability ensures users can quickly adapt to different project requirements without leaving the Atmosly platform. The user-friendly interface of the marketplace displays an array of Helm charts, categorized for easy access, whether they are maintained by Atmosly, managed by users, or provided by third-party entities like Bitnami. Teams can deploy tools and applications, such as Apache, Elasticsearch, or custom enterprise solutions, straight into their Kubernetes environment with a simple click. By seamlessly integrating public and private Helm charts into a unified deployment experience, Atmosly's marketplace facilitates a level of agility and control that is essential for modern DevOps teams. It represents a strategic move towards simplifying complex deployment tasks, reducing the potential for error, and accelerating the journey from development to production. Wrapping Up Helm is an excellent tool for handling Kubernetes deployments. It makes things easy, even for complex apps and setups. You can have better, more stable, and customizable Kubernetes deployments using Helm's features. As Kubernetes keeps growing, Helm remains an essential tool to simplify and improve the deployment process. If you still need to look at Helm, it's time to see how it can help you with your Kubernetes management.

Serverless architecture is a way of building and running applications without the need to manage infrastructure. You write your code, and the cloud provider handles the rest - provisioning, scaling, and maintenance. AWS offers various serverless services, with AWS Lambda being one of the most prominent. When we talk about "serverless," it doesn't mean servers are absent. Instead, the responsibility of server maintenance shifts from the user to the provider. This shift brings forth several benefits: Cost-efficiency: With serverless, you only pay for what you use. There's no idle capacity because billing is based on the actual amount of resources consumed by an application. Scalability: Serverless services automatically scale with the application's needs. As the number of requests for an application increases or decreases, the service seamlessly adjusts. Reduced operational overhead: Developers can focus purely on writing code and pushing updates, rather than worrying about server upkeep. Faster time to market: Without the need to manage infrastructure, development cycles are shorter, enabling more rapid deployment and iteration. Importance of Resiliency in Serverless Architecture As heavenly as serverless sounds, it isn't immune to failures. Resiliency is the ability of a system to handle and recover from faults, and it's vital in a serverless environment for a few reasons: Statelessness: Serverless functions are stateless, meaning they do not retain any data between executions. While this aids in scalability, it also means that any failure in the function or a backend service it depends on can lead to data inconsistencies or loss if not properly handled. Third-party services: Serverless architectures often rely on a variety of third-party services. If any of these services experience issues, your application could suffer unless it's designed to cope with such eventualities. Complex orchestration: A serverless application may involve complex interactions between different services. Coordinating these reliably requires a robust approach to error handling and fallback mechanisms. Resiliency is, therefore, not just desirable, but essential. It ensures that your serverless application remains reliable and user-friendly, even when parts of the system go awry. In the subsequent sections, we will examine the circuit breaker pattern, a design pattern that enhances fault tolerance and resilience in distributed systems like those built on AWS serverless technologies. Understanding the Circuit Breaker Pattern Imagine a bustling city where traffic flows smoothly until an accident occurs. In response, traffic lights adapt to reroute cars, preventing a total gridlock. Similarly, in software development, we have the circuit breaker pattern—a mechanism designed to prevent system-wide failures. Its primary purpose is to detect failures and stop the flow of requests to the faulty part, much like a traffic light halts cars to avoid congestion. When a particular service or operation fails to perform correctly, the circuit breaker trips and future calls to that service are blocked or redirected. This pattern is essential because it allows for graceful degradation of functionality rather than complete system failure. It’s akin to having an emergency plan: when things go awry, the pattern ensures that the rest of the application can continue to operate. It provides a recovery period for the failed service, wherein no additional strain is added, allowing for potential self-recovery or giving developers time to address the issue. Relationship Between the Circuit Breaker Pattern and Fault Tolerance in Distributed Systems In the interconnected world of distributed systems where services rely on each other, fault tolerance is the cornerstone of reliability. The circuit breaker pattern plays a pivotal role in this by ensuring that a fault in one service doesn't cascade to others. It's the buffer that absorbs the shock of a failing component. By monitoring the number of recent failures, the pattern decides when to open the "circuit," thus preventing further damage and maintaining system stability. The concept is simple yet powerful: when the failure threshold is reached, the circuit trips, stopping the flow of requests to the troubled service. Subsequent requests are either returned with a pre-defined fallback response or are queued until the service is deemed healthy again. This approach not only protects the system from spiraling into a state of unresponsiveness but also shields users from experiencing repeated errors. Relevance of the Circuit Breaker Pattern in Microservices Architecture Microservices architecture is like a complex ecosystem with numerous species—numerous services interacting with one another. Just as an ecosystem relies on balance to thrive, so does a microservices architecture depend on the resilience of individual services. The circuit breaker pattern is particularly relevant in such environments because it provides the necessary checks and balances to ensure this balance is maintained. Given that microservices are often designed to be loosely coupled and independently deployable, the failure of a single service shouldn’t bring down the entire system. The circuit breaker pattern empowers services to handle failures gracefully, either by retrying operations, redirecting traffic, or providing fallback solutions. This not only improves the user experience during partial outages but also gives developers the confidence to iterate quickly, knowing there's a safety mechanism in place to handle unexpected issues. In modern applications where uptime and user satisfaction are paramount, implementing the circuit breaker pattern can mean the difference between a minor hiccup and a full-blown service interruption. By recognizing its vital role in maintaining the health of a microservices ecosystem, developers can craft more robust and resilient applications that can withstand the inevitable challenges that come with distributed computing. Leveraging AWS Lambda for Resilient Serverless Microservices When we talk about serverless computing, AWS Lambda often stands front and center. But what is AWS Lambda exactly, and why is it such a game-changer for building microservices? In essence, AWS Lambda is a service that lets you run code without provisioning or managing servers. You simply upload your code, and Lambda takes care of everything required to run and scale your code with high availability. It's a powerful tool in the serverless architecture toolbox because it abstracts away the infrastructure management so developers can focus on writing code. Now, let's look at how the circuit breaker pattern fits into this picture. The circuit breaker pattern is all about preventing system overloads and cascading failures. When integrated with AWS Lambda, it monitors the calls to external services and dependencies. If these calls fail repeatedly, the circuit breaker trips and further attempts are temporarily blocked. Subsequent calls may be routed to a fallback mechanism, ensuring the system remains responsive even when a part of it is struggling. For instance, if a Lambda function relies on an external API that becomes unresponsive, applying the circuit breaker pattern can help prevent this single point of failure from affecting the entire system. Best Practices for Utilizing AWS Lambda in Conjunction With the Circuit Breaker Pattern To maximize the benefits of using AWS Lambda with the circuit breaker pattern, consider these best practices: Monitoring and logging: Use Amazon CloudWatch to monitor Lambda function metrics and logs to detect anomalies early. Knowing when your functions are close to tripping a circuit breaker can alert you to potential issues before they escalate. Timeouts and retry logic: Implement timeouts for your Lambda functions, especially when calling external services. In conjunction with retry logic, timeouts can ensure that your system doesn't hang indefinitely, waiting for a response that might never come. Graceful fallbacks: Design your Lambda functions to have fallback logic in case the primary service is unavailable. This could mean serving cached data or a simplified version of your service, allowing your application to remain functional, albeit with reduced capabilities. Decoupling services: Use services like Amazon Simple Queue Service (SQS) or Amazon Simple Notification Service (SNS) to decouple components. This approach helps in maintaining system responsiveness, even when one component fails. Regular testing: Regularly test your circuit breakers by simulating failures. This ensures they work as expected during real outages and helps you refine your incident response strategies. By integrating the circuit breaker pattern into AWS Lambda functions, you create a robust barrier against failures that could otherwise ripple across your serverless microservices. The synergy between AWS Lambda and the circuit breaker pattern lies in their shared goal: to offer a resilient, highly available service that focuses on delivering functionality, irrespective of the inevitable hiccups that occur in distributed systems. While AWS Lambda relieves you from the operational overhead of managing servers, implementing patterns like the circuit breaker is crucial to ensure that this convenience does not come at the cost of reliability. By following these best practices, you can confidently use AWS Lambda to build serverless microservices that aren't just efficient and scalable but also resilient to the unexpected. Implementing the Circuit Breaker Pattern With AWS Step Functions AWS Step Functions provides a way to arrange and coordinate the components of your serverless applications. With AWS Step Functions, you can define workflows as state machines, which can include sequential steps, branching logic, parallel tasks, and even human intervention steps. This service ensures that each function knows its cue and performs at the right moment, contributing to a seamless performance. Now, let's introduce the circuit breaker pattern into this choreography. When a step in your workflow hits a snag, like an API timeout or resource constraint, the circuit breaker steps in. By integrating the circuit breaker pattern into AWS Step Functions, you can specify conditions under which to "trip" the circuit. This prevents further strain on the system and enables it to recover, or redirect the flow to alternative logic that handles the issue. It's much like a dance partner who gracefully improvises a move when the original routine can't be executed due to unforeseen circumstances. To implement this pattern within AWS Step Functions, you can utilize features like Catch and Retry policies in your state machine definitions. These allow you to define error handling behavior for specific errors or provide a backoff rate to avoid overwhelming the system. Additionally, you can set up a fallback state that acts when the circuit is tripped, ensuring that your application remains responsive and reliable. The benefits of using AWS Step Functions to implement the circuit breaker pattern are manifold. First and foremost, it enhances the robustness of your serverless application by preventing failures from escalating. Instead of allowing a single point of failure to cause a domino effect, the circuit breaker isolates issues, giving you time to address them without impacting the entire system. Another advantage is the reduction in cost and improved efficiency. AWS Step Functions allows you to pay per transition of your state machine, which means that by avoiding unnecessary retries and reducing load during outages, you're not just saving your system but also your wallet. Last but not least, the clarity and maintainability of your serverless workflows improve. By defining clear rules and fallbacks, your team can instantly understand the flow and know where to look when something goes awry. This makes debugging faster and enhances the overall development experience. Incorporating the circuit breaker pattern into AWS Step Functions is more than just a technical implementation; it's about creating a choreography where every step is accounted for, and every misstep has a recovery routine. It ensures that your serverless architecture performs gracefully under pressure, maintaining the reliability that users expect and that businesses depend on. Conclusion The landscape of serverless architecture is dynamic and ever-evolving. This article has provided a foundational understanding. In our journey through the intricacies of serverless microservices architecture on AWS, we've encountered a powerful ally in the circuit breaker pattern. This mechanism is crucial for enhancing system resiliency and ensuring that our serverless applications can withstand the unpredictable nature of distributed environments. We began by navigating the concept of serverless architecture on AWS and its myriad benefits, including scalability, cost-efficiency, and operational management simplification. We understood that despite its many advantages, resiliency remains a critical aspect that requires attention. Recognizing this, we explored the circuit breaker pattern, which serves as a safeguard against failures and an enhancer of fault tolerance within our distributed systems. Especially within a microservices architecture, it acts as a sentinel, monitoring for faults and preventing cascading failures. Our exploration took us deeper into the practicalities of implementation with AWS Step Functions and how they orchestrate serverless workflows with finesse. Integrating the circuit breaker pattern within these functions allows error handling to be more robust and reactive. With AWS Lambda, we saw another layer of reliability added to our serverless microservices, where the circuit breaker pattern can be cleverly applied to manage exceptions and maintain service continuity. Investing time and effort into making our serverless applications reliable isn't just about avoiding downtime; it's about building trust with our users and saving costs in the long run. Applications that can gracefully handle issues and maintain operations under duress are the ones that stand out in today's competitive market. By prioritizing reliability through patterns like the circuit breaker, we not only mitigate the impact of individual component failures but also enhance the overall user experience and maintain business continuity. In conclusion, the power of the circuit breaker pattern in a serverless environment cannot be overstated. It is a testament to the idea that with the right strategies in place, even the most seemingly insurmountable challenges can be transformed into opportunities for growth and innovation. As architects, developers, and innovators, our task is to harness these patterns and principles to build resilient, responsive, and reliable serverless systems that can take our applications to new heights.

By Satrajit Basu CORE

It’s one thing to build powerful machine-learning models and another thing to be able to make them useful. A big part of it is to be able to build applications to expose its features to end users. Popular examples include ChatGPT, Midjourney, etc. Streamlit is an open-source Python library that makes it easy to build web applications for machine learning and data science. It has a set of rich APIs for visual components, including several chat elements, making it quite convenient to build conversational agents or chatbots, especially when combined with LLMs (Large Language Models). And that’s the example for this blog post as well — a Streamlit-based chatbot deployed to a Kubernetes cluster on Amazon EKS. But that’s not all! We will use Streamlit with LangChain, which is a framework for developing applications powered by language models. The nice thing about LangChain is that it supports many platforms and LLMs, including Amazon Bedrock (which will be used for our application). A key part of chat applications is the ability to refer to historical conversation(s) — at least within a certain time frame (window). In LangChain, this is referred to as Memory. Just like LLMs, you can plug-in different systems to work as the memory component of a LangChain application. This includes Redis, which is a great choice for this use case since it’s a high-performance in-memory database with flexible data structures. Redis is already a preferred choice for real-time applications (including chat) combined with Pub/Sub and WebSocket. This application will use Amazon ElastiCache Serverless for Redis, an option that simplifies cache management and scales instantly. This was announced at re:Invent 2023, so let’s explore while it’s still fresh! To be honest, the application can be deployed on other compute options such as Amazon ECS, but I figured since it needs to invoke Amazon Bedrock, it’s a good opportunity to also cover how to use EKS Pod Identity (also announced at re:Invent 2023!!) GitHub repository for the app. Here is a simplified, high-level diagram: Let’s go!! Basic Setup Amazon Bedrock: Use the instructions in this blog post to set up and configure Amazon Bedrock. EKS cluster: Start by creating an EKS cluster. Point kubectl to the new cluster using aws eks update-kubeconfig --region <cluster_region> --name <cluster_name> Create an IAM role: Use the trust policy and IAM permissions from the application GitHub repository. EKS Pod Identity Agent configuration: Set up the EKS Pod Identity Agent and associate EKS Pod Identity with the IAM role you created. ElastiCache Serverless for Redis: Create a Serverless Redis cache. Make sure it shares the same subnets as the EKS cluster. Once the cluster creation is complete, update the ElastiCache security group to add an inbound rule (TCP port 6379) to allow the application on the EKS cluster to access the ElastiCache cluster. Push the Docker Image to ECR and Deploy the App to EKS Clone the GitHub repository: Shell git clone https://github.com/abhirockzz/streamlit-langchain-chatbot-bedrock-redis-memory cd streamlit-langchain-chatbot-bedrock-redis-memory Create an ECR repository: Shell export REPO_NAME=streamlit-chat export REGION=<AWS region> ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text) aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com aws ecr create-repository --repository-name $REPO_NAME Create the Docker image and push it to ECR: Shell docker build -t $REPO_NAME . docker tag $REPO_NAME:latest $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:latest docker push $ACCOUNT_ID.dkr.ecr.$REGION.amazonaws.com/$REPO_NAME:latest Deploy Streamlit Chatbot to EKS Update the app.yaml file: Enter the ECR docker image info In the Redis connection string format, enter the Elasticache username and password along with the endpoint. Deploy the application: Shell kubectl apply -f app.yaml To check logs: kubectl logs -f -l=app=streamlit-chat Start a Conversation! To access the application: Shell kubectl port-forward deployment/streamlit-chat 8080:8501 Navigate to http://localhost:8080 using your browser and start chatting! The application uses the Anthropic Claude model on Amazon Bedrock as the LLM and Elasticache Serverless instance to persist the chat messages exchanged during a particular session. Behind the Scenes in ElastiCache Redis To better understand what’s going on, you can use redis-cli to access the Elasticache Redis instance from EC2 (or Cloud9) and introspect the data structure used by LangChain for storing chat history. keys * Don’t run keys * in a production Redis instance — this is just for demonstration purposes. You should see a key similar to this — "message_store:d5f8c546-71cd-4c26-bafb-73af13a764a5" (the name will differ in your case). Check it’s type: type message_store:d5f8c546-71cd-4c26-bafb-73af13a764a5 - you will notice that it's a Redis List. To check the list contents, use the LRANGE command: Shell LRANGE message_store:d5f8c546-71cd-4c26-bafb-73af13a764a5 0 10 You should see a similar output: Shell 1) "{\"type\": \"ai\", \"data\": {\"content\": \" Yes, your name is Abhishek.\", \"additional_kwargs\": {}, \"type\": \"ai\", \"example\": false}" 2) "{\"type\": \"human\", \"data\": {\"content\": \"Thanks! But do you still remember my name?\", \"additional_kwargs\": {}, \"type\": \"human\", \"example\": false}" 3) "{\"type\": \"ai\", \"data\": {\"content\": \" Cloud computing enables convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort.\", \"additional_kwargs\": {}, \"type\": \"ai\", \"example\": false}" 4) "{\"type\": \"human\", \"data\": {\"content\": \"Tell me about Cloud computing in one line\", \"additional_kwargs\": {}, \"type\": \"human\", \"example\": false}" 5) "{\"type\": \"ai\", \"data\": {\"content\": \" Nice to meet you, Abhishek!\", \"additional_kwargs\": {}, \"type\": \"ai\", \"example\": false}" 6) "{\"type\": \"human\", \"data\": {\"content\": \"Nice, my name is Abhishek\", \"additional_kwargs\": {}, \"type\": \"human\", \"example\": false}" 7) "{\"type\": \"ai\", \"data\": {\"content\": \" My name is Claude.\", \"additional_kwargs\": {}, \"type\": \"ai\", \"example\": false}" 8) "{\"type\": \"human\", \"data\": {\"content\": \"Hi what's your name?\", \"additional_kwargs\": {}, \"type\": \"human\", \"example\": false}" Basically, the Redis memory component for LangChain persists the messages as a List and passes its contents as additional context with every message. Conclusion To be completely honest, I am not a Python developer (I mostly use Go, Java, or sometimes Rust), but I found Streamlit relatively easy to start with, except for some of the session-related nuances. I figured out that for each conversation, the entire Streamlit app is executed (this was a little unexpected coming from a backend dev background). That’s when I moved the chat ID (kind of unique session ID for each conversation) to the Streamlit session state, and things worked. This is also used as part of the name of the Redis List that stores the conversation (message_store:<session_id>) — each Redis List is mapped to the Streamlist session. I also found the Streamlit component-based approach to be quite intuitive and pretty extensive as well. I was wondering if there are similar solutions in Go. If you know of something, do let me know. Happy building!

By Abhishek Gupta CORE

In the ever-evolving landscape of cloud-native computing, containers have emerged as the linchpin, enabling organizations to build, deploy, and scale applications with unprecedented agility. However, as the adoption of containers accelerates, so does the imperative for robust container security strategies. The interconnected realms of containers and the cloud have given rise to innovative security patterns designed to address the unique challenges posed by dynamic, distributed environments. Explore the latest patterns, anti-patterns, and practices that are steering the course in an era where cloud-native architecture, including orchestration intricacies of Kubernetes that span across Amazon Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), Google Kubernetes Engine (GKE), including nuances of securing microservices. Related: Amazon ETL Tools Compared. What Is Container Security? Container security is the practice of ensuring that container environments are protected against any threats. As with any security implementation within the software development lifecycle (SDLC), the practice of securing containers is a crucial step to take, as it not only protects against malicious actors but also allows containers to run smoothly in production. Learn how to incorporate CI/CD pipelines into your SDLC. The process of securing containers is a continuous one and can be implemented on the infrastructure level, runtime, and the software supply chain, to name a few. As such, securing containers is not a one-size-fits-all approach. In future sections, we will discuss different container management strategies and how security comes into play. Review additional CI/CD design patterns. How to Build a Container Strategy With Security Forensics Embdedded A container management strategy involves a structured plan to oversee the creation, deployment, orchestration, maintenance, and discarding of containers and containerized applications. It encompasses key elements to ensure efficiency, security, and scalability throughout the software development lifecycle based around containerization. Let's first analyze the prevailing and emerging anti-patterns for container management and security. Then, we will try to correlate possible solutions or alternative recommendations corresponding to each anti-pattern along with optimization practices for fortifying container security strategies for today's and tomorrow's threats. Review more DevOps anti-pattern examples. "Don't treat container security like a choose-your-own-adventure book; following every path might lead to a comedy of errors, not a happy ending!" Container Security Best Practices Weak Container Supply Chain Management This anti-pattern overlooks container supply chain management visible in "Docker history," risking compromised security. Hastily using unofficial Docker images without vetting their origin or build process poses a significant threat. Ensuring robust container supply chain management is vital for upholding integrity and security within the container environment. Learn how to perform a docker container health check. Anti-Pattern: Potential Compromise Pushing malicious code into Docker images is straightforward, but detecting such code is challenging. Blindly using others' images or building new ones from these can risk security, even if they solve similar problems. Pattern: Secure Practices Instead of relying solely on others' images, inspect their Dockerfiles, emulate their approach, and customize them for your needs. Ensure FROM lines in the Dockerfile point to trusted images, preferably official ones or those you've crafted from scratch, despite the added effort, ensuring security over potential breach aftermaths. Installing Non-Essential Executables Into a Container Image Non-essential executables for container images encompass anything unnecessary for the container's core function or app interpreter. For production, omit tools like text editors. Java or Python apps may need specific executables, while Go apps can run directly from a minimal "scratch" base image. Anti-Pattern: Excessive Size Adding non-essential executables to a container amplifies vulnerability risks and enlarges image size. This surplus bulk slows pull times and increases network data transmission. Pattern: Trim the Fat Start with a minimal official or self-generated base image to curb potential threats. Assess your app's true executable necessities, avoiding unnecessary installations. Exercise caution while removing language-dependent executables to craft a lean, cost-effective container image. Cloning an Entire Git Repo Into a Container Image It could look something like : GitHub Flavored Markdown RUN git clone https://github.org/somerepo Anti-Pattern: Unnecessary Complexity External dependency: Relying on non-local sources for Docker image files introduces risk, as these files may not be vetted beforehand. Git clutter: A git clone brings surplus files like the .git/ directory, increasing image size. The .git/ folder may contain sensitive information, and removing it is error-prone. Network dependency: Depending on container engine networking to fetch remote files adds complexity, especially with corporate proxies, potentially causing build errors. Executable overhead: Including the Git executable in the image is unnecessary unless directly manipulating Git repositories. Pattern: Streamlined Assembly Instead of a direct git clone in the Dockerfile, clone to a sub-directory in the build context via a shell script. Then, selectively add needed files using the COPY directive, minimizing unnecessary components. Utilize a .dockerignore file to exclude undesired files from the Docker image. Exception: Multi-Stage Build For a multi-stage build, consider cloning the repository to a local folder and then copying it to the build-stage container. While git clone might be acceptable, this approach offers a more controlled and error-resistant alternative. Building a Docker Container Image “On the Fly” Anti-Pattern: Skipping Registry Deployment Performing cloning, building, and running a Docker image without pushing it to an intermediary registry is an anti-pattern. This skips security screenings, lacks a backup, and introduces untested images to deployment. The main reason is that there are security and testing gaps: Backup and rollback: Skipping registry upload denies the benefits of having a backup, which is crucial for quick rollbacks in case of deployment failures. Vulnerability scanning: Neglecting registry uploads means missing out on vulnerability scanning, a key element in ensuring data and user safety. Untested images: Deploying unpushed images means deploying untested ones, a risky practice, particularly in a production environment. DZone's previously covered how to use penetration tests within an organization. Pattern: Registry Best Practices Build and uniquely version images in a dedicated environment, pushing them to a container registry. Let the registry scan for vulnerabilities and ensure thorough testing before deployment. Utilize deployment automation for seamless image retrieval and execution. Running as Root in the Container Anti-Pattern: Defaulting to Root User Many new container users inadvertently run containers with root as the default user, a practice necessitated by container engines during image creation. This can lead to the following security risks: Root user vulnerabilities: Running a Linux-based container as root exposes the system to potential takeovers and breaches, allowing bad actors access inside the network and potentially the container host system. Container breakout risk: A compromised container could lead to a breakout, granting unauthorized root access to the container host system. Pattern: User Privilege Management Instead of defaulting to root, use the USER directive in the Dockerfile to specify a non-root user. Prior to this, ensure the user is created in the image and possesses adequate permissions for required commands, including running the application. This practice reduces security vulnerabilities associated with root user privileges. Running Multiple Services in One Container Anti-Pattern: Co-Locating Multiple Tiers This anti-pattern involves running multiple tiers of an application, such as APIs and databases, within the same container, contradicting the minimalist essence of container design. The complexity and deviation from the design cause the following challenges: Minimalism violation: Containers are meant to be minimalistic instances, focusing on the essentials for running a specific application tier. Co-locating services in a single container introduces unnecessary complexity. Exit code management: Containers are designed to exit when the primary executable ends, relaying the exit code to the launching shell. Running multiple services in one container requires manual management of unexpected exceptions and errors, deviating from container engine handling. Pattern: Service Isolation Adopt the principle of one container per task, ensuring each container hosts a single service. Establish a local virtualized container network (e.g., docker network create) for intra-container communication, enabling seamless interaction without compromising the minimalist design of individual containers. Embedding Secrets in an Image Anti-Pattern: Storing Secrets in Container Images This anti-pattern involves storing sensitive information, such as local development secrets, within container images, often overlooked in various parts like ENV directives in Dockerfiles. This causes the following security compromises: Easy to forget: Numerous locations within container images, like ENV directives, provide hiding spots for storing information, leading to inadvertent negligence and forgetfulness. Accidental copy of secrets: Inadequate precautions might result in copying local files containing secrets, such as .env files, into the container image. Pattern: Secure Retrieval at Runtime Dockerignore best practices: Implement a .dockerignore file encompassing local files housing development secrets to prevent inadvertent inclusion in the container image. This file should also be part of .gitignore. Dockerfile security practices: Avoid placing secrets in Dockerfiles. For secure handling during build or testing phases, explore secure alternatives to passing secrets via --build-arg, leveraging Docker's BuildKit for enhanced security. Runtime secret retrieval: Retrieve secrets at runtime from secure stores like HashiCorp Vault, cloud-based services (e.g., AWS KMS), or Docker's built-in secrets functionality, which requires a docker-swarm setup for utilization. Failing to Update Packages When Building Images Anti-Pattern: Static Base Image Packages This anti-pattern stems from a former best practice where container image providers discouraged updating packages within base images. However, the current best practice emphasizes updating installed packages every time a new image is built. The main reason for this is outdated packages, which causes lagging updates. Base images may not always contain the latest versions of installed packages due to periodic or scheduled image builds, leaving systems vulnerable to outdated packages, including security vulnerabilities. Pattern: Continuous Package Updates To address this, regularly update installed packages using the distribution's package manager within the Dockerfile. Incorporate this process early in the build, potentially within the initial RUN directive, ensuring that each new image build includes updated packages for enhanced security and stability. When striving to devise a foolproof solution, a frequent misstep is to undervalue the resourcefulness of total novices. Building Container Security Into Development Pipelines Creates a Dynamic Landscape In navigating the ever-evolving realm of containers, which are at an all-time high in popularity and directly proportional to the quantum of security threats, we've delved into a spectrum of crucial patterns and anti-patterns. From fortifying container images by mastering the intricacies of supply chain management to embracing the necessity of runtime secrets retrieval, each pattern serves as a cornerstone in the architecture of robust container security. Unraveling the complexities of co-locating services and avoiding the pitfalls of outdated packages, we've highlighted the significance of adaptability and continuous improvement. As we champion the ethos of one-container-per-task and the secure retrieval of secrets, we acknowledge that container security is not a static destination but an ongoing journey. By comprehending and implementing these patterns, we fortify our containers against potential breaches, ensuring a resilient and proactive defense in an ever-shifting digital landscape.

By Pratik Prakash CORE

Does the time your CI/CD pipeline takes to deploy hold you back during development testing? This article demonstrates a faster way to develop Spring Boot microservices using a bare-metal Kubernetes cluster that runs on your own development machine. Recipe for Success This is the fourth article in a series on Ansible and Kubernetes. In the first post, I explained how to get Ansible up and running on a Linux virtual machine inside Windows. Subsequent posts demonstrated how to use Ansible to get a local Kubernetes cluster going on Ubuntu 20.04. It was tested on both native Linux- and Windows-based virtual machines running Linux. The last-mentioned approach works best when your devbox has a separate network adaptor that can be dedicated for use by the virtual machines. This article follows up on concepts used during the previous article and was tested on a cluster consisting of one control plane and one worker. As such a fronting proxy running HAProxy was not required and commented out in the inventory. The code is available on GitHub. When to Docker and When Not to Docker The secret to faster deployments to local infrastructure is to cut out on what is not needed. For instance, does one really need to have Docker fully installed to bake images? Should one push the image produced by each build to a formal Docker repository? Is a CI/CD platform even needed? Let us answer the last question first. Maven started life with both continuous integration and continuous deployment envisaged and should be able to replace a CI/CD platform such as Jenkins for local deployments. Now, it is widely known that all Maven problems can either be resolved by changing dependencies or by adding a plugin. We are not in jar-hell, so the answer must be a plugin. The Jib build plugin does just this for the sample Spring Boot microservice we will be deploying: <build> <plugins> <plugin> <groupId>com.google.cloud.tools</groupId> <artifactId>jib-maven-plugin</artifactId> <version>3.1.4</version> <configuration> <from> <image>openjdk:11-jdk-slim</image> </from> <to> <image>docker_repo:5000/rbuhrmann/hello-svc</image> <tags> <tag>latest10</tag> </tags> </to> <allowInsecureRegistries>false</allowInsecureRegistries> </configuration> </plugin> </plugins> </build> Here we see how the Jib Maven plugin is configured to bake and push the image to a private Docker repo. However, the plugin can be steered from the command line as well. This Ansible shell task loops over one or more Spring Boot microservices and does just that: - name: Git checkouts ansible.builtin.git: repo: "{{ item.git_url }" dest: "~/{{ item.name }" version: "{{ item.git_branch }" loop: "{{ apps }" **************** - name: Run JIB builds ansible.builtin.command: "mvn clean compile jib:buildTar -Dimage={{ item.name }:{{ item.namespace }" args: chdir: "~/{{ item.name }/{{ item.jib_dir }" loop: "{{ apps }" The first task clones, while the last integrates the Docker image. However, it does not push the image to a Docker repo. Instead, it dumps it as a tar ball. We are therefore halfway towards removing the Docker repo from the loop. Since our Kubernetes cluster uses Containerd, a spinout from Docker, as its container daemon, all we need is something to load the tar ball directly into Containerd. It turns out such an application exists. It is called ctr and can be steered from Ansible: - name: Load images into containerd ansible.builtin.command: ctr -n=k8s.io images import jib-image.tar args: chdir: "/home/ansible/{{ item.name }/{{ item.jib_dir }/target" register: ctr_out become: true loop: "{{ apps }" Up to this point, task execution has been on the worker node. It might seem stupid to build the image on the worker node, but keep in mind that: It concerns local testing and there will seldom be a need for more than one K8s worker - the build will not happen on more than one machine. The base image Jib builds from is smaller than the produced image that normally is pulled from a Docker repo. This results in a faster download and a negligent upload time since the image is loaded directly into the Container daemon of the worker node. The time spent downloading Git and Maven is amortized over all deployments and therefore makes up less and less percentage of time as usage increases. Bypassing a CI/CD platform such as Jenkins or Git runners shared with other applications can save significantly on build and deployment time. You Are Deployment, I Declare Up to this point, I have only shown the Ansible tasks, but the variable declarations that are ingested have not been shown. It is now an opportune time to list part of the input: apps: - name: hello1 git_url: https://github.com/jrb-s2c-github/spinnaker_tryout.git jib_dir: hello_svc image: s2c/hello_svc namespace: env1 git_branch: kustomize application_properties: application.properties: | my_name: LocalKubeletEnv1 - name: hello2 git_url: https://github.com/jrb-s2c-github/spinnaker_tryout.git jib_dir: hello_svc image: s2c/hello_svc namespace: env2 config_map_path: git_branch: kustomize application_properties: application.properties: | my_name: LocalKubeletEnv2 It concerns the DevOps characteristics of a list of Spring Boot microservices that steer Ansible to clone, integrate, deploy, and orchestrate. We already saw how Ansible handles the first three. All that remains are the Ansible tasks that create Kubernetes deployments, services, and application.properties ConfigMaps: - name: Create k8s namespaces remote_user: ansible kubernetes.core.k8s: kubeconfig: /home/ansible/.kube/config name: "{{ item.namespace }" api_version: v1 kind: Namespace state: present loop: "{{ apps }" - name: Create application.property configmaps kubernetes.core.k8s: kubeconfig: /home/ansible/.kube/config namespace: "{{ item.namespace }" state: present definition: apiVersion: v1 kind: ConfigMap metadata: name: "{{ item.name }-cm" data: "{{ item.application_properties }" loop: "{{ apps }" - name: Create deployments kubernetes.core.k8s: kubeconfig: /home/ansible/.kube/config namespace: "{{ item.namespace }" state: present definition: apiVersion: apps/v1 kind: Deployment metadata: creationTimestamp: null labels: app: "{{ item.name }" name: "{{ item.name }" spec: replicas: 1 selector: matchLabels: app: "{{ item.name }" strategy: { } template: metadata: creationTimestamp: null labels: app: "{{ item.name }" spec: containers: - image: "{{ item.name }:{{ item.namespace }" name: "{{ item.name }" resources: { } imagePullPolicy: IfNotPresent volumeMounts: - mountPath: /config name: config volumes: - configMap: items: - key: application.properties path: application.properties name: "{{ item.name }-cm" name: config status: { } loop: "{{ apps }" - name: Create services kubernetes.core.k8s: kubeconfig: /home/ansible/.kube/config namespace: "{{ item.namespace }" state: present definition: apiVersion: v1 kind: List items: - apiVersion: v1 kind: Service metadata: creationTimestamp: null labels: app: "{{ item.name }" name: "{{ item.name }" spec: ports: - port: 80 protocol: TCP targetPort: 8080 selector: app: "{{ item.name }" type: ClusterIP status: loadBalancer: {} loop: "{{ apps }" These tasks run on the control plane and configure the orchestration of two microservices using the kubernetes.core.k8s Ansible task. To illustrate how different feature branches of the same application can be deployed simultaneously to different namespaces, the same image is used. However, each is deployed with different content in its application.properties. Different Git branches can also be specified. It should be noted that nothing prevents us from deploying two or more microservices into a single namespace to provide the backend services for a modern JavaScript frontend. The imagePullPolicy is set to "IfNotPresent". Since ctr already deployed the image directly to the container runtime, there is no need to pull the image from a Docker repo. Ingress Routing Ingress instances are used to expose microservices from multiple namespaces to clients outside of the cluster. The declaration of the Ingress and its routing rules are lower down in the input declaration partially listed above: ingress: host: www.demo.io rules: - service: hello1 namespace: env1 ingress_path: /env1/hello service_path: / - service: hello2 namespace: env2 ingress_path: /env2/hello service_path: / Note that the DNS name should be under your control or not be entered as a DNS entry on a DNS server anywhere in the world. Should this be the case, the traffic might be sent out of the cluster to that IP address. The service variable should match the name of the relevant microservice in the top half of the input declaration. The ingress path is what clients should use to access the service and the service path is the endpoint of the Spring controller that should be routed to. The Ansible tasks that interpret and enforce the above declarations are: - name: Create ingress master kubernetes.core.k8s: kubeconfig: /home/ansible/.kube/config namespace: default state: present definition: apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: ingress-master annotations: nginx.org/mergeable-ingress-type: "master" spec: ingressClassName: nginx rules: - host: "{{ ingress.host }" - name: Create ingress minions kubernetes.core.k8s: kubeconfig: /home/ansible/.kube/config namespace: "{{ item.namespace }" state: present definition: apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: nginx.ingress.kubernetes.io/rewrite-target: " {{ item.service_path } " nginx.org/mergeable-ingress-type: "minion" name: "ingress-{{ item.namespace }" spec: ingressClassName: nginx rules: - host: "{{ ingress.host }" http: paths: - path: "{{ item.ingress_path }" pathType: Prefix backend: service: name: "{{ item.service }" port: number: 80 loop: "{{ ingress.rules }" We continue where we left off in my previous post and use Nginx Ingress Controller and MetalLB to establish Ingress routing. Once again, the use is made of the Ansible loop construct to cater to multiple routing rules. In this case, routing will proceed from the /env1/hello route to the Hello K8s Service in the env1 namespace and from the /env2/hello route to the Hello K8s Service in the env2 namespace. Routing into different namespaces is achieved using Nginx mergeable ingress types. More can be read here, but basically, one annotates Ingresses as being the master or one of the minions. Multiple instances thus combine together to allow for complex routing as can be seen above. The Ingress route can and probably will differ from the endpoint of the Spring controller(s). This certainly is the case here and a second annotation was required to change from the Ingress route to the endpoint the controller listens on: nginx.ingress.kubernetes.io/rewrite-target: " {{ item.service_path } " This is the sample controller: @RestController public class HelloController { @RequestMapping("/") public String index() { return "Greetings from " + name; } @Value(value = "${my_name}") private String name; } Since the value of the my_name field is replaced from what is defined in the application.properties and each instance of the microservice has a different value for it, we would expect a different welcome message from each of the K8S Services/Deployments. Hitting the different Ingress routes, we see this is indeed the case: On Secrets and Such It can happen that your Git repository requires token authentication. For such cases, one should add the entire git URL to the Ansible vault: apps: - name: mystery git_url: "{{ vault_git_url }" jib_dir: harvester image: s2c/harvester namespace: env1 git_branch: main application_properties: application.properties: | my_name: LocalKubeletEnv1 The content of variable vault_git_url is encrypted in all/vault.yaml and can be edited with: ansible-vault edit jetpack/group_vars/all/vault.yaml Enter the password of the vault and add/edit the URL to contain your authentication token: vault_git_url: https://AUTH TOKEN@github.com/jrb-s2c-github/demo.git Enough happens behind the scenes here to warrant an entire post. However, in short, group_vars are defined for inventory groups with the vars and vaults for each inventory group in its own sub-directory of the same name as the group. The "all" sub-folder acts as the catchall for all other managed servers that fall out of this construct. Consequently, only the "all" sub-directory is required for the master and workers groups of our inventory to use the same vault. It follows that the same approach can be followed to encrypt any secrets that should be added to the application.properties of Spring Boot. Conclusion We have seen how to make deployments of Sprint Boot microservices to local infrastructure faster by bypassing certain steps and technologies used during the CI/CD to higher environments. Multiple namespaces can be employed to allow the deployment of different versions of a micro-service architecture. Some thought will have to be given when secrets for different environments are in play though. The focus of the article is on a local environment and a description of how to use group vars to have different secrets for different environments is out of scope. It might be the topic of a future article. Please feel free to DM me on LinkedIn should you require assistance to get the rig up and running. Thank you for reading!

It has been over two months since the Kubernetes Gateway API made its v1.0 release, signifying graduation to the generally available status for some of its key APIs. I wrote about the Gateway API when it graduated to beta last year, but a year later, the question remains. Should you switch to the Gateway API from the Ingress API? My answer from last year was you shouldn't. And I had strong reasons. The Gateway API and its implementations were still in their infancy. The Ingress API, on the other hand, was stable and covered some primary use cases that might work for most users. For users requiring more capabilities, I suggested using the custom resources provided by the Ingress controllers by trading off portability (switching between different Ingress implementations). With the v1.0 release, this might change. The Gateway API is much more capable now, and its 20+ implementations are catching up quickly. So, if you are starting anew and choosing between the Ingress and the Gateway API, I suggest you pick the Gateway API if the API and the implementation you choose support all the features you want. What’s Wrong With the Ingress API? The Ingress API works very well, but only for a small subset of common use cases. To extend its capabilities, Ingress implementations started using custom annotations. For example, if you chose Nginx Ingress, you will use some of its dozens of annotations that are not portable if you decide to switch to another Ingress implementation like Apache APISIX. These implementation-specific annotations are also cumbersome to manage and defeat the purpose of managing Ingress in a Kubernetes-native way. Eventually, Ingress controller implementations started developing their CRDs to expose more features to Kubernetes users. These CRDs are specific to the Ingress controller. But if you can sacrifice portability and stick to one Ingress controller, the CRDs are easier to work with and offer more features. The Gateway API aims to solve this problem once and for all by providing the vendor agnosticism of the Ingress API and the flexibility of the CRDs. It is positioned very well to achieve this goal. In the long term, the Ingress API is not expected to receive any new features, and all efforts will be made to converge with the Gateway API. So, adopting the Ingress API can cause issues when you inadvertently hit limits with its capabilities. Obvious Benefits Expressive, extensible, and role-oriented are the key ideas that shaped the development of the Gateway API. Unlike the Ingress API, the Gateway API is a collection of multiple APIs (HTTPRoute, Gateway, GatewayClass, etc.), each catering to different organizational roles. For example, the application developers need to only care about the HTTPRoute resource, where they can define rules to route traffic. They can delegate the cluster-level details to an operator who manages the cluster and ensures that it meets the developers' needs using the Gateway resource. Adapted from gateway-api.sigs.k8s.io This role-oriented design of the API allows different people to use the cluster while maintaining control. The Gateway API is also much more capable than the Ingress API. Features that require annotations in the Ingress API are supported out-of-the-box in the Gateway API. An Official Extension Although the Gateway API is an official Kubernetes API, it is implemented as a set of CRDs. This is no different from using default Kubernetes resources. But you just have to install these CRDs like an official extension. The Ingress controller translates the Kubernetes resources to APISIX configuration implemented by API gateway. This allows for fast iteration compared to Kubernetes, which is slowly moving toward long-term stability. Will It Proliferate? As this famous XKCD comic reminds us frequently, standards tend to proliferate. A version of this was seen in the Ingress and Gateway APIs. It usually goes like this: A standard emerges to unify different projects/their standards (Kubernetes Ingress API). The unified standard has limitations the implementors want to overcome (Ingress API was limited). Implementations diverge from the standard because of these limitations (Custom CRDs, annotations). Each implementation now has its standard (non-portable CRDs, annotations). A new standard emerges to unify these different standards (Kubernetes Gateway API). It is reasonable to think that the Gateway API might not be the end game here. But I believe it has every chance of being the standard for routing in Kubernetes. Again, I have my strong reasons. Broad adoption is critical to prevent standard proliferation as there are fewer incentives for the implementations to work on a different standard. The Gateway API already has more than 25 implementations. An implementation can conform to the Gateway API on different levels: Core: All implementations are expected to conform to these. Extended: These might only be available in some implementations but are standard APIs. Implementation-specific: Specific to implementations but added through standard extension points. A niche feature can move from implementation-specific to extended to the core as more implementations support these features. i.e., the API allows room for custom extensions while ensuring it follows the standard. The Service Mesh Interface (SMI) project was a similar attempt to standardize configuring service meshes in Kubernetes. However, the project received little traction after the initial involvement of the service mesh projects and slowly died out. SMI did not support many common denominator features that users expected in a service mesh. It also did not move fast enough to support these features. Eventually, service mesh implementations fell behind in conforming to SMI (I used to work closely with SMI under the CNCF TAG Network on a project that reported SMI conformance). These are universal reasons, but the project is now being resurrected through the Gateway API. The Gateway API for Mesh Management and Administration (GAMMA) initiative aims to extend the Gateway API to work with service meshes. The SMI project recently merged with the GAMMA initiative, which is excellent for the Gateway API. Istio, undoubtedly the most popular service mesh, also announced that the Gateway API will be the default API to manage Istio in the future. Such adoptions prevent proliferation. Migration Guide The Gateway API documentation has a comprehensive guide on migrating your Ingress resources to Gateway resources. Instead of restating it, let's try using the ingress2gateway tool to convert our Ingress resources to corresponding Gateway API resources. You can download and install the binary for your operating system directly from the releases page. Let's take a simple Ingress resource: YAML apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: httpbin-route spec: ingressClassName: apisix rules: - host: local.httpbin.org http: paths: - backend: service: name: httpbin port: number: 80 path: / pathType: Prefix This will route all traffic with the provided host address to the httpbin service. To convert it to the Gateway API resource, we can run: Shell ingress2gateway print --input_file=ingress.yaml This Gateway API resource will be as shown below: YAML apiVersion: gateway.networking.k8s.io/v1alpha2 kind: HTTPRoute metadata: name: httpbin-route spec: hostnames: - local.httpbin.org rules: - matches: - path: type: PathPrefix value: / backendRefs: - name: httpbin port: 80 Viable Alternatives There are other viable alternatives for configuring gateways in Kubernetes. In Apache APISIX, you can deploy it in standalone mode and define route configurations in a YAML file. You can update this YAML file through traditional workflows, and it can be pretty helpful in scenarios where managing the gateway configuration via the Kubernetes API is not required. Implementation-specific custom CRDs are also viable alternatives if you don't plan to switch to a different solution or if your configuration is small enough to migrate easily. In any case, the Gateway API is here to stay.

Are you excited to become a Cloud Solutions Architect and take your career to new heights? Cloud computing is transforming the way organizations use digital infrastructure, making it a crucial skill to master. If you are interested in the limitless potential of cloud technology, then this guide is tailor-made for you. In this guide, you’ll learn about the top 9 role-based cloud certifications, specifically curated for Solutions Architects. As we approach 2024, we are on the cusp of an exciting era in cloud technology. Together, we will explore nine paramount certifications offered by industry leaders and esteemed organizations, each of which is a stepping stone on your journey to becoming a certified cloud professional. Whether you are a seasoned IT professional looking to advance your career or a university student launching your tech career, these certifications are your gateway to the world of cloud computing. So, fasten your seatbelts, and let's take a flight to explore the world of cloud solutions architecture! Top 9 Role-Based Cloud Certifications for Solution Architects The following are nine role-based cloud certifications that, when earned, can help you secure a high-paying job by 2024. 1. AWS Cloud Practitioner 2. AWS Solutions Architect Associate 3. AWS Solutions Architect Professional 4. Microsoft Certified: Azure Fundamentals 5. Microsoft Certified: Azure Solutions Architect Expert 6. Google Cloud Certified - Associate Cloud Engineer 7. Google Cloud Certified - Professional Cloud Architect 8. IBM Certified Technical Advocate - Cloud v4 9. CompTIA Cloud+ 1. AWS Certified Cloud Practitioner The AWS Cloud Practitioner certification, provided by Amazon Web Services (AWS), is the first step into the world of cloud computing. It is specially designed for individuals who are new to both AWS and cloud computing in general, and it provides candidates with a fundamental understanding of AWS services and basic cloud concepts. Topics Covered The key topics covered include: Fundamentals of Cloud Computing Overview of AWS global infrastructure AWS core services and their use cases Best practices for architectural design in the cloud Prerequisites This certification is open to candidates without prior AWS experience or technical background. It truly serves as an entry point for those eager to embark on a cloud computing journey. Pricing 100 USD. 2. AWS Solutions Architect Associate The certification AWS Certified Solutions Architect – Associate is intended for those who possess a deep understanding of the AWS platform and are looking to create systems that are scalable, highly available, and fault-tolerant. This certification validates your ability to manage, plan, and create AWS cloud-based systems, making it an ideal starting point for anyone interested in cloud technology. According to Skillsoft, individuals who hold the AWS Certified Solutions Architect-Associate certification earn an average salary of approximately $148,348, with some earning up to $150,000. However, this certification is not just about the financial rewards. Topics Covered In-depth exploration of AWS services Architectural design principles on AWS Cloud design patterns and their applications Adherence to the AWS Well-Architected Framework Prerequisites While there are no official prerequisites, having a basic understanding of AWS services is recommended. Pricing 150 USD. 3. AWS Solutions Architect Professional The AWS Certified Solutions Architect – Professional certification is designed for individuals who not only possess an in-depth understanding of AWS but also can design, deploy, and scale complex applications effortlessly within the AWS environment. This certification indicates that you are an expert in building systems that are not only scalable but also highly available and fault-tolerant in the ever-changing world of cloud computing. To obtain this prestigious certification, you must successfully pass a challenging multiple-choice exam. Professionals who hold the AWS Certified Solutions Architect – Professional certification earn an impressive average salary of around $158,485. Yes, you read that correctly. This certification is one of the most valuable credentials in cloud technology. With this certification, you can expect to earn a substantial salary that reflects your advanced skills and expertise in AWS. Topics Covered Advanced AWS architectural principles Hybrid cloud architectures and their implementation Strategies for cost optimization in the cloud High availability and fault tolerance considerations Prerequisites Candidates must hold the AWS Certified Solutions Architect - Associate certification as a prerequisite. Pricing The AWS Solutions Architect Professional exam fee is 300 USD. 4. Microsoft Certified: Azure Fundamentals Microsoft Certified: Azure Fundamentals Topics Covered Overview of Azure cloud services Azure pricing and SLAs (Service Level Agreements) Governance and compliance in Azure Identity and access management in Azure Prerequisites No prior experience or certification is required to pursue the Azure Fundamentals certification. Pricing The Microsoft Certified: Azure Fundamentals exam carries a fee of 99 USD. 5. Microsoft Certified: Azure Solutions Architect Expert The certification Microsoft Certified: Azure Solutions Architect Expert is specifically designed for individuals who possess extensive knowledge of the Azure platform and are capable of designing and implementing solutions on it. This certification is proof of the holder's ability to create and execute scalable, highly available, and secure solutions on Azure. To be certified, candidates must pass multiple-choice exams that test their comprehension of Azure architecture, tools, and services, as well as their ability to design and implement Azure solutions. Topics Covered Designing Azure infrastructure Implementation of solutions in Azure Security considerations and best practices in Azure Governance and compliance in Azure Prerequisites Candidates must first obtain the Microsoft Certified: Azure Administrator Associate certification. Pricing The Azure Solutions Architect Expert certification consists of two exams, each priced at USD 165. 6. Google Cloud Certified — Associate Cloud Engineer The accreditation of Google Cloud Certified – Associate Cloud Engineer is considered to be one of the most prestigious certifications for individuals who possess a comprehensive understanding of the Google Cloud Platform and its features. It validates their ability to efficiently deploy, manage, and troubleshoot cloud infrastructure issues. The certification is an assurance of the candidate's expertise in managing solutions on the Google Cloud Platform utilizing various services such as Google Compute Engine and Google Cloud Storage. To obtain the certification, the candidate must successfully pass a multiple-choice exam that assesses their knowledge of the Google Cloud Platform and its various services. The GCP-ACE certification holders are highly sought after by employers as it confirms their profound understanding of the Google Cloud Platform and their ability to deploy, manage, and troubleshoot cloud infrastructure. On average, individuals who hold this certification earn approximately $107,272 per year. This credential is particularly beneficial for professionals who are interested in specializing in the Google Cloud Platform and acquiring the essential skills to manage and deploy solutions on it. Obtaining this certification demonstrates the candidate's ability to perform tasks such as managing Google Cloud Platform services, ensuring successful operations, and analyzing data processing systems. Topics Covered Overview of Google Cloud services and products Google Cloud Identity and Access Management (IAM) Resource management and configuration in Google Cloud Networking concepts and practices in Google Cloud Prerequisites While no formal prerequisites exist, a basic understanding of Google Cloud Platform basics is recommended. Pricing The Google Associate Cloud Engineer exam fee is set at USD 125. 7. Google Cloud Certified - Professional Cloud Architect The Google Cloud Certified – Professional Cloud Architect certification is specifically designed for those individuals who possess a comprehensive understanding of the Google Cloud Platform and are capable of creating, designing, and managing solutions within it. This certification is a testament to the holder's ability to design and implement highly secure, scalable, and available solutions utilizing Google Cloud services such as Google Compute Engine, Google Kubernetes Engine, and Google Cloud Storage. To qualify for this certification, one must pass a multiple-choice exam that tests their knowledge of the Google Cloud Platform and its various services, as well as their capability to design and implement solutions within the platform. Individuals who have the Google Cloud Certified - Professional Cloud Architect certification often earn an average salary of around $166,057 per year. This certification is widely recognized within the industry and serves as an indication of the holder's comprehensive understanding of the Google Cloud Platform as well as their ability to create, design, and manage solutions within it. This certification is particularly valuable for those individuals who wish to specialize in cloud architecture and acquire the necessary skills to design and implement highly secure, scalable, and available solutions within the Google Cloud Platform. Topics Covered In-depth understanding of Google Cloud services and products Architectural design considerations for GCP Security and compliance in GCP Data storage and processing strategies in GCP Prerequisites While not mandatory, relevant experience is highly beneficial for candidates pursuing the Google Professional Cloud Architect certification. Pricing The Google Professional Cloud Architect exam fee is USD 200. 8. IBM Certified Technical Advocate — Cloud V4 The IBM Certified Technical Advocate — Cloud v4 certification is a highly sought-after credential in the IT industry, recognized globally for its value and relevance. It is a testament to your expertise in IBM Cloud technologies and your ability to deliver successful cloud solutions. With this certification, you'll have an added advantage in the job market to take on challenging roles in the field of cloud computing. Topics Covered IBM Cloud services and offerings Cloud architecture and design IBM Cloud security and compliance Cloud deployment and management Prerequisites USD 100 to USD 200, depending on the location. Pricing The exam fee for the IBM Certified Technical Advocate - Cloud v4 certification is USD 200. 9. CompTIA Cloud+ The CompTIA Cloud+ certification is a credential that confirms the proficiency of an individual in designing, implementing, and maintaining cloud infrastructure. It is intended for professionals who possess a comprehensive understanding of cloud computing and can apply cloud concepts to real-world scenarios. The certification test evaluates an individual's understanding of cloud computing, encompassing cloud principles, cloud infrastructure, and cloud security. A passing score on the exam indicates that the candidate has the necessary skills to establish, maintain, and safeguard cloud-based environments. Topics Covered Cloud concepts and terminology Cloud infrastructure and virtualization Cloud management and security Troubleshooting in cloud environments Prerequisites While not mandatory, having some experience in IT networking or storage is beneficial. Pricing The CompTIA Cloud+ exam fee typically ranges from USD 319 to USD 338, depending on your location. Choosing the Right Certification Path Selecting the appropriate certification is a pivotal decision on your journey toward a cloud computing career. It should align with your current knowledge, experience, and long-term career goals. Here are some considerations: Entry-Level: If you are new to cloud computing, starting with the AWS Cloud Practitioner, Microsoft Certified: Azure Fundamentals, Google Associate Cloud Engineer, IBM Certified Technical Advocate - Cloud v4, or CompTIA Cloud+ certifications are advisable. Architectural Focus: Aspiring cloud architects should target certifications like AWS Solutions Architect Associate/Professional, Microsoft Certified: Azure Solutions Architect Expert, Google Professional Cloud Architect, or IBM Certified Technical Advocate - Cloud v4. Career Goals: Ensure your certification choice aligns with your career aspirations, whether you intend to specialize in a specific cloud platform or seek broader knowledge. Conclusion Role-based cloud certifications for Solution Architects have assumed a pivotal role in today's competitive job market. They provide a structured path for acquiring specialized knowledge and skills, making you a valuable asset to employers. Whether you choose to embark on a journey with AWS, Microsoft Azure, Google Cloud, or IBM Cloud or opt for a vendor-neutral certification like CompTIA Cloud+, these certifications serve as guiding stars on your path to becoming a certified cloud professional. Take the first step today by selecting the certification that best suits your aspirations and start your transformative journey toward becoming a certified cloud professional. Your future in cloud computing beckons, and these certifications are the keys to unlocking your potential in this dynamic and ever-expanding field.

Welcome to the first post in our exciting series on mastering offline data pipeline's best practices, focusing on the potent combination of Apache Airflow and data processing engines like Hive and Spark. This post focuses on elevating our data engineering game, streamlining your data workflows, and significantly cutting computing costs. The need to optimize offline data pipeline optimization has become a necessity with the growing complexity and scale of modern data pipelines. In this kickoff post, we delve into the intricacies of Apache Airflow and AWS EMR, a managed cluster platform for big data processing. Working together, they form the backbone of many modern data engineering solutions. However, they can become a source of increased costs and inefficiencies without the right optimization strategies. Let's dive into the journey to transform your data workflows and embrace cost-efficiency in your data engineering environment. Why Focus on Airflow and Apache Hive? Before diving deep into our best practices, let us understand why we focus on the two specific technologies in our post. Airflow, an open-source platform, is a powerful tool for orchestrating complex computational workflows and data processing. On the other hand, AWS EMR (Elastic MapReduce) provides a managed cluster platform that simplifies running big data frameworks. Combined, they offer a robust environment for managing data pipelines but can incur significant costs if not optimized correctly. Apache Hive is widely recognized for its exceptional ability to efficiently manage and query massive datasets in offline data processing and warehousing scenarios. The architecture of Hive is optimized explicitly for batch processing of large data volumes, which is crucial in data warehousing scenarios. Hive is an optimal selection for organizations with significant big data and analytics demands due to its distributed storage and processing capabilities, enabling it to seamlessly handle data at a petabyte scale. Key Configuration Parameters for Apache Hive Jobs Timeouts Purpose: Prevents jobs from running indefinitely. Parameter: execution_timeout Python from datetime import timedelta from airflow.operators.hive_operator import HiveOperator hive_task = HiveOperator(task_id='hive_task' , hql='SELECT * FROM your_table;' , execution_timeout=timedelta(hours=2), ) Retries: Purpose: Handles transient errors by re-attempting the job. The number of retries that should be performed before failing the task. Parameter: retries Python hive_task = HiveOperator(task_id='hive_task', hql='SELECT * FROM your_table;', retries=3, ) Retry Delay Purpose: Sets the delay between retries. Parameter: retry_delay Python from datetime import timedelta hive_task = HiveOperator(task_id='hive_task', hql='SELECT * FROM your_table;', retry_delay=timedelta(minutes=5), ) Retry Exponential Backoff Purpose: allow progressive longer waits between retries by using an exponential backoff algorithm on retry delay (delay will be converted into seconds.) Parameter: retry_exponential_backoff Python from datetime import timedelta hive_task = HiveOperator(task_id='hive_task' , hql='SELECT * FROM your_table;' , retry_delay=timedelta(minutes=5) , retry_exponential_backoff=True ) Task Concurrency Purpose: Limits the number of tasks run simultaneously. Parameter: task_concurrency Python hive_task = HiveOperator(task_id='hive_task', hql='SELECT * FROM your_table;', task_concurrency=5, ) Best Practices for Job Backfilling Offline data pipeline backfilling in Hive, especially for substantial historical data, requires a strategic approach to ensure efficiency and accuracy. Here are some best practices: Incremental Load Strategy: Instead of backfilling all data simultaneously, break the process into smaller, manageable chunks. Incrementally loading data allows for better monitoring, easier error handling, and reduced resource strain. Leverage Hive's Merge Statement: For updating existing records during backfill, use Hive's MERGE statement. It efficiently updates and inserts data based on specific conditions, reducing the complexity of managing upserts. Data Validation and Reconciliation: Post-backfill and validate the data to ensure its integrity. Reconcile the backfilled data against source systems or use checksums to ensure completeness and accuracy. Resource Allocation and Throttling: Carefully plan the resource allocation for the backfill process. Utilize Hive's configuration settings to throttle the resource usage, ensuring it doesn't impact the performance of concurrent jobs. Error Handling and Retry Logic: Implement robust error handling and retry mechanisms. In case of failures, having a well-defined retry logic helps maintain the consistency of backfill operations. Refer to the retry parameters in the section above. Optimize Hive Queries: Use Hive query optimization techniques such as filtering early, minimizing data shuffling, and using appropriate file formats (like ORC or Parquet) for better compression and faster access. Conclusion Optimizing Airflow data pipelines on AWS EMR requires a strategic approach focusing on efficiency and cost-effectiveness. By tuning job parameters, managing retries and timeouts, and adopting best practices for job backfilling, organizations can significantly reduce their AWS EMR computing costs while maintaining high data processing standards. Remember, the key is continuous monitoring and optimization. Data pipeline requirements can change, and what works today might not be the best approach tomorrow. In the next post, we will learn how to fine-tune Spark jobs for optimal performance, job reliability, and cost savings. Stay agile, and keep optimizing.

Kubernetes Ingress has been around for a while. It helps to expose services in the cluster to the external world and provides basic traffic routing functionalities. But for advanced networking features DevOps and architects have to rely on vendors and their CRDs and custom annotations. If you have ever written custom annotations on NGINX Ingress, you know how tedious the process is. Luckily Kubernetes Gateway API is here to solve it and some other drawbacks with native Ingress. What Is Kubernetes Gateway API? Kubernetes Gateway API is a collection of resources that helps to standardize the specifications for implementing Gateway/Ingress rules in Kubernetes. Gateway API provides resources such as, GatewayClass, Gateway, and *Route, which are also role-delineated (see Fig. A). Fig.A – Kubernetes Gateway API resources Note that Gateway API only standardizes the specifications; for implementation, DevOps and architects will need an Ingress controller or a service mesh like Envoy Gateway or Istio. The resources help DevOps and architects to configure basic to advanced networking rules in Kubernetes without relying on vendor-specific CRDs and annotations. There are various implementors and integrators of K8s Gateway API. Challenges of Native Kubernetes Ingress Kubernetes Gateway API is introduced to solve some pressing challenges with native Kubernetes Ingress: Limited functionality: Native K8s Ingress only provides basic load-balancing functionality. Advanced traffic routing capabilities, such as path-based routing, require DevOps folks to configure tedious custom annotations provided by their respective Ingress controller. Lack of standardization: Vendors providing Ingress controllers have their annotation specifications. If DevOps has to switch Ingress controllers, they have a learning curve regarding vendor-specific CRDs and annotations. Besides, they will have to rewrite the current vendor’s specifications using the new ones. It takes a reasonable amount of time to write and test annotations before they can be implemented. Single Ingress resource: Usually, in a shared K8s cluster, there will be a single Ingress resource and multiple tenants will work on it. An admin will create multiple Ingress objects for various app teams at max, but it is still hard to implement role-based access controls on any object. If proper RBAC policies are not in place, teams may mess up the route configuration. Gateway API not only solves the challenges at the Gateway/Ingress level, but it could also evolve to manage service-to-service/east-west traffic within the same cluster. Kubernetes Gateway API Resources and the Request Flow Kubernetes Gateway API provides the following resources that give freedom for IT teams to work on resources that fall under their respective roles: GatewayClass (Infra Provider) The resource specifies the controller that implements the Gateway API CRDs associated with the class in the cluster. The Gateway API implementation controller is specified using controllerName, which then manages the GatewayClass. YAML apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: istio spec: controllerName: istio.io/gateway-controller In the above sample configuration, GatewayClass(es) the controllerName: istio.io/gateway-controller will be managed by the Istio service mesh. Gateway (Cluster admin/Architect) A Gateway resource acts as a network endpoint that gets the traffic inside the cluster, like a cloud load balancer. DevOps or Infra team can add multiple listeners to the external traffic and apply filters, TLS, and traffic forwarding rules. Gateway is attached to a GatewayClass and is implemented using the respective controller defined in GatewayClass. YAML apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: k8s-gateway namespace: k8s-gw spec: gatewayClassName: istio listeners: - name: default port: 80 protocol: HTTP allowedRoutes: namespaces: from: All In the above Gateway resource, gatewayClassName refers to the respective GatewayClass the resource is attached to, and listeners specify that the Gateway listens to HTTP traffic on port: 80. Route (Devs/DevOps) Route resources manage the traffic from the Gateway to the back-end services. Multiple Route resources, such as HTTPRoute, TCPRoute, GRPCRoute, etc., are used to configure the routing rules of the respective traffic from a Gateway listener to a backend service. The app team can configure different path names and Header filters (under the rules section in the YAML) in the Route resources to handle the traffic to the backend. Each Route can be attached to a single or multiple Gateways as per the requirements, which can be specified under parentRefs. YAML apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: k8s-http-route-with-istio namespace: with-istio spec: parentRefs: - name: k8s-gateway namespace: k8s-gw rules: - matches: - path: type: PathPrefix value: /with-istio backendRefs: - name: echoserver-service-with-istio port: 80 The rules field allows the set of advanced routing behaviors such as header-based matching, traffic splitting, balancing, etc. The above configuration routes HTTP traffic with the request path /with-istio from the k8s-gateway — to echoserver-service-with-istio service on port: 80, which is the destination. Combining the above resources, the request flow in Gateway API would look like this: a request would first come to the Gateway, from which *Route applies the routing rules before the request finally ends up on the respective backend service (see Fig. B). Fig. B – Request flow in Kubernetes Gateway API Benefits of K8s Gateway API By now, I hope you must be convinced of the role delineation benefits of Kubernetes Gateway API. Let us explore it and some other crucial benefits and features of Gateway API. Fig. C - Crucial benefits and features of Gateway API Proper Role Delineation and RBAC The cluster admin no longer needs to worry about developers accidentally introducing any wrong configuration to the Ingress resource, as they do not need to share the Gateway resource with anyone. Devs/DevOps can create *Routes and attach them to particular Gateways without distracting other teams’ routes. No Vendor Lock-in The standardization brought by Gateway API makes it seamless to switch between vendors providing controllers. DevOps and architects can use the same API to configure networking with the new vendor by changing the gatewayClassName in the Gateway resource. Almost all popular Gateway controllers and service meshes support integration with K8s Gateway API. Increased Developer Experience Advanced traffic routing rules can now be configured in *Routes Gateway API, which spares developers/DevOps from writing and rigorously testing vendor-specific CRDs and annotations. Besides, K8s Gateway API allows users to extend it by creating custom resources that suit their unique needs. How To Implement Kubernetes Gateway API With Istio Service Mesh In the demo below, I have a service deployed in both Istio-enabled and non-Istio namespace. I have used Gateway to let the traffic inside the cluster and HTTPRoute resource to apply path-based routing for the traffic — using Istio as the controller. Watch the tutorial to see Gateway API implementation for north-south traffic with Istio service mesh: (As a side note, Istio made it clear that Gateway API will be Istio’s default API for traffic management in the future.) Parting Thoughts I hope you have a good understanding of Gateway API by now. I deliberately left a few things to mention in this piece, like the GAMMA initiative and ReferenceGrant, to make it easier to consume. We’ll go over each concept around Kubernetes Gateway API in detail in the coming days.