{"id":279,"date":"2022-05-30T15:33:51","date_gmt":"2022-05-30T10:03:51","guid":{"rendered":"https:\/\/www.cloud-kinetics.com\/?p=279"},"modified":"2024-05-06T15:42:30","modified_gmt":"2024-05-06T10:12:30","slug":"enabling-ci-cd-pipeline-for-container-based-workloads","status":"publish","type":"post","link":"https:\/\/www.cloud-kinetics.com\/blog\/enabling-ci-cd-pipeline-for-container-based-workloads\/","title":{"rendered":"CI\/CD Pipeline For <br>Container-Based Workloads: <br>A DevOps Strategy"},"content":{"rendered":"<p>Continuous integration and continuous deployment (CI\/CD) pipeline is a strategy that helps integrate the work of many people, quickly and precisely, into one cohesive product. It includes the series of steps that need to be performed to deliver a new version of software. It is focused on enhancing software delivery via automation throughout the software development lifecycle (SDLC). By automating CI\/CD during development, production, monitoring and testing, higher quality code can be developed faster.<\/p>\n<p><span style=\"font-weight: 400;\">True, every step of a CI\/CD pipeline can be executed manually, but it is automation that shows its true value. Meanwhile, pipelines are predefined tasks that decide what needs to be completed and when. Tasks are usually executed in parallel to accelerate delivery. A typical CI\/CD pipeline includes stages where code is pushed to the repository and stored, code changes trigger the build, which is tested and then deployed to the production environment.<\/span><\/p>\n<h2><b>Enabling CI\/CD pipeline for container-based workloads<\/b><\/h2>\n<ul>\n<li><b>CI\/CD, a DevOps strategy: <\/b>CI\/CD is a DevOps tactic, in fact it is the backbone of the DevOps methodology, which brings together developers and IT operations teams to deploy software. CI\/CD facilitates DevOps teams with a single repository to keep automation tools and store work so that the code can be continuously integrated and tested for quality.<\/li>\n<li><b>Containerization, a DevOps tool: <\/b>In containerization, all the components of an application &#8211; the software, its environment, dependencies and configuration &#8211; are bundled into a single isolated unit called a container. Each unit can be deployed in its own space on a shared operating system, on any computing environment, on-premise or on the cloud. Containers are lightweight and portable, and very conducive to automation. Containers and orchestration tools facilitate CI and CD.<\/li>\n<li><b>Docker, a containerization solution: <\/b>Docker is a containerization solution used widely in DevOps and workflows. It is an open source platform that allows developers to quickly and easily build, deploy, update, run and manage containers. Docker makes it easy to decouple apps from their surroundings and it also contains a collection of container images that can be used for development.<\/li>\n<\/ul>\n<h3><b>Common use cases for containerization workloads<\/b><\/h3>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Modernizing legacy application development practices to container-based platforms<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Moving pipelines and workflows across multiple microservices and applications with ease<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Providing DevOps support for CI\/CD<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">DevOps enables compliance with industry standards and organizational policies while shipping releases faster to production.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Minimizing errors during the build, deploy, test, and release process of a new software release<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Providing easier deployment of repetitive tasks.\u00a0<\/span><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<h3><b>CI\/CD pipeline architecture<\/b><\/h3>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-full wp-image-287\" src=\"https:\/\/www.cloud-kinetics.com\/wp-content\/uploads\/2023\/06\/image3-1.png\" alt=\"CI\/CD Pipeline Architecture\" width=\"974\" height=\"456\" \/><\/p>\n<h3><strong>DevOps with containers: The workflow<\/strong><\/h3>\n<ol>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">After coding, developers push the code to a shared repository such as GitHub. Frequently merging the code and validating it is one way to ensure CI is error-free. To start the process, a GitHub webhook triggers a Jenkins project build. When code changes are made and committed to the repository, the pipeline gets activated. It downloads the code and triggers a build process.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">In this step, the code is compiled, artifacts are built, dependencies are sorted out and stored in the repository. Environments are created, containers are built and images are stored for roll out. This is followed by the testing processes. The Jenkins build job uses a dynamic build agent in AWS Elastic Kubernetes Service (<a href=\"https:\/\/www.cloud-kinetics.com\/competencies\/driving-container-orchestration-with-kubernetes\/\" target=\"_blank\" rel=\"noopener\">EKS<\/a>) to perform a container build process.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">A\u00a0container image is created\u00a0from the code in source control and is then pushed to an AWS\/Docker Container Registry.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Using the process of CD,\u00a0Jenkins deploys an updated container image to the <a href=\"https:\/\/www.cloud-kinetics.com\/competencies\/driving-container-orchestration-with-kubernetes\/\" target=\"_blank\" rel=\"noopener\">Kubernetes cluster<\/a>.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">The web application uses Dynamo DB as its back end. Both Dynamo DB and AWS EKS report metrics to the AWS Monitor.<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">A Grafana instance provides visual dashboards of the application performance based on the data from AWS Monitor.<\/span><\/li>\n<\/ol>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-288 size-full\" src=\"https:\/\/www.cloud-kinetics.com\/wp-content\/uploads\/2023\/06\/image1.jpg\" alt=\"CI\/CD Pipeline Architecture\" width=\"946\" height=\"482\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone wp-image-289 size-full\" src=\"https:\/\/www.cloud-kinetics.com\/wp-content\/uploads\/2023\/06\/image2-1.jpg\" alt=\"CI\/CD Pipeline Architecture\" width=\"1016\" height=\"537\" \/><\/p>\n<h2><b>Containerization infrastructure and configuration as code<\/b><\/h2>\n<p><span style=\"font-weight: 400;\">The true power of containers becomes visible when orchestrating with Kubernetes and DevOps pipelines can be automated in better ways. Kubernetes is a portable open source platform used to manage containerized workloads and services. It facilitates both automation and declarative configuration. YAML, a data-serialization language frequently used for writing configuration files, is utilised in Kubernetes deployments and resources. Its advantage is that YAML files can be created and stored in a Git repository and all changes can be tracked and audited.\u00a0<\/span><\/p>\n<ul>\n<li><b>Continuous deployment pipeline with no downtime: <\/b>The objective of the pipeline is to perform a set of tasks that will deploy a fully tested and functional service or application to production. The need for frequent deployments is handled best by Kubernetes via its container orchestration mechanism.<\/li>\n<li><b>Easy rollbacks: <\/b>The Kubernetes framework has\u00a0 a built-in rollback mechanism. When new code is ready to be pushed to a container, the new desired state is defined, and Kubernetes orchestrates creating new containers and removing existing ones.\u00a0If a problem arises, the immutable nature of Kubernetes containers allows easy rollbacks to the previous state.<\/li>\n<li><b>On-demand infrastructure:\u00a0<\/b><span style=\"font-weight: 400;\">Kubernetes, through the use of the configurations, can easily scale infrastructure up and down based on the resources needed to handle the workloads of the application. And it is elastic by nature.<\/span><\/li>\n<li><b style=\"font-family: inherit; font-size: 20px; font-style: inherit;\">Run everywhere pipelines: <\/b>With Kubernetes architecture, we can easily migrate\u00a0<a href=\"https:\/\/www.cloud-kinetics.com\/competencies\/driving-container-orchestration-with-kubernetes\/\">Containers and pipelines<\/a><span style=\"font-weight: 400;\">\u00a0to anywhere in the same cloud or all on-premises.<\/span><\/li>\n<\/ul>\n<h3><b>Containerization features<\/b><\/h3>\n<ul>\n<li><b>Availability: <\/b>Amazon Elastic Kubernetes Service\u00a0(EKS)\u00a0<span style=\"font-weight: 400;\"> operates and scales the Kubernetes control plane across many AWS availability zones to offer high availability. <\/span><span style=\"font-weight: 400;\">As part of the Amazon Kubernetes Service cluster, application traffic is distributed to one or more containers (pods) that run the application as individual microservices. This approach to running containerized applications in Kubernetes provides a highly available infrastructure for the applications. <\/span><\/li>\n<li><strong>Scalability:<\/strong> Amazon EKS makes it easy to run Kubernetes on AWS and on-premises. It automatically allows scaling of the number of cluster\u2019s worker nodes to meet the application\u2019s workload demands. As the application size increases, the EKS cluster can scale up the number of Kubernetes nodes.<\/li>\n<li><strong>Resiliency:<\/strong> Amazon EKS is built into the Kubernetes architecture and its components are resilient by nature. Kubernetes components monitor and restart the containers (pods) if there is any issue. Combined with running multiple Kubernetes nodes, applications can tolerate a pod or node being unavailable.<\/li>\n<\/ul>\n<h3><b>Security and security threats in containers<\/b><\/h3>\n<p><span style=\"font-weight: 400;\"><a href=\"https:\/\/www.cloud-kinetics.com\/services\/applications-software-engineering\/containerization\/\" target=\"_blank\" rel=\"noopener\">Container security<\/a> is an important part of a complete security assessment. It involves the practice of protecting the containerized environment and applications from potential risks and threats by implementing a combination of security policies and tools.<\/span><\/p>\n<ul>\n<li><b>Access and authorization exploits:\u00a0<\/b><span style=\"font-weight: 400;\">Providing access to authorized users and blocking all other users accessing the platform. And encrypting K8\u2019s configuration files (for example, web. config and appsettings.json), particularly in a containerized setup.<\/span><\/li>\n<li><strong>Container image vulnerabilities:<\/strong> Security mechanism to prevent malicious attacks is the key.<\/li>\n<\/ul>\n<p><span style=\"font-weight: 400;\">Detecting code vulnerabilities, outdated packages, malicious code, and other harmful threats during the build stage can improve security dramatically.\u00a0<\/span><\/p>\n<h3><b>Monitoring CI\/CD pipelines, end-to-end<\/b><\/h3>\n<ul>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Monitor health of the CI\/CD build pipeline and set up cognitive, proactive alerts spanning various tools<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Assess performance and quality of deployments in a unified way across multiple tools<\/span><\/li>\n<li style=\"font-weight: 400;\" aria-level=\"1\"><span style=\"font-weight: 400;\">Monitoring the pipeline performance and reporting issues combines Amazon Monitoring Service (CloudWatch) with Grafana for visual dashboards; or extending build pipeline monitoring to include application monitoring (Nagios) and container monitoring (<\/span><a href=\"https:\/\/kubernetes.io\/\" target=\"_blank\" rel=\"noopener\"><span style=\"font-weight: 400;\">Kubernetes<\/span><\/a><span style=\"font-weight: 400;\">).<\/span><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Continuous integration and continuous deployment (CI\/CD) pipeline is a strategy that helps integrate the work of many people, quickly and precisely, into one cohesive product. It includes the series of steps that need to be performed to deliver a new version of software. It is focused on enhancing software delivery via automation throughout the software &#8230; <a title=\"CI\/CD Pipeline For Container-Based Workloads: A DevOps Strategy\" class=\"read-more\" href=\"https:\/\/www.cloud-kinetics.com\/blog\/enabling-ci-cd-pipeline-for-container-based-workloads\/\" aria-label=\"More on CI\/CD Pipeline For Container-Based Workloads: A DevOps Strategy\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":1724,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[37,35,38,39],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/posts\/279"}],"collection":[{"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/comments?post=279"}],"version-history":[{"count":6,"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/posts\/279\/revisions"}],"predecessor-version":[{"id":4669,"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/posts\/279\/revisions\/4669"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/media\/1724"}],"wp:attachment":[{"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/media?parent=279"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/categories?post=279"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cloud-kinetics.com\/wp-json\/wp\/v2\/tags?post=279"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}