Data
Orchestrating Flink on Kubernetes — Deployment Methodology, Metrics, and Logging Strategies
Our design approach involved setting up the Flink on Kubernetes operator. We also integrated cert-manager to manage webhooks for the
Mirror Maker Implementation: Our Journey
In our quest to implement a robust and efficient Mirror Maker service, we find ourselves at a crossroads where architecture,
Seamless Topic Migration: A Step-by-Step Guide from Confluent Kafka to MSK
As we explored options for migrating our Kafka clusters from Confluent to MSK, we discovered two potential solutions: MirrorMaker and
Publishing Artifacts to Maven Central using GitHub Actions: A Step-by-Step Guide
In this blog, we will guide you through the process of publishing your internal artifact to Maven Central. We will
CI/CD
Continuous Integration in AWS Using Jenkins Pipelines: Best Practices and Strategies
Learn about implementing CI using Jenkins, a popular automation tool, and how this approach can optimize and streamline your software
Slack for CI/CD Pipelines — Using Groovy
Discover how to leverage the power of Slack integration in Jenkins pipelines to deploy intricate pipelines, facilitate team coordination, and
Harness the Power of Jenkins Shared Libraries
With the growing adoption of the Jenkins Pipeline in organizations, common patterns emerge, necessitating the sharing of reusable components to
Revolutionize Your DevOps Workflow with Continuous Deployment Pipelines
Unlock the full potential of your CI pipeline with powerful continuous deployment capabilities Deploying software releases quickly and reliably is
Accelerate Development with Jenkins Pipelines and Continuous Integration
Developing and releasing new software versions is an ongoing process that demands careful attention to detail. It’s essential to have
Creating Customized Kubernetes Pod Templates using Groovy in Jenkins Pipeline
DevOps can modularize processes across the entire project from development to testing and deployment, including configuration changes, quality checks or
Infrastructure
Unlock the Power of Terragrunt’s Hierarchy
In this article, readers will learn about Terragrunt (associated with Terraform) and how to unlock the power of Terragrunt’s hierarchy,
Provisioning EKS Cluster With Terragrunt — Part 2
In part one of this blog post, we detailed the VPC structuring in a cloud environment, we broke down the
Structuring VPC By Using Terragrunt — Part I
Terragrunt is a thin wrapper that provides extra tools for keeping your configurations DRY, working with multiple Terraform modules, and
Monitoring
Integrating Blackbox Exporter in Datadog, Prometheus, and Grafana for Custom Metrics
In this article, we will explore the process of utilizing the Blackbox Exporter to extract metrics from endpoints, as well
Autoscaling your Airflow using DataDog External Metrics
Follow this step-by-step guide and learn how to autoscale your Airflow K8s pods based on ExternalMetrics One of the strengths
Monitoring Confluent Cloud Kafka with Datadog — Natural Intelligence
When I came across Apache Kafka and its concept of a streaming platform I asked myself — how will I monitor it?
Data\ML
Three Approaches To Scaling Up Your Workload With Airflow
While exploring options for executing complex workflows that contain multiple tasks, we came across Airflow. It proved to be a
Leveraging OpenMetadata and Airflow in the Data Mesh Era
As organizations increasingly embrace the data mesh paradigm, similar to the way Airflow is leveraged for orchestrating ETL flows and
ML Orchestration Made Easy with Apache Liminal and Amazon MWAA
by Lior Schachter & Lidor Ettinger The challenges involved in operationalizing machine learning models are one of the main reasons
DevOps
Deploying Prometheus and Grafana as Applications Using ArgoCD — Including Dashboards
Goodbye to the headaches of manual infrastructure management, and hello to a more efficient and scalable approach with ArgoCD. If
Microservices
First Steps to Microservices — Natural Intelligence
At Natural Intelligence we are running a Kubernetes cluster on AWS which makes it is easy to deploy, manage and
AWS
Configuring Atlantis on AWS ECS with Custom Metrics Integration using Datadog
In our R&D team, as developers, we keep our source code stored on GitHub or Bitbucket. When we want to
Deploying a Node.js Application using Docker: Three Effective Ways, including AWS ECS Deployment
In this post, we’ll guide you through different methods of deploying a Node.js application and share the considerations we made
Using AWS SSM As A Unified Credentials Storage
At Natural Intelligence, we have plenty of dockerized services that run on kubernetes and swarm cluster on AWS. Each service
Latest Posts
Unlock the Power of Terragrunt’s Hierarchy
In this article, readers will learn about Terragrunt (associated with Terraform) and how to unlock the power of Terragrunt’s hierarchy,
Continuous Integration in AWS Using Jenkins Pipelines: Best Practices and Strategies
Learn about implementing CI using Jenkins, a popular automation tool, and how this approach can optimize and streamline your software
Integrating Blackbox Exporter in Datadog, Prometheus, and Grafana for Custom Metrics
In this article, we will explore the process of utilizing the Blackbox Exporter to extract metrics from endpoints, as well
Deploying Prometheus and Grafana as Applications Using ArgoCD — Including Dashboards
Goodbye to the headaches of manual infrastructure management, and hello to a more efficient and scalable approach with ArgoCD. If
Orchestrating Flink on Kubernetes — Deployment Methodology, Metrics, and Logging Strategies
Our design approach involved setting up the Flink on Kubernetes operator. We also integrated cert-manager to manage webhooks for the
Configuring Atlantis on AWS ECS with Custom Metrics Integration using Datadog
In our R&D team, as developers, we keep our source code stored on GitHub or Bitbucket. When we want to
Mirror Maker Implementation: Our Journey
In our quest to implement a robust and efficient Mirror Maker service, we find ourselves at a crossroads where architecture,
Seamless Topic Migration: A Step-by-Step Guide from Confluent Kafka to MSK
As we explored options for migrating our Kafka clusters from Confluent to MSK, we discovered two potential solutions: MirrorMaker and
Publishing Artifacts to Maven Central using GitHub Actions: A Step-by-Step Guide
In this blog, we will guide you through the process of publishing your internal artifact to Maven Central. We will
Three Approaches To Scaling Up Your Workload With Airflow
While exploring options for executing complex workflows that contain multiple tasks, we came across Airflow. It proved to be a
Leveraging OpenMetadata and Airflow in the Data Mesh Era
As organizations increasingly embrace the data mesh paradigm, similar to the way Airflow is leveraged for orchestrating ETL flows and
Slack for CI/CD Pipelines — Using Groovy
Discover how to leverage the power of Slack integration in Jenkins pipelines to deploy intricate pipelines, facilitate team coordination, and
