How to Fix the Kubernetes Node NotReady Error

Kubernetes is an environment that works with nodes, clusters, pods, etc. on a physical or virtual machine depending on the cluster setup. One or more nodes that are typically managed by the Kubernetes control plane make up a cluster. A node is a significant component of a Kubernetes cluster to run the pods. Pods run on nodes or clusters to execute a command or run an application in Kubernetes. If the node is not available to run a pod or an application in Kubernetes, that node is considered in a NotReady state. In this article, we will explore when and how a node enters the NoteReady state and how to fix that state so that the pods can run on it.

What Is Node NotReady State in Kubernetes?

A node in the Kubernetes environment is a virtual machine that makes up a cluster to run the pods. The nodes must function properly because they perform important tasks. A cluster consists of multiple nodes, and each node has its state. A node enters the NotReady state if it crashes or is terminated. The Node NotReady state is a state of the node when a node is unable to run the pods on it. All stateful pods which are already running on a node and that node gets into the NotReady state, all the pods become unavailable.

As discussed previously, a cluster is made up of one or more nodes and nodes are used to run the pods. Whenever a pod is scheduled to be run on a node, Kubernetes review the health condition of the node to check whether it is capable of running the pods or not. You can use the following command to acquire a list of the nodes that manage a cluster:

This gives you all the nodes which are currently running in a cluster with their associated properties like name, status, roles, age, version, etc. See the following given output of the command:

Here, you can see the different statuses of the nodes. The “my-clsuter-m03” is in the NotReady state while the others are in the Ready state. There could be different reasons why this node is in a NotReady state. We can find out that reason by debugging the node. It is essential to debug a NotReady node and know the reason so that the issue can be resolved, and the node does not remain unused.

Why Does a Node Gets into the Node NotReady State?

There can be several reasons for a node to enter the Node NotReady state. Some of them are as follows:

• The network on the node is not configured properly or it is unable to connect to the internet.
• The Kubectl command line tool is not responding or having some problems.
• Lack of resources or unavailability of essential resources for the node. Sufficient memory, disk space, and processing ability are required for a node to be able to run properly. The node enters a state called “NotReady” if any of these resources are not accessible.
• A kube-proxy error like a network agent on the node. The network rules are not maintained, or the kube-proxy stops or crashes.
• Problems with specific vendors who are responsible for running the nodes.

Those nodes which are in the NotReady state are not used in the cluster and they accumulate the costs without partaking in running the pods while negatively impacting the production workload. As soon as you know that a node is in a NotReady state, debug it quickly so that it does not remain idle for so long.

How to Resolve the Node NotReady Problem?

The simplest and quickest solution to resolve the Node NotReady error is debugging or troubleshooting. The following are the steps that you can follow to troubleshoot a node:

Step 1: Start the Minikube

Minikube cluster must be in an active state so that you can run your applications or commands in it. To ensure that it is up and running properly, use the following command:

Step 2: Display All Nodes in the Cluster

To know which node is in the NotReady state, display all the nodes which are available in the cluster with the help of the following command:

Now, this command only gives a summary of the properties of the nodes which are available in the cluster. If you need a detailed description of the associated properties of a node, you can use the following command:

You can see the difference now that you have a detailed description of the properties of the available nodes. This helps you know more about the state of a node in the cluster. You can easily know the reason behind the node being in a NotReady state. It lets you solve the issue easily and in a quick manner.

Step 3: Check the Events

The events in the Kubernetes environment refer to any changes that happen with containers, nodes, or pods. These events give you a complete picture of what is going on in the Kubernetes cluster. Hence, when you check these events, you get to know the reason behind a node in a NotReady state. Now, enter the following command on your terminal to get all the events that happen in the Kubernetes cluster:

The events in the Kubernetes realm are the breadcrumbs that provide valuable insights into how the Kubernetes cluster is running. They also provide a detailed context for the troubling behaviors. Let us check the output which is given in the following snapshot:

From the given output, you can see all the details of the events or in simple words, all the work which are performed in the cluster so far. Each change is recorded in the form of a log, giving a complete picture of every change.

Conclusion

This document is about what a Node NotReady state is and the reasons for a node to enter the NotReady state. Kubernetes is a virtual or physical environment that is made up of clusters that run one or more nodes. These nodes have different statuses like ready, NotReady, SchedulingDisabled, etc. and they are used to run the pods. We explored how to resolve the node NotReady state using the kubectl command in the minikube cluster.