Like so many other words in English (antique, chief, expert, orange, phony, suspect, etc.), "racist" works as both a noun and an adjective.

He is racist. (adjective)

He is a racist. (noun)

Both have approximately the same nuance, but are used differently. As an adjective "racist" can describe actions, concepts, and objects as well as people. Examples:

A racist decision.

A racist doctrine.

A racist document.

The noun "a racist" can only be applied to people, as in "a person who is racist".

Both are grammatically correct, and the difference is just about what nuance the author wants the sentence to have. They do have slightly different meanings, however. A racist is a person who is racist, which is the state of holding prejudice against a specific demographic. So the sentence "He is racist" is saying "He is prejudiced", whereas the sentence "He is a racist" is saying "He is the type of person who holds prejudices". Both mean approximately the same thing, just going at it from a different angle.

Depending on context, they can imply degrees of prejudice, but this is not intrinsic to the usage. In my experience, being called "a racist" is usually slightly more serious, just because "a racist" usually means they are a wholly racist and hateful person, whereas being called simply "racist" can mean the same thing, or a lesser degree, such as subconsciously fearing a minority even without hateful intention.

You mention entering the phrase into a search. Do you mean a search engine?

If that is the case, then the reason that people often omit articles from internet searches is that they're largely ignored by the search engine itself.

If you type...

He is a racist

...into a search engine, the search engine is going to ignore the 'a' because it appears possibly literally on every website on the internet. In other words searching for...

He is a racist

vs....

He is racist

...will very likely pull up the same search results.

If you add quotation marks, then the engine will search for that exact phrase:

"He is a racist"

...will return all websites where that exact sentence, including the 'a', appears, whereas...

"He is racist"

...will return all websites where that sentence appears.

"Racist" is a word that is both a noun and an adjective. It is both a word to identify a person (he is a teacher, he is a politician, he is a racist), and a property of the person (he is fat, he is tall, he is racist). So you can use both forms, with a slightly different meaning. "He is a racist" means he is one of the group of people called racist. "He is racist" means he is a person with the property of being racist.

Compare to "lazy": Lazy is an adjective, but not a noun. You can say "he is lazy", you can't say "he is a lazy". But compare to "coward": Coward is a noun, but not an adjective. You can say "he is a coward", but you can't say "he is coward".

Now compare to "black". It's also both a noun and an adjective. But depending on where you live, saying "he is a black" will be taken as insulting, while saying "he is black" is not. Both are grammatically correct, but one will get you into trouble with people.

I agree with Andrew and Nathan's answers. You asked why. I think people use a "trick" of argument here.

Many sentences that use "a racist", say "I am not a racist" or "He is not a racist". It is used as a response, when we say something he did was racist and bad.

This is equivalent to saying "I am not a bad person", in response to "Please do not do _, that is a bad thing to do".

Saying "I am not a racist", puts more emphasis on the "I". He tries to say "I am not a racist. Do not say that what I did was racist. Do not dis-respect me."

I think "He is racist" is used because it anticipates this response. It is less important to decide whether Boris Johnson matches some definition of "a racist", or not. It is more important to decide whether or not he is being racist. E.g. writing racist arguments, spreading racist ideas which damage our society.

He does not need to be a committed follower of racist ideology. He need only take advantage of long-standing constructs of racial oppression, to become more popular and gain power.