Randy Bias chronicles the history of the term stating that it probably originated in 2011 or 2012 when Bill Baker used the analogy when describing "scale-up" vs. "scale-out" architectural strategies. Bias adopted this into his presentations about cloud architectural patterns:
In the old way of doing things, we treat our servers like pets, for example Bob the mail server. If Bob goes down, it’s all hands on deck. The CEO can’t get his email and it’s the end of the world. In the new way, servers are numbered, like cattle in a herd. For example, www001 to www100. When one server goes down, it’s taken out back, shot, and replaced on the line.
Bias continues to define Pets as
Servers or server pairs that are treated as indispensable or unique systems that can never be down. Typically they are manually built, managed, and “hand fed”. Examples include mainframes, solitary servers, HA loadbalancers/firewalls (active/active or active/passive), database systems designed as master/slave (active/passive), and so on.
and cattle as
Arrays of more than two servers, that are built using automated tools, and are designed for failure, where no one, two, or even three servers are irreplaceable. Typically, during failure events no human intervention is required as the array exhibits attributes of “routing around failures” by restarting failed servers or replicating data through strategies like triple replication or erasure coding. Examples include web server arrays, multi-master datastores such as Cassandra clusters, multiple racks of gear put together in clusters, and just about anything that is load-balanced and multi-master.
Fundamentally, what Bias and Baker are trying to convey is there has to be a transition from how we treat servers from being "Unique Snowflakes" with names and emotional attachments, to a model whereby if we have a problem with the server we create a replacement and destroy the problematic server.
Finally, it is probably worth mentioning that in regulated environments taking a server out the back and shooting it may not be optimal. In these cases it is often advantageous to "freeze" the server, for example using
docker pause to freeze a container. This can then be used to perform a Root Cause Analysis as part of the Incident or Problem Management Process.
To add to Richards answer, generally the analogy is helpful in terms of considering the impact of the loss of a server.
If you would feel some sort of distress over the loss of any individual piece of infrastructure, then consider it a pet (read antipattern).
If you would feel pretty comfortable knowing that if any of the fleet stopped functioning there would be no real impact to operations, then you are talking about cattle.
It's often tempting to use the analogy to simply classify your servers, ie "our workload nodes are cattle but our load balancers are pets" but falling into that trap is exactly the problem. There is no place for pets in a modern computing environment (ie. In the cloud, on commodity hardware etc.) If all of your servers are considered cattle, and are easily replaceable, then you can start looking at things like chaos monkey to help build assurance that your infrastructure truly is resilient.