The traditional understanding of coyote evolution is that coyotes are basal wolf-like canids. This understanding comes from the hypothesis that coyotes directly evolved from Canis lepophagus in North America alone. Coyotes look and behave a lot like jackals of the Old World, and because we know that the larger wolf-like canids evolved from jackal-like ones, we just assumed that the coyote was a primitive form.
One problem with this positioning has always bothered me. Jackals tend to have proportionally smaller brains than wolves, but coyotes have proportionally larger brains than wolves. Domestic dogs have evolved smaller brains from wolves, although wolf and dog brain size comparisons aren’t as cut and dry as people think.
No one thinks of dogs as basal forms of Canis, so it is possible for animals in this lineage to lose brain size, just as it is possible for a primitive lineage of canids known as coyotes to evolve a larger brain.
Please note that my discussion on brain size here isn’t really a discussion about intelligence, because the literature on which form is most intelligent is quite all over the map. Domestic dogs kept in Western countries in the modern way do appear to have social cognitive abilities that virtually all other species lack, while wolves are much better at working with each other to complete tasks.
But coyotes have proportionally larger brains than either wolves or dog do, and in this lineage, larger brains are generally a derived characteristic.
However, the really important data about coyote evolution is the discovery that they shared a common ancestor with gray wolves much more recently than commonly suggested. A genome-comparison study of various North American canids found that the common ancestor of both gray wolves and coyotes lived around 50,000 years ago. Because anatomically modern gray wolves replace the Mosbach wolf in the fossil record between 300,000 and 500,000 years ago, the ancestor of both had to have been a form of gray wolf from Eurasia.
The coyote is thus a jackal that has evolved in parallel out of the gray wolf lineage, which means it is not a primitive canid at all. It likely evolved this jackal -like morphology and behavior because the form of gray wolf that it derives from was unable to compete with the dire wolf, the American lion, the short-faced bear and the machairodonts as a top-level predator. It was forced to evolve a smaller body that could be fed on carrion and small prey.
We know now that there is a big difference in what prey predators target once they exceed 20 kg. Predators that weigh more than that mass target large vertebrates, while those that are smaller than that weight target smaller prey. Although coyotes do cooperatively hunt deer, they primarily feed on rabbits and mice. So by becoming smaller, coyotes were not directly in conflict with dire wolves or the other large predators of Pleistocene North America.
Only through analyzing full genomes of coyotes and gray wolves did we realize that our assumptions about their evolution were wrong. Earlier studies that looked at mitochondrial DNA alone found that coyotes fit within a basal position of the wolf-like canid lineage. However, recent full genome comparison of various wolf-like canids that looked at the role hybridization played in their evolution found something interesting. The lineage that leads to wolves, dogs, and coyotes experienced some introgression from a ghost species that was closely related to the dhole. The authors think that the reason why coyotes turn up so basal in these mitochondrial DNA studies but appear so wolf-like when their full genomes are compared is coyotes have retained a mitochondrial line that comes from that ghost species.
So the generalist coyote is a re-invention out of the gray wolf lineage. It is not basal to the wolf-like canids. It just merely resembles the basal forms in some of their ecology, in some of their behavior, and in their odd mitochondrial inheritance.
