Mammalian spatial and temporal activity patterns can vary depending on foraging behavior or the perception of predation or competition risk among species. These behaviors may in turn be altered by human influences such as defaunation. Herein, we evaluate whether frugivores avoid areas with high visitation rates by potential predators or competitors, and whether this avoidance changes in areas with different degrees of defaunation. We installed 189 cameras under fruit trees in six areas of the Atlantic Forest, Brazil, that differ in the abundance of top predators and large frugivores. Small predators and small frugivores were more frequent at night while large frugivores were more frequent during the day, but small frugivores visited and spent less time at fruiting trees on brighter nights, unlike large predators and large frugivores. Small frugivores also were less frequent in areas with high visitation by large frugivores and more frequent in highly defaunated areas. Our results suggest that the dynamics among mammalian functional groups varied according to diel patterns, potential competitors, and defaunation. We highlight the importance of understanding how species interactions are changing in areas exposed to strong human impacts to mitigate the indirect effects of defaunation.